Jaron Lanier recently wrote about virtual reality and its potential application to learning, utilizing some evocative terms and offering an educational scenario that reminds me of a seminal 1997 paper that described how a Nobel prize-winning biologist fused with her objects of study. The Saturday Wall Street Journal article gave me a keener appreciation for the potential applications of virtual reality (VR) – immersive computer-generated environments that model real or imaginary worlds – and for the pervasiveness of object-centered sociality, a concept I first encountered via Jyri Engestrom.
Lanier's article is about new frontiers for avatars – "movable representations of ourselves in cyberspace" – and how they can be used to manifest somatic cognition – the mapping of human body motion "into a theater or thought and strategy not usually available to us" in which one's hands (or presumably, other body parts) can solve complicated puzzles more quickly than one's head (or conscious mind). The examples he gives of somatic cognition outside the realm of virtual reality include professional musicians, athletes, surgeons and pilots, and I found myself thinking of a documentary I saw years ago on heavy machinery, and the way that a crane operator who was interviewed described the bewildering array of levers as virtual extensions of his arms and hands.
After describing a software bug in an early VR system that gave his humanoid avatar a gigantic hand, Lanier generalizes homuncular flexibility as a more general principle: "people can learn to inhabit other bodies not just with oddly shaped limbs [gigantic hands], or limbs attached in unfamiliar places, but even bodies with different numbers of limbs [lobster avatars]". Dean Eckles generalizes this notion even further – in a 2009 blog post reviewing a 2006 article by Lanier on homonucular flexibility (which offers more details about the lobster) – to distal attribution: our propensity for attributing sensory perceptions to internal or external – or proximal or more distant – sources.
However, it is Lanier's reference to an experiment with elementary school children being turned into the things they were studying that I found most interesting [although I have not been able to track down the reference]:
Some [students] were turned into molecules, dancing and squirming to dock with other molecules. In this case the molecule serves the role of the piano, and instead of harmony puzzles, you are learning chemistry. Somatic cognition offers an overwhelming emotional appeal for education, because it leverages vanity. You become the thing you are studying. Your sensory motor loop is modified to incorporate the logic of a science, and you develop body intuition about that logic.
This idea of fusing or becoming one with the object of study is one of the two primary manifestations of object-centered sociality articulated in Karin Knorr Cetina's seminal paper, "Sociality with Objects: Social Relations in Postsocial Knowledge Societies", [Theory, Culture & Society, 1997, Vol. 14(4):1-30]. As I noted in an earlier post on place-centered sociality, the other manifestation of object-centered sociality – sociality (interactions and relationships) through objects, such as online photos, videos or even blog posts – is better known, at least among many of those who study online social media (and mediation). But Lanier's article evokes the manifestation of sociality with objects themselves, reminding me of what I earlier wrote about Knorr Cetina's articulation of how this can promote deeper investigation and learning:
[Knorr Cetina] looks specifically at knowledge objects, and how they are increasingly produced by specialists and experts rather than through a broader form of participatory interpretation. She argues that experts' relationships with knowledge objects can be best characterized by a the notion of lack and a corresponding structure of wanting [emphasis hers] because these objects "seem to have the capacity to unfold indefinitely": new results that add to objects of knowledge have the side effect of opening up new questions. This perpetual unfolding gives rise to "a libidinal dimension or dimension of knowledge activities" – an "arousal" and "deep emotional investment" – by the person studying the knowledge object. As an example, she describes the way that biologist Barbara McClintock, who won the Nobel Prize for her discovery of genetic transposition, would totally immerse herself in her study of plant chromosomes, identifying with the chromosomes and imagining how they might see the world – evoking an image (for me) of object-centered empathy more than sociality.
The prospect of empowering future Nobel laureates with virtual reality technology to engage with and virtually embody objects of knowledge at an early age is very exciting. Lanier mentions the Kinect camera for Xbox 360 made by Microsoft (his employer), which will likely put virtual reality technology in the hands (or homes) of millions of people in the near future.
The primary emphasis of Kinect marketing is on fun and games, but based on Lanier's article, and Knorr Cetina's insights into object-centered learning, Kinect might also provide a platform for a new approach to education. In an ideal world, of course, fun and learning would not be such distinct concepts … perhaps this new technology will help promote a new dimension of convergence in the not-too-distant future.
Comments
3 responses to “Virtual Reality, Somatic Cognition, Homuncular Flexibility and Object-Centered Sociality and Learning”
Usually when we think about distal attribution, incorporation, transparency, homuncular flexibility, the ready-to-hand, etc., this is quite the opposite of a social relationship to an entity. When incorporated, the entity is, in some senses, not even an object, let alone an agent. (See Leila Takayama’s clear & brief exposition http://www.willowgarage.com/sites/default/files/HRI%202009%20-%20Agentic%20Objects.pdf) So, as in your excerpt above, I would not really think of this as object-centered sociality.
But there are some fascinating exceptions, where entities move back and forth between incorporation and agency, or even are both at once. For example, cars both become an extension of the self (e.g., you feel the road, you are the car), but also are attributed personalities and agency.
Nice to see my blog post mentioned in this context. I wonder if Lanier still views this capability as a spandrel…
@Dean: thanks for sharing the link to Leila’s Human-Robotic Interaction 2009 paper, Making Sense of Agentic Objects and Teleoperation: In-the-moment and Reflective Perspectives. I’m not an expert in the areas you mention, but I do think the popularization of the concept of object-centered sociality – especially as it pertains to social media – has narrowed it considerably from its presentation in Knorr-Cetina’s paper, which talks about sociality through objects (e.g., comments on blog posts) but also sociality with objects, which is what I find exciting about the educational possibilities outlined in Lanier’s article.
Both Takayama and Knorr-Cetina reference George Mead’s 1934 book, Mind, Self and Society. Knorr-Cetina invokes Mead’s role-taking formula, entailing “interpersonal reflexivity coming about through an individual taking the attitudes of the other toward him/herself”. She goes on to propose following Mead “halfway” regarding McClintock describing how “she as a subject and scientist partakes in the object world, and how the object world she studies partakes in herself” (the descriptions come from Evelyn Fox Keller’s book, A Feeling for the Organism: The Life and Work of Barbara McClintock). In Knorr-Cetina’s view, “the object of knowledge has now become an internal object”.
Takayama distinguishes in-the-moment vs. reflective attributions of agency. In the case of the chromosomes studied by McClintock, I suspect she would have said that they are agentic (in-the-moment and upon reflection), and that Knorr-Cetina sees the fusion between the scientist and her objects of study as a key enabler for her to make such significant contributions to science. What excites me about Lanier’s work is that virtual reality approximations of this kind of fusion may enable a broader array of students to learn more effectively.
At least my understanding of the chromosome example has the scientist incorporating the chromosome as part of the scientists’ embodiment. Thus, the chromosome is reflectively a (non-agentic) object and in the moment has “negative agency” (it is incorporated, transparent, etc.). But biologists also quite often take the intentional stance towards biological entities, that is treating them as agents and explaining and predicting their behavior with belief-desire “folk” psychology.
Of course, I don’t think it is as simple as reflective vs. in-the-moment (this is best as a heuristic). Rather, we can think about different psychological processes (of which people are aware of to varying degrees) that correspond to relating to entities as having negative agency (incorporated), no agency, or agency.