On the embodiment of unfeasible objects

Further to my last post detailing experiments relating embodiment of invisible and even third hands, here are some notes on my first ‘Clay hand Illusion’ experiments…

The ‘Rubber hand illusion’ shows it is possible to convince participants that a rubber hand is their own by placing it in front of them while stroking it in the same way as their hidden real hand. The use of self-made clay hands, or objects [see below] in place of the rubber hand raises several interesting possibilities for exploration, which move away from the embodiment of replica body parts, and towards the possible embodiment of modified body parts, or completely ‘unfeasible’ objects.tactile_object3.jpgThe clay allows for the gradual and immediate morphing of forms and for the participant to build a sensory connection with the object through its creation. As an artist [who has worked with clay] I feel a sense of deep connection with the objects I make, especially during making them. For example, I feel my face move and contort when I am trying to draw a face. I wonder if this is true of others? This is why for my first participants I am choosing those who work with clay.

In regard to the embodiment of ‘unfeasible’ objects, the possibility of such a thing has been loosely disproven in several studies [See ‘The Invisible Hand Illusion’]. Whereby a plank of wood and a spoon were substituted in place of the embodied hand. Therefore I am keen to explore this further. So far I have had encouraging results which build on the results from the first workshop session [See image below]. Is it easier to achieve a connection with self-made objects, rather than an irrelevant object, such as the plank of wood?
hands03In my first ‘beta’ study, I worked with a participant who is a maker and uses clay in their work. After making a good connection with their self-made clay hand, I asked if they could make a non-hand like an object, or a modification to the clay hand, for a further experiment. They immediately opted to make a roundish blob. Followed by a further iteration; a doughnut shape. They were able to make a strong feeling of connection with the blob and the Doughnut, though not as strong as to the hand, and the connection took longer to achieve. The connection was patchy, in parts, and mapped over the surface.

I found that the fingers could be mapped around the object by using a combination of synchronous tapping and swapping over the embodied finger with other digits, and the moving the already embodied finger over to a new location on the object. For example, the index finger feels fully connected to the clay object but, the ring finger does not. So I tap on the real ring finger and say I’m tapping on the index finger. They then seem to believe that this finger is now connected, when I return to the index finger – they now believe the finger next to the index finger is now newly connected to the clay. It’s not easy to explain! Mor on this later. The main part of the hand did not feel fully embodied.

It seemed like that with time, and combinations of synchronous and non-synchronous tapping, the hand could be mapped in two dimensions over the surface of the object.

I also found an interesting ‘compression’ in the perceived length of the fingers! When I tapped on the knuckle where the finger joins the hand, the participant thought this was the middle of their finger.

Working with the clay has an interesting effect of leaving traces of the tapping and stroking process so over time a textural surface is built up. When experimenting with the clay blob the participant made the following comment…

IMG_1640Unfeasible Object #1 Participant 1

 “I fully believe that’s my hand, but like there’s an obstacle, like you cant push past it…it feels like my hand is made out of this clay, but there is only so far you can push into it…it feels like a barrier”

The barrier seemed to be just under the surface of the clay. Perhaps this suggests a  surface-deep sensation of the perceived embodiment? Another interesting comment…

“Its like when I’m making something, I can feel my hand moulding something, and it feels like I’m on the opposite side of that”

The significance of this comment hit home when my supervisor picked up on the idea of ‘actions’ ie how we might hold an object, or how the object was made. Could embodiment of these objects be stronger if I considered them from the point of view of how they might be handled? This makes me think of ways in which the underside of the hand could be used in the illusion, rather than the back of the hand as is traditionally used. An upside down or inside out version of the Rubber hand Illusion? My experiment continues to evolve…


Illusions of Invisible, alien hands, 3 arms, and shrinking bodies…

I have previously posted about the original ‘rubber hand illusion‘ in which participants are convinced a fake rubber hand is their own. A classic low tech experiment that can help us “understand how sight, touch and “proprioception” the sense of body position, combine to create a convincing feeling of body ownership, one of the foundations of self-consciousness  (Nature 1998, vol 391, p 756 )” Further to this research recent studies have gone on to experimentally induct of out of body experiences or create body swap illusions. [ Henrik Ehrsson and Olaf Blanke http://www.eurekalert.org/pub_releases/2007-08/ucl-foe081407.php ] Between these two experiments, several interesting versions of hand-related experiments also exist which demonstrate the plasticity of our body perception…

The Alien-Hand Experiment (Nielsen 1963;1978 https://link.springer.com/article/10.1007/s11097-005-5854-4]  

The Alien Hand Experiment first created in the 1960s predates the Rubber hand experiment, seems more ‘trick’ than an experiment, but the element of deception is necessary, participants are asked to draw a straight line while viewing their gloved hand in a mirror. Unbeknown to them, they are viewing the experimenter’s hand mimicking their own actions. This could also relate to the Alien hand syndrome [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4059570/ ] a rare condition in which a person believes their hand is controlled by someone else.

[Guterstam, Gentile & Ehrsson. 2013. The Invisible Hand Illusion: Multisensory Integration Leads to the Embodiment of a Discrete Volume of Empty Space. J Cogn Neurosci http://dx.doi.org/10.1162/jocn_a_00393 ]
The Invisible Hand Illusion is a version of the Rubber hand illusion which uses no hand at all as the fake hand, sometimes a handless stump, and as a control, a wooden plank. they found that participants could embody a ‘Discrete Volume of Empty Space’ a concept which personally I find too exciting not to try at the next available opportunity.

[Guterstam, Petkova & Ehrsson. 2011. The Illusion of Owning a Third Arm. PLoS ONE http://dx.doi.org/10.1371/journal.pone.0017208]
This experiment uses a rubber hand placed next to the real hand of the participant. They found participants were able to feel as if they had two hands, as long as they were both the same hand [left or right]

Stelarc ‘Third Hand’

There is obvious resonance here with the work of Stelarc http://stelarc.org/?catID=20265  whos project ‘Third Hand’ as he explains “A mechanical human-like hand that is attached to my right arm as an additional hand. It is made to the dimensions of my real right hand.”

[van der Hoort, B., Guterstam, A., & Ehrsson, H. (2011). Being Barbie: The Size of One’s Own Body Determines the Perceived Size of the World 10.1371/journal.pone.0020195 ]
The project “Being Barbie: The Size of One’s Own Body Determines the Perceived Size of the World” Builds on the set up created for the body swap illusion and plays with body scale with fascinating results.

There are more but will have to wait for another post.

Blind drawing of hand

Blind drawing of hand starting from the left and using a continuous line, before and after experiments.
before-afterCould this exercise be used as a measure of “proprioceptive drift” before and after ‘Rubber hand illusion’ type experiments? I will be exploring this idea soon with the research group at BEAM lab…

Clay hand drawings

Here are are a selection of drawn outlines of clay hands created by workshop participants aged 6 to 12 for the original clay hand experiment (see posts tagged rubber hand or clay hand) Participants of all ages were asked to create a hand and use it in place of a replica rubber hand, the idea was to test if a self-created hand was easier to connect with. The hands were then taken and worked on further, sometimes becoming more distorted and abstract.

Face as Interface

Trying to create a simple motion tracking patch using PD_extended and Gem, I came across this project by Elektro Moon Vision http://elektromoon.co.nr/ the mini App provides OSC data from movements such as eyebrows, nose, mouth, orientation scale etc. This is massively useful for an experiment I have in mind related to the “strange face in the mirror illusion” The data can be captured and used to control a 3D model in virtual space for example. Matching rotation, scale and orientation to the model and the movement of my head…

This is a simple motion detection patch that tracks the difference between two frames creating ghostly outlines of momentarily disembodied features. Its sensitive enough to pick up facial expressions such as the movement of muscles and eyeballs. A combination of these two systems should be enough to develop a system of projecting on to a face and follow its movements.

Screen Shot 2018-01-19 at 12.50.44.png
In this video still, the tracking detects the movement of my left side of the face as I smile on one side, to see if it detects the movement of my mouth as well as cheek muscles. the Red circle detects the centre of mass of the image 


Screen Shot 2018-01-19 at 12.49.53
Sensing motion by measuring differences in live video frames Using Pure Data and GEM


See here for the experiments with the strange face in the mirror illusion so far…

Drawing a strange face in the dark

Shortly after my experience of the “Strange face in the mirror experiment”  I made these drawings in low light conditions as a way of recording the perceivable elements of my face and shape of the head. The particles of carbon and graphite reflect well the visual noise, like static, one experiences in the experiment. These drawings don’t illustrate the hallucinations I experienced [these will follow]

“staring at one’s own reflection in a mirror in a darkened room for some time can induce vivid hallucinations. For purposes of research, I had to try it”  My experience of the ‘strange face illusion’…

Drawing in near darkness
Drawing in near darkness

For my description of my experience of the ‘strange face’, illusion see here… “staring at one’s own reflection in a mirror in a darkened room for some time can induce vivid hallucinations. For purposes of research, I had to try it” 
My experience of the ‘strange face illusion’…



Mixing object actions and context

This research looked at how we respond to the observation of actions and how the context affects our neural processing of that action. I was fascinated to learn simply by observing an action with an object that can be potentially used for an action – such as a potato peeler – the brain seems to simulate this action.

But what happens when we observe an object being used for an action in an unusual context? Or what happens if we see an object being used for an unusual action, perhaps even in an unusual space.

The experiments used videos of a person performing a task with specific objects in specific contexts such as cracking and egg into a bowl in the kitchen. and was then repeated in a different context or with a different tool. such as cracking an egg into a shoe in a garage for example. The experiments yielded interesting results [see links below]

For me, this reminded me of the mechanisms artists use all the time. Mixing of materials that would otherwise rarely come together – or simply mixing contexts actions and labels.

“The influence of contextual information on action observation and anticipation
“In order to properly interact with each other it is of utter importance to understand what people around us are doing: that is, to be able to derive intentions, to predict and anticipate future steps of an unfolding action and hence to capture overarching action goals. As actions are highly complex stimuli, action recognition and more specifically goal inference involve not only the readout of core information (manipulation movements, manipulated objects) but also the integration of contextual information, i.e. the scene (including actor, room and contextual object information) in which an action takes place. The latter may be especially important for the inference of action goals. Hence, information that is not necessary for the recognition of an on-going action (e.g. cutting a carrot) might be crucial for higher-level inferences of goals (preparing a salad) and the prediction of forthcoming action steps (e.g. cutting more vegetables, preparing dressing)”
Nadiya El-Sourani, University of Muenster, Germany

Link to related paper:
Making sense of objects lying around: How contextual objects shape brain activity during action observation