A conversation with Garry Stewart

The following is an extensive email conversation, which happened over a period of time between Garry Stewart and Ian Gibbins during the preparation of ‘Proximity‘. It will give you an idea of the depth of research that Garry delves into for his work on any particular subject. It really shows that contemporary dance can be the thinking person’s art form as well as satisfying the draw of physical expression and virtuosity. 

Q. I’m wondering if you could say that ‘when we see we do it with our entire body, not just our eyes and brain’.

A. I think the purists would not like to say “see” but certainly “perceive” would be well accepted. There is no doubt that how we visually perceive the world around us is influenced by our prior knowledge and / or expectations of the things we see. In terms of the “entire body”, something that is so intrinsic to us it is generally taken for granted, is that we perceive the world and its objects necessarily adjusted for our body scale: how big we are compared to everything else. This determines how we interact with things at various distances from us, how we judge the weight of objects, how feasible it would be to manipulate them (their “heft” to use a nice poetic term…), etc.

When you “not just our eyes and brain”, I think you need to be careful there… no problem with “not just our eyes…”, but all perception, experience, knowledge etc necessarily comes to our awareness via our brain: everything we know about our body we experience with our brain, even if it seems so intrinsic that it doesn’t feel like conscious experience… This is totally incontrovertible for neuroscientists / cognitive scientists, but some philosophers still don’t like it and try to argue otherwise. As far as I’m concerned they can’t win that one… there is a growing group of “cognitive philosophers” who explicitly accept this position.

Q. Do you think this is true from the point of view that our brain creates a sense of our entire bodies position in space and relationship/point of view to the objects within our visual field?

A. Yes… a big chunk of our brain is doing just this job (the parietal cortex, which sits between the visual cortex and the sensory-motor areas)

Q. Does my body take on a sort of global ‘attitude’ in the act of seeing and perceiving?

A. Yes… but it’s even more interesting than that. The “attitude” can be interpreted in different ways, all of which can operate at the same time. In terms of spatial maps, we can orient to the world using body-based coordinates (“egocentric”): ie everything is referred to our own body, as in left and right, for example. However we also map in external, world-based coordinated (“allocentric”): eg objects referred to absolute directions such as north, south, east, west, etc…  there are subsets of both of these coordinate systems that relate to the distances involved (near, eg within reach; far, known only by integrating a virtual map… and several intermediate levels) and to the landmarks we use to navigate these spaces.

The other way you could interpret “attitude” is that overlain on all the spatial processing is what we choose to attend to. Attention is driven from several areas of the brain, but mostly form the prefrontal cortex. We cannot perceive everything that is going on in the outside world or in our bodies, let alone both, equally well at the same time. So brain selects what we will attend to = what we  focus upon = what we are consciously aware of at any point in time… This process seriously impacts on what we perceive. There are all the famous (notorious?) examples of people missing really “obvious” events because they attend to something else (most well known is the example where you are asked to watch a video of two teams playing basketball and you are to keep track of the number of passes each team makes; while you are doing this a bloke in a gorilla suit runs into the frame, waves his hands, and runs out… most people report not seeing him at all until he is pointed out… and then they cannot believe they missed it!!!).

Q. Through this, it seems our bodies are sort of in a state of readiness. Readiness to act, or not act. We seem to move from one state to another depending on how we are feeling and what is going on externally and internally.

A. Yes. An increasingly common concept is that the brain is basically working as a prediction machine. In one sense, we are always ready to do something… exactly what that is depends on what ever else is going on modulated by what out focus of attention is and our current plans, whatever they may be. If we are doing something actively, what our readiness is actually directed to usually depends on what has just happened, ie it is modified by past experience.

Q. So are we perceiving our situation in space and 3 dimensions corporeally? With our whole bodies? It seems like when we see our entire body goes into the state of this seeing and hence adopting an ‘attitude’ toward what we are seeing.

A. I think so. In this sense, I would again replace “see” with “perceive”. The sense of “our whole body” comes from different types of sensory nerves in the muscles, tendons, ligaments, bones, skin, etc that record all kinds of things like muscle tension, muscle length, touch, texture, temperature, and so on. The amazing thing is that our conscious perception of all this feels like a continuous integrated sensation (most of the time). One of the things that has driven more conventional philosophers nuts is that you simply cannot unravel all this by introspection. Our “body sense” is such a derived construction, it’s impossible to identify all its components without doing some pretty sophisticated experiments.

Q. Perhaps I’m encroaching on philosophy here?

A. See above: depends how you define philosophy!!

Q. My understanding of how we see is that it is a form of trained blindness. We are born with an over supply of neural connections for seeing but these hypertrophise as we learn how to see. So the number of neural connections for seeing diminishes throughout early childhood as we become more and more specific in how we perceive through seeing. Is this correct? We ‘sculpt’ an oversupply of nerve connections in our visual cortext to shape a more specific network of the connection.

Is it also correct to say that when we see as adults, we draw on our history of cognitively conditioned seeing (that is culturally prescribed) so that our ability to see is not just a mechanical function of the eyes/brain but also a composite of this along with our learnt cognition for seeing?

i.e. we have already learnt about depth, colour, shading, perspective etc so our conditioned version of this informs how we see as adults.

A. don’t think that idea of ‘trained blindness’ is really correct, though I can easily see where why you would conclude that!

It is true that we are born with more connections than we need, not just for the visual system, but for most other parts of the brain too. The issue here is that these connections are sort of a default pre-wiring that ensure all the different brain regions are connected up to each other at all.

After birth, sensory experience fine tunes these connections to make sense of the sensory-motor world within in which we find ourselves. The best example of this is language processing: by the time we are about 6 months old, the language we have been hearing up until that time strongly biases the way we perceive the language sounds we hear.

The visual system rapidly becomes “trained” to recognise all the elements of a visual field – edges, textures, colours, shading, etc – and synthesise them into recognisable objects. People who are born with damaged eyes but an intact brain (eg congenital cataracts) usually are unable to see if the eye problem is fixed later in life. This ‘training’ is a combination of fine tuning overall wiring between different regions of visual cortex, as well as regulation of synaptic density between specific sets of neurons in those pathways. So in this sense, your idea of ‘sculpting’ is correct.

All of the low level processing of information in the visual system takes place below consciousness. Some of it actually occurs in the eye itself (in the retina) before the information is sent to visual cortex. So the retina can process relative contrast, detect edges of different orientation, for example.

Visual illusions are usually set up to muck around with these low level processes, either in the retina or in the visual cortex. By introspection, we cannot tell the difference. Also even knowing that the image is an illusion, we cannot consciously over-ride it. As our visual system develops, we also develop an incredibly rich underpinning of memory that allows us to recognise, name, manipulate the things we see. This corresponds to your ‘learnt cognition of seeing’: this refers to understanding what we see, but not the act of seeing itself…

[people with strokes or other forms of selective brain injury have shown how fragile these processes are and how they can be broken down and dissociated. So someone may have cortical blindness, in which they actually can see, but they don’t know that they can… ]

It really isn’t clear how much vision is culturally prescribed. However, it’s nothing like language, for example. I think most vision scientists would say that everyone with normal vision processes a visual scene in pretty much the same way in terms of generating the image itself. But then it’s how you interpret it. For example, if I gave you one of our microscopic images, it’s unlikely you’d know  what it represents, what scale it is, what the colours mean etc, without me telling you. Conversely, when you watch a dance piece, you’d be picking up on all sorts of visual cues that I would miss.

The other thing that feeds into a cultural modification of the perception of vision is attention. We cannot focus on the whole visual scene at once. In fact we can only attend to a very small part of it. Exactly what we attend to is a mix of our own motivation, our training or cultural background, and the circumstances of the moment. For example, if we are out in the bush with an Aboriginal bloke who’s grown up there. all sorts of things would attract his visual attention which may entirely escape our notice, even though we’d experienced more or less the same raw visual inputs.

Q. Do the saccadic movements of the eye take ‘pictures’ that are then sutured together to create a whole image? Is that an acceptable way of describing it. 

If you had to describe saccades and how they form a total image in just one or two pithy sentences what would you say? 

A. That’s pretty much right Garry.

The saccades are rapid movements of the eyes that scan the scene before us. Each saccade takes about 1/20th of a second. The movements are controlled by several areas in the brain: one small part of the cortex generates the movement itself, another relates each movement to the spatial coordinates extracted from visual cues. Other areas control the speed and magnitude of the eye movements. Yet other areas (in the frontal cortex) direct the saccades to the locations that we wish to attend to.

During the saccades, the brain is effectively blind, ie. there is no visual processing taking place during the saccade (this is the little demo I did with the clock at your studio…). Inhibition of visual processing during the saccade is essential for the smooth stitching together of the visual scene.

Once we have fixed out attention on something in the world, our eyes still jitter around a bit. This means that the same photoreceptors in the retina are not exposed continuously to the same stimulus. If they are, the receptors desensitise and we become effectively blind…

Hmmm… more than a sentence or two there… how about this:

Saccades are rapid movement of the eyes that take a snapshot of the visual world. The visual cortex assembles the conscious experience of viewing a continuous scene from a series of saccades. This requires the activity of many areas of the brain, as well as a kind of visual memory that keeps track of what has been captured visually in each saccade.

Q. How do our eyes and brain creates depth and perspective in seeing. How do we know that something is close to us or further away. How do we situate ourselves in relation to our field of perception.

A. As it turns out, this area is actually one of the best known in just about all of neuroscience (even if there are still lots of unanswered questions). Here’s a quick run through:

The brain uses several sources of visual information to create a sensation of depth.

The main way we judge depth is via stereoscopic binocular vision. Each eye sees the world from a slightly different angle. The information from each eye is sent to the visual cortex which then compares the views from each eye. Objects that are closer look more different when seen from each eye than do more distant objects. This binocular disparity is then used by the visual cortex to encode distance of objects from the viewer.

[ 3D movies / TV etc work by showing two views of the scene that correspond to the view seen by each eye. The technological trick here is to make sure that the left-eye’s view is seen only by the left eye and similarly for the right eye… ]

We also judge depth by focus. We cannot focus on near and far objects at the same time. The visual cortex uses information from the eye-focus muscles to help us interpret distance. Usually when we look at something up close, we not only have to focus, but we also have to converge our eyes onto the object [ ie go a little cross-eyed ]. Again information from the muscles that move the eyes is sent to the visual cortex.

The visual cortex also extracts depth information from the relative locations of objects in space, especially as we move through visual space or the objects move or both.  Two effects contribute to this information. Closer objects block or occlude the view of object further away. And when we move, closer objects seem to move through our visual space more quickly than more distant ones.

Finally visual information on the spatial locations of object and their relationships to the viewer can be reinforced by other senses, especially proprioception, which can tell us if an object is within reach or not. We also know distances of objects by having physically navigated the space within which they and we co-exist [ eg by walking past the objects.]

Sounds of moving objects can tell us if they are nearby or distant. The Doppler effect can tell us if an object is moving towards us or away from us [ this is when a siren or jet engine seems to increase in pitch as it moves towards us and decreases pitch at it moves away. ]

This multi-sensory integration takes place in the inferior parietal cortex, the same general area that builds up our own body sense. This high-level processing is essential to centre visual perception of space and depth of the external world on our body-centred frame of reference.

Q. How does the brain construct a sense of unified self, or how does the conscious mind filter the enormous incoming stream of data to construct an idea of the world and self that has a contained, cohesive meaning. Does this question make sense? Can we describe the phenomenon of unified selfhood from a neurophysiological viewpoint?

A. The brain constructs your sense of self from the memories of your past, your experience of now, and your imagined future.

Your brain sets your self as the reference point for the predicted outcomes for all your actions, real or imagined, physical or mental, rational or emotional.

Your sense of self is the reference point for your feeling of space and time and scale and relationships with the world, the objects within it and the people who inhabit it.

A complete sense of self requires interactions between multiple regions of the brain, regions that process sensory inputs, motor outputs, concepts of space and time, emotional state and social context.

The parietal cortex integrates sensory inputs from the outside world with your sensations from your own body. If it is damaged, you may feel that parts of your body are not part of your self, they are not yours.

Your conscious experience of the self as a complete autonomous entity in the here and now takes about one-third of a second to synthesise: our conscious sense of self inhabiting a physical world is always running a fraction of a second behind real time..!

Your self is the conscious manifestation of your total cerebral activity. However, much of your self-awareness, your feeling of who you are, is a consequence of the subconscious activities of your brain. This subconscious brain processes sensory inputs, makes decisions, moves your body, keeps your balance and emotional state in check, controls the activity of your internal organs. If any of this goes wrong, you “do not feel your self”…

Your sense of self necessarily must be a product of your genes, your upbringing, and your environment.

Pure introspection cannot identify the cerebral functions that generate our conscious sense of self.

No-one really knows how we experience our conscious sense of self… Although we can identify the types of brain activity that are required for a complete sense of self, we can have direct experience only of our own consciousness…