A view From the Bridge

Machines moved by mind

3Q: José Millán

A 'mental worker' (behind screen at right) with Machine 1 at the exhibition Mental Work.

A visitor (behind screen at right) driving Machine 1 using the force of their own thoughts, at the exhibition Mental Work.

© Photography Adrien Baraka / Mental Work

At Mental Work, an exhibition at the Swiss Federal Institute of Technology Lausanne ArtLab (EPFL), visitors can drive simple machines using the force of their own thoughts. Probing the rapidly changing relationship between humans and technology, these artworks will also generate vast amounts of data that will be shared with researchers around the world. The show is a collaboration between experimental philosopher Jonathan Keats and EPFL neuroengineer José Millán, who develops brain-machine interfaces (BMIs) to help people with paralysis. Here, Millán talks pistons, probability and the debate over who or what is in control.

What will visitors experience at the show?

Some will be active participants in three experiments; others will watch them work. The participants, or ‘mental workers’, wear an EEG helmet studded with 19 dry electrodes — which continuously pick up electrical activity in their brains. In the first experiment they sit in front of a 2-metre-long construction (Machine 1) comprising a piston, fly-wheel and horizontal shaft. Using mental imagery, they try to move the piston onto the fly wheel; this starts the wheel turning, driving the shaft through a bolt. The brain-machine interface or BMI that makes this possible is an algorithm that has to be trained to ‘read’ the mind of each driver. The driver instigates the training by making a binary movement of the hand or foot, such as clenching and opening a fist, while simultaneously imagining the piston moving or stopping. The algorithm learns the stop-go instructions from patterns of the data from the electrodes, and converts them into commands for the piston. Because the data are always noisy and variable, the command is based on probability; but we program the piston motor to generate movement only when the probability is high — usually in the 70-90% range.

xxx

Another view of a ‘mental worker’ with Machine 1.

© Photography Adrien Baraka / Mental Work

What happens in the other two experiments?

They are more complex, and so are the machines. Participants take the role of either ‘driver’ or ‘supervisor’. Supervisors may change the level of probability through their own mental imagery, so the driver has a harder or easier (but messier) job of getting the machine to work. Or the supervisors may use their mental imagery to instruct the BMI to stop using mental imagery altogether, and switch to a different algorithm that use patterns of alpha waves — the brain-wide oscillations generated when the brain is at rest — to drive movement. In this case, the supervisor also uses mental imagery to instruct the driver to relax and ‘empty’ his or her brain. This is the part I am terrified about! We can get this to work in the lab, but it gets so complicated we don’t know what will happen when it is tested in more open conditions. We’ll also distribute a questionnaire asking participants whether they felt they were controlling the machines or if the machines were controlling them.

 

xxx

Machine 2, where ‘drivers’ have their threshold adjusted using a brain-machine interface or BMI.

© Photography Adrien Baraka / Mental Work

What do you want to emerge from the exhibition?

We are entering a cognitive revolution in which we will increasingly use many different new technologies to tap into or extend the capabilities of our brains. I hope that Mental Work will help generate a societal debate about this. Could brain power be used to carry out real work in the real world? What would that mean for employment? Will machines take control of our minds, or will our minds always have the control of machines? Personally, I am optimistic – I think the future is up to us. But the debate needs to start now. I hope visitors to this show will also enjoy the aesthetics of these artistic machines. Meanwhile, the data will be very valuable scientifically. We will capture and share it with the BMI research community, which is constantly trying to improve interfaces, for example by increasing the probability that brain signals are correctly read. Our experience suggests that many participants improve their performance as they move from one machine to another, and I expect that the research community will also be able to develop better machine-learning techniques for BMI users. At the end of the day what I really want is help BMI users, particularly  people with paralysis, to generate brain signals that are more stable and easier to decode.

Interview by Alison Abbott, senior European correspondent for Nature. She tweets at @alison_c_abbott

 This interview has been edited for length and clarity.

Mental Work runs from 27 October – 31 January 2018. The first two weeks are open for registered participants only, so any visitors wishing to participate as ‘mental workers’ must first sign up on the website mentalwork.net. The show opens to the general public on 13 November. It will subsequently move on to swissnex San Francisco in California. 

 

For Nature’s full coverage of science in culture, visit www.nature.com/news/booksandarts.

Comments

There are currently no comments.