Using electrical signals from the human brain, new software can perform computerized image editing

Soon computers might sense that users have a problem and come to the rescue. This is one of the possible implications of new research at the University of Copenhagen and the University of Helsinki.

“We can make a computer edit images based entirely on thoughts generated by human subjects. The computer has absolutely no prior information about what features it is supposed to edit or how. No one has never done this before,” says Associate Professor Tuukka Ruotsalo, Department of Computer Science, University of Copenhagen.

The results are presented in a paper accepted for publication at CVPR 2022 (Computer Vision and Pattern Recognition).

Brain activity as the only input

In the underlying study, 30 participants were fitted with hoods containing EEG electrodes that map electrical signals from the brain. All participants were given the same 200 facial images to view. In addition, they were given a series of tasks such as finding female faces, finding older peoplelooking for blonde hair etc.

Participants performed no action and looked at the images briefly – 0.5 seconds for each image. Based on their brain activity, the machine first mapped the given preference and then modified the images accordingly. So if the task was to look for older people, the computer would alter the portraits of younger people, making them look older. And if the task was to search for a given hair color, all images would have that color.

“Notably, the computer had no knowledge of facial recognition and would have no idea of ​​gender, hair color or any other relevant characteristic. Yet it only changed the characteristic in question, leaving others facial features unchanged,” comments PhD student Keith Davis, University of Helsinki.

Some will say that there is already a lot of software capable of manipulating facial features. That would be missing the point, says Keith Davis:

“All software out there has been pre-trained with labeled inputs. So if you want an app that can make people look older, you feed it thousands of portraits and tell the computer which ones are young and which ones are old. Here, the brain activity of the subjects was the only input.This is an entirely new paradigm in artificial intelligence– directly using the human brain as an input source.”

Possible applications in medicine

A possible application could be in medicine: “Doctors already use artificial intelligence in the interpretation of scanned images. However, errors do occur. After all, doctors are only assisted by the images but will take the decisions. Perhaps certain features of the images are more often misinterpreted than others. Such patterns could be uncovered through an application of our research,” says Tuukka Ruotsalo.

Another application could be assistance to certain groups of disabled people, for example by enabling a paralyzed person to operate his computer.

“That would be fantastic,” says Tuukka Ruotsalo. “However, that is not the focus of our research. We have a broad scope, seeking to improve machine learning in general. The range of possible applications will be wide. For example, in 10 or 20 years, we may no longer need to use a mouse or type commands to operate our the computer. Maybe we can just use mind control.”

Calls for political regulation

However, there is a flip side to the coin, according to Tuukka Ruotsalo: “Collecting individual brain signals involves ethical issues. Whoever acquires this knowledge could potentially gain deep insight into a person’s preferences. We are already seeing some trends. People buy smart products.’ Watches and similar devices capable of recording heartbeat etc., but are we sure that data is not generated that gives private companies knowledge that we would not want to share?”

“I see this as an important aspect of academic work. Our research shows what is possible, but we shouldn’t do things just because they can be done. This is an area that I believe needs to be regulated by public guidelines and policies. . If these are not adequate, private companies will go ahead.”