Researchers trained a digital fly brain to play video games
- Google created a 3D model of a fly's brain with 166,000 neurons
- Enthusiasts taught the digital fly to play DOOM, Beat Saber, and other games
- In Beat Saber, the model showed more impressive results
- Experiments open new possibilities in studying neural systems
Google has introduced a complete three-dimensional reconstruction of the brain and central nervous system of an adult male fruit fly. Scientists, in collaboration with the HHMI Janelia research center and the scientific community, combined millions of two-dimensional images to create a detailed digital model. It contains over 166,000 neurons and is intended to help researchers gain a deeper understanding of the mechanisms of brain function at the cellular level.
Shortly after the publication of the results, programmers and enthusiasts decided to apply the digital model in an unusual way. The first experiment was training a virtual fly to play the classic shooter DOOM. Programmer Alex Wormuth implemented a system in which each frame of the game activates specific sensory neurons of the model. The activity of these neurons is then converted into character control commands. The damage taken by the player affects two dopamine cells PPL101, which serve as a reinforcement signal — a kind of reward for successful actions. At the time of publication, the digital fly had spent nearly 50,000 seconds in the game, but it was still too early to call it a successful player.
Successes in the musical game
More impressive results were achieved in the rhythm game Beat Saber. Enthusiast Lyra Bubbles applied a method of repeating compositions and reinforcement learning to train the model to determine when and where to strike. At this stage, the digital fly receives additional information about the notes from the outside, allowing it to react to events in the game. The project author plans to gradually reduce dependence on external signals and achieve a fully autonomous reaction of the fly's brain to the gameplay.
The digital insect was also launched in Super Mario 64 on the Nintendo platform. User X Jessica Packett demonstrated the ability of the fly's brain model to interact with a three-dimensional game environment and execute various control commands.
Expanding the experiments
An enthusiast with the nickname Ro0oney implemented the launch of a digital fly in the Minecraft sandbox. For this, the FlyWire.AI project was used, created with the participation of Princeton University. This platform is based on the brain map of a female fruit fly, containing about 139,000 neurons — slightly fewer than Google's model with its 166,000. Despite the differences in the number of neurons, both approaches use similar principles of operation.
At the moment, all these experiments remain more like interesting projects of enthusiasts than a full-fledged artificial intelligence based on the neural structure of a living organism. However, the very fact that researchers can use detailed maps of neural connections to control digital entities in video games already demonstrates the enormous potential of this direction. If programmers were able to train a virtual fly brain to interact with complex gaming systems in just a few days, one can only speculate what other experiments will be implemented as the nervous systems of other living beings are studied.
Source: StopGame
