Open-Source AI Platform A‑SOiD Enables Video‑Based Behavior Recognition and Prediction

Researchers from Carnegie Mellon University, the University of Bonn Hospital, and the University of Bonn have created an open‑source platform called A‑SOiD that learns and predicts user‑defined behaviors exclusively from video; the results were published in the journal Nature Methods .

Artificial intelligence tools treat each class in the dataset fairly

Eric Yttri, an associate professor at Carnegie Mellon, explained that the technology is well‑suited for classifying a wide range of animal and human behaviors and can also be applied to any domain with recognizable patterns, such as the stock market, earthquakes, or proteomics, making it a powerful pattern‑recognition machine.

Unlike many AI programs that act as black boxes, A‑SOiD allows researchers to retrain the system when it makes mistakes. The team first trains the algorithm on a subset of the data, emphasizing the program’s weaker beliefs, and when uncertainty arises, the algorithm reinforces the credibility of the training data.

According to a user with a disease, Alex Hsu, A‑SOiD’s focus on algorithmic uncertainty rather than treating all data uniformly helps avoid the biases common in other AI models.

The platform’s supervised training makes it highly precise; it can distinguish normal tremors from those of Parkinson’s patients and serves as a complementary method to the earlier unsupervised behavior‑segmentation platform B‑SOiD.

A‑SOiD is not only effective but also easily accessible, capable of running on ordinary computers, and its source code is openly available on GitHub.

Video examples illustrate how the system can segment two sub‑classes of mouse behavior: one where a mouse approaches another’s mirror genital area and another where a mouse investigates the throat region of another mouse.

Jens Tillmann, a postdoctoral researcher at the University of Bonn Hospital, highlighted that making the project open to all researchers is a key part of its impact.

Tillmann emphasized that without the collaborative spirit of the two labs and the broader neuro‑behavioral community, the project would not have been possible, and he looks forward to future collaborations.

Eric Yttri and Martin K. Schwarz plan to use A‑SOiD in their respective labs to further explore the relationship between brain activity and behavior, combining it with other tools for fine‑grained analysis of social interactions.

Both researchers hope that A‑SOiD will be adopted by interdisciplinary and international teams, providing a unique opportunity to study causal links between brain activity and behavior and to foster collaborative research across Europe and the Atlantic.