It’s all about the body language.

A new AI method for tracking the social lives of rats may help researchers better understand the relationship between the brain and social behavior, with possible implications for human conditions such as autism.

The machine-learning technique was detailed in a paper, “Mapping the Landscape of Social Behavior,” recently published in Cell. Bence P. Ölveczky, professor of organismic and evolutionary biology (OEB) and co-author of the paper, explained: “We are really mapping the social life of rats by capturing the details of their every movement. We see how they interact with each other, and we see the same forms of engagement over and over again.

“We see personalities in these animals that are intriguing. In many ways, these variations can help us understand the basis for a lot of interesting behavioral phenomena, including sociality,” he added.

Rats are social creatures. Much like humans, they interact with each other in ways that influence their behavior through complex social patterns of touching and body language. These rat interactions are not that far from our own, the researchers say.

Ölveczky gave a real-life example: “When people come into my lab, I scratch my head a little bit and soon after they will scratch their heads, or I cross my legs, and they cross their legs. We are subconsciously communicating with each other.”

Although studies of rat behavior have existed for years, in the past they relied on observation and a limited number of data points.

“The standard in the field is for somebody to just watch hours and hours of rat videos and say, ‘Oh, I think that they touched each other there. I think that this guy was mimicking the other guy,’” Ölveczky said. 

The new study was able to take an in-depth look at how those social behaviors are communicated thanks to groundbreaking technology. From videos of the interactions, a machine-learning pipeline extracted more than 110 million 3D poses tracking various points on the rats’ bodies as they moved and interacted. Researchers could then graph how these animals behaved around others, including how they learned and changed through these exchanges.

“By having this methodology, we can replace the subjective human observer with a very rigorous and reproducible method for behavioral quantification and identification of particular gestures or even interaction motifs,” said Ölveczky.

AI also allowed the researchers to “analyze amounts of data that would take humans years and years to scroll through,” said Ugne Klibaite, a postdoctoral fellow in the Ölveczky Lab and lead author on the paper.

“Given how far computer vision and deep learning has progressed and the technology, the cameras, and the computers we have, we can actually get high-resolution animal movement in 3D,” said Klibaite. Now, she continued, “we have a chance to think about what that might mean.”

This advance is already opening new areas for research into autism. A complex disorder, autism probably has environmental components, said Ölveczky. It also seems clear, however, that there is a genetic component, with certain high-risk genes predisposing an individual to autism.

“The question then is how does a mutation or a knockout in this gene affect the brain, and how does that lead to changes in social behavior?”

With funding from the Simons Foundation for Autism Research, which provided rats that had variations in these specific genes, the researchers were able to look at how these genetically modified rats socialized.

While stressing that autism is a human condition, the researchers did find some intriguing parallels.

“This is a spectrum disorder, and we see some of that variability in our different rat models as well,” said Ölveczky.

Noting that children on the autism spectrum often socialize in different ways than children not on the spectrum, he said, “We also see a whole variety of different types of differences in social interactions in these rats that depend on the particular gene that was knocked out.”

Ongoing research will explore these similarities and how they might relate to the altered genes.

“Using this platform, we are going to ask questions about how different parts of the brain process social gestures,” said Ölveczky. “Can we go deeper and really pinpoint the circuits that are responsible for this difference in behavior? And when we can do that — if we can do that — then that could very well inspire new approaches to therapy.”

Adding to the value of the study, the data — the films of the rats and the movement trajectories distilled from them — will be shared, said Klibaite, who led the data collection and behavioral analysis.

“Hopefully by releasing this to the community and getting people to engage with the data as well, we’ll have people in the conversation making better models of how the brain underlies social behavior.”

---

Funding for this research came, in part, from the National Institutes of Health.