Autism spectrum disorder (ASD) is a neurodevelopmental disorder that affects communication and behavior in individuals. This disorder in autistic people is often present from early childhood and can impact daily functioning and pose a challenge to the child at multiple levels. Given that Autism is less understood, research on it lags that of other psychiatric disorders and medical conditions. Autism is a disorder on a spectrum and is hence more widely spread than understood. Therefore, it is critical to study and understand the most prevalent characteristics indicative of the disorder.

In this project, the team investigated a fresh approach to recognize the characteristics or STIMS prevalent in children with ASD via video recordings of their regular activities. The team created a Deep Learning solution for autistic actions recognition in children below eight years of age to generate a final algorithm that would classify Autism STIM in children and store them in a PostgreSQL database. The deep neural networks were trained on videos for the model to perform symptomatic action- recognition to foster accurate predictions. The input of one or multiple videos of the autistic child performing daily activities was taken. The team used an R (2+1) D model for action recognition and an MTCNN (Multi-Task Cascaded Convolutional Neural Networks) model for face recognition.

The team claims this to be an indicative approach, not a confirmatory one.

This approach can enable early intervention by parents or guardians, which might otherwise have been delayed due to dependencies and a lack of medical experts. Depending on the number or frequency of STIMS (Self-Stimulatory Behaviour) exhibited by subjects, this model can help prioritize those that require a diagnosis from psychiatrists or experts. Traditionally, ASD was diagnosed through a series of rigorous tests and analyses, which, because subjective, differ from expert to expert. Also, these exhaustive tests can be challenging for the subject. The model would also help medical practitioners to prioritize their consultation services to children with a higher probability of ASD.

The project attempts to make an action-recognition model and does not claim to identify and classify a child as autistic. The input to the model is only videos that are insufficient for ASD detection. Detection requires additional sources like medical history, EEG, etc.

The final algorithm would be able to classify the Autism STIM, if present, in children based on the videos provided by the company and store them in a PostgreSQL database. The model would perform symptomatic action recognition by training the deep neural networks on the videos to aid in accurate predictions. The expectation is also to create reports of the results and classes obtained.

This project's successful implementation and advancement will impact the subject, medical fraternity, and society. Not only will it detect the general characteristics of ASD, but it also help the child and parents to support early intervention and treatment. It can prioritize consultation and diagnostic services and aid further research.


1) Pamuru V., Rastogi A., Bhardwaj K., Kamalapurkar R., Tekle R.