Mental illness impacts emotions of an individual and can have serious consequences. It then becomes imperative to innovate strategies for prevention, accurate diagnosis, and intervention. Mental illness can compel a patient to take extreme steps, predictive analytics can play a significant role in abating any life-threatening situations.


The new age technology can help in early detection of mental health issues through emotion analysis. Artificial Intelligence can aid monitoring emotions on multiple modes. Chatbots or a robotic companion, in real-time, can help predict individual’s risk of depression and suicidal behaviour.

This study developed a working model to supplement research work that augmented telepsychology and psychotherapists for observations and interventions. Important to note is that this alone will not determine the disease and its cure. The study guided psychologists in selection and timing of therapeutic tools.


The objective of the project was to cover all the modalities for prediction (Image, Video, Audio, Text and Bio-signals), to determine fusion techniques across modalities and to uncover the temporal aspects across modalities to support real-time predictions.

The team found a strong corelation between few emotions such as anger and fear and sadness. Some modalities were strongly associated with certain emotions. For example, anger, happiness and surprise are video dominated emotions while sadness and fear were audio- dominated emotions. The team also observed increased accuracy in video (59%), audio (87%) and text (62%).

The managerial implications of the project was that the it helped develop a systematic approach for predicting human emotions using multiple modalities. The results can be useful in analysing emotions and act as an aid or tool to the psychiatrists in telepsychology. The work in exploitation of bio signals as a modality can be used to investigate clinical use cases. Use of multi-modality (video, audio, text) and added bio-signal modes (colour on the face) were suggested for more accurate results. The team systematically developed step-by-step method for model development and use. The model thus presented can aid the psychiatrist treatment recommendation with remote therapy.

There is immense user value of the project undertaken. Coupled with the proposed model, psychiatrists can identify the emotional state of the subject better, modify treatment and analyse.

With most of the world moving towards virtual form of interactions post Covid pandemic, Emotion AI plays a significant role when it comes to detecting emotions in virtual conversations. It supports the psychiatrists or the healthcare professionals to provide the best treatment and continuously monitor their well-being. By knowing the emotional state of the subject better, the psychiatrist can modify their treatment and can even understand how the subject is reacting to the intervention introduced. Hence this ability to detect emotions correctly, will not only be extra ordinarily useful but will also bring positive changes to the realm of mental health. User value of this project can be achieved by helping individuals:

  • Identifying signs of depression, anxiety, grief and pain.
  • Creating awareness of their emotional states.
  • Achieving emotional well-being of one and all.

There are other potential business use cases of this emotion detection which were also identified as a part of this project. In car driver emotion detection system A detection system can be deployed over vehicles which may aid the drivers to understand their emotion state and to drive carefully. This if integrated with the vehicles sensors systems can also be used for controlling speeds and informing the near-and-dear ones of the situations. Emotion detection and scores during interviews The emotion detection system can come up with temporal scores during different stages of interviews which can help in the evaluation process, making it more quantitative and less subjective and giving a score for emotional intelligence during different situations.

References:

1) Kumar S., Nandkumar A., Nidhi M., Nagpal S., Dhawan V., ‘Prediction of Emotional State using AI’