As part of my PhD research at Osaka University, I developed an LLM-driven virtual robot system that can estimate children's anxiety and social self-esteem simply by observing how they talk and play with a virtual agent during an online game. The system combines a GPT-powered virtual version of CommU, a social robot developed at Osaka University in collaboration with Vstone Co., Ltd., with real-time analysis of children's language and gameplay behaviour to provide a non-invasive approach to early mental health assessment.

Children from India played the puzzle game Suika while interacting naturally with the virtual robot, which encouraged them, celebrated their achievements, and held conversations throughout the session. Without relying on questionnaires during gameplay or wearable sensors, the system analysed subtle behavioural cues—including speech patterns, response times, and gameplay decisions—to estimate children's anxiety and social self-esteem.

The results were highly promising. The AI models successfully estimated both anxiety and social self-esteem from children's behaviour and accurately distinguished children who may be at risk from their typically developing peers. Importantly, the game-based interaction proved substantially more informative than conversation alone, highlighting how playful, engaging experiences can reveal behavioural patterns that traditional assessments often miss. Children also showed short-term improvements in anxiety and social self-esteem after the interaction and perceived the virtual robot as kind, trustworthy, and enjoyable, demonstrating the potential of LLM-driven social robots as scalable, accessible tools for supporting the early identification of children's mental health needs.