While AI models can perform complex tasks such as writing essays and generating art, they have yet to master some skills that humans carry out with ease, researchers say.
An Edinburgh team has shown that state-of-the-art AI models are unable to reliably interpret clock-hand positions or correctly answer questions about dates on calendars.
AI challenge
Unlike simply recognising shapes, understanding analogue clocks and calendars requires a combination of spatial awareness, context and basic maths – something that remains challenging for AI, the team says.
Overcoming this could enable AI systems to power time-sensitive applications like scheduling assistants, autonomous robots and tools for people with visual impairments, researchers say.
The team tested if AI systems that process text and images – known as multimodal large language models (MLLMs) – can answer time-related questions by looking at a picture of a clock or a calendar.
Reading clocks
Researchers tested various clock designs, including some with Roman numerals, with and without second hands, and different coloured dials.
Their findings show that AI systems, at best, got clock-hand positions right less than a quarter of the time. Mistakes were more common when clocks had Roman numerals or stylised clock hands.
AI systems also did not perform any better when the second hand was removed, suggesting there are deep-seated issues with hand detection and angle interpretation, the team says.
Calendar dates
The researchers asked AI models to answer a range of calendar-based questions, such as identifying holidays and working out past and future dates.
The team found that even the best-performing AI model got date calculations wrong one-fifth of the time.
The findings are reported in a peer-reviewed paper that will be presented at the Reasoning and Planning for Large Language Models workshop at The Thirteenth International Conference on Learning Representations (ICLR) in Singapore on 28 April 2025.