TV show gives computers a lead in solving problems

Edinburgh scientists are training computers to solve problems using episodes of a popular TV crime drama.

Researchers from the School of Informatics taught artificially intelligent machines to study the show CSI, and identify the perpetrator in each case.

Their research was aimed at enabling machines to solve a problem by assimilating information from images, audio, transcribed dialogue and scene descriptions.

Difficult tasks

The researchers studied if and how artificially intelligent machines can solve problems that are challenging for humans.

The computer model processed data in various forms – spoken, visual or textual – as the plot of each episode developed.

Complex problems

Scientists designed their computer model to solve arbitrary problems based on acquiring data.

Such devices could play a role in developing efficient algorithms for real-world tasks that require complex reasoning, researchers say.

Data processing

Scientists from the University of Edinburgh mapped footage, script and background sounds from five seasons of the show into a machine-readable format.

The data was fed into a computer model that learned to process the plot as each episode unfolded, continually revising the criminal’s identity.

The computer correctly identified the perpetrator during the final part of an episode 60 per cent of the time.

People who watched the same shows were able to identify who was responsible 85 per cent of the time, the study found.

Solving cases

The American series Crime Scene Investigation, or CSI, began in 2000 and ran for 15 seasons.

Scientists taught machines to approach solving the crimes in the same way that people would – by considering which characters might be responsible from their behaviour as the plot unfolds.

The study, published in Transactions of the Association for Computational Linguistics, was funded by the European Research Council and H2020.

Pinpointing the perpetrator in a TV show is a very difficult task for computers, but our model performed encouragingly well. We hope our findings will aid the development of machines that can take on board – and make sense of – large streams of information in real time.

Dr Lea FrermannSchool of Informatics

Homepage image ©