Short bio

Kristiina Jokinen is Senior Researcher at AIRC, the Artificial Intelligence Research Center of Japan’s National Institute for Advanced Industrial Science and Technology (AIST), located at AIST Tokyo Waterfront in central Tokyo. She has a PhD from UMIST (University of Manchester Institute of Science and Technology) and is Adjunct Professor at University of Helsinki.

After her PhD she was awarded a JSPS Research Fellowship at NAIST in Japan and was next an Invited Researcher at ATR Research Labs in Kyoto. She was Nokia Foundation Fellow at Stanford University in 2006, NICT Visiting Professor at Doshisha University in 2009-10, and is a Life Member of Clare Hall at University of Cambridge. Before joining AIRC, she was Professor and Project Manager at University of Helsinki and at University of Tartu.

Her research focuses on spoken dialogue systems, multimodal communication (speech, gaze, gesturing), and human-robot interaction. She led the development of WikiTalk, a Wikipedia-based robot dialogue system which won an award in Best Robot Design (Software Category) at ICSR 2017. She has been Principal Investigator in multiple international projects, and was Director of the Academy of Finland DigiSami project. She has published four books, and has organised many international workshops including the northernmost spoken dialogue conference IWSDS 2016 in Saariselkä, Finland.

News

WikiTalk and WikiListen at ECAI 2020

In September 2020, Graham Wilcock and Kristiina Jokinen presented WikiTalk and WikiListen by video at 24th European Conference on Artificial Intelligence (ECAI 2020) in Santiago de Compostela, Spain. Their paper WikiTalk and WikiListen: Towards listening robots that can join in conversations with topically relevant contributions was published in the ECAI 2020 proceedings.

ERICA and WikiTalk at IJCAI 2019

ERICA and WikiTalk

In August 2019, a demo of ERICA and WikiTalk was presented at 28th International Joint Conference on Artificial Intelligence (IJCAI 2019) in Macao, China. A paper ERICA and WikiTalk by D. Lala, G. Wilcock, K. Jokinen and T. Kawahara was published in the IJCAI 2019 proceedings.

IWSDS 2019

IIn April 2019, Kristiina Jokinen and Graham Wilcock attended 10th International Workshop on Spoken Dialogue Systems (IWSDS 2019) in Siracusa, Italy, and gave a presentation Towards increasing naturalness and flexibility in human-robot dialogue systems.

WikiTalk wins an award at ICSR 2017

In November 2017, WikiTalk won a Special Recognition award for Best Robot Design (Software Category) at 9th International Conference on Social Robotics (ICSR 2017) in Tsukuba, Japan. The award was given to Kristiina Jokinen and Graham Wilcock for their competition entry Next Steps for Social Robots.

Springer book: Dialogues with Social Robots

In February 2017, Dialogues with Social Robots edited by Kristiina Jokinen and Graham Wilcock was published by Springer in the series Lecture Notes in Electrical Engineering. The publisher says: This book explores novel aspects of social robotics, spoken dialogue systems, human-robot interaction, spoken language understanding, multimodal communication, and system evaluation. It offers a variety of perspectives on and solutions to the most important questions about advanced techniques for social robots and chat systems. Chapters by leading researchers address key research and development topics in the field of spoken dialogue systems, focusing in particular on three special themes: dialogue state tracking, evaluation of human-robot dialogue in social robotics, and socio-cognitive language processing.

WikiTalk at COLING 2016

Mulitilingual WikiTalk

In December 2016, Kristiina Jokinen and Graham Wilcock demonstrated WikiTalk at 26th International Conference on Computational Linguistics (COLING 2016) in Osaka, Japan. A paper What topic do you want to hear about? A bilingual talking robot using English and Japanese Wikipedias by G. Wilcock, K. Jokinen and S. Yamamoto was published in the COLING 2016 proceedings.

DigiSami at IWSDS 2016

SamiTalk Sami-speaking robot

In January 2016, Kristiina Jokinen was Conference Chair of 7th International Workshop on Spoken Dialogue Systems (IWSDS 2016) in Saariselkä, Finland. The conference was organized by the Academy of Finland DigiSami project directed by Kristiina.

WikiTalk

WikiTalk is basically a talking Wikipedia. You interact with the robot by speech, and navigate by speech to whatever topics in Wikipedia you are interested in, and the robot tells you about them using the information from Wikipedia.

The robot gets the information directly from Wikpedia by wi-fi, so you get the latest up-to-date information even for rapidly changing topics. Wikipedia information is trustworthy, because it is contimuously validated by a worldwide community of volunteer editors.

WikiTalk anticipates what you will probably want to hear about next and extracts links to related topics in Wikipedia. If you just say the name of a related topic, the robot switches to the new topic and starts talking about it. Predicting likely topic shifts like this also helps speech recognition.

WikiTalk is multimodal and multilingual. It integrates face-tracking, nodding and communicative gesturing with speech synthesis and speech recognition, following the CDM Constructive Dialogue Model. The system now works in English, Finnish and Japanese.

A first implementation of WikiTalk on Nao robots was made at Supélec in Metz, France in 2012 during a one-month international PhD summer school. A Finnish language localisation was made at University of Helsinki by the Academy of Finland DigiSami project as a step towards SamiTalk.

Since 2016 WikiTalk has been developed by CDM Interact, a Finnish startup company. A Japanese language localisation was made at Doshisha University, Japan. Multilingual WikiTalk with language-switching on a Nao robot was demonstrated at COLING 2016 (International Conference on Computational Linguistics) in Osaka, Japan.

In 2017 WikiTalk won an award in Best Robot Design (Software Category) at ICSR 2017 (International Conference on Social Robotics) in Tsukuba, Japan.

In 2019 WikiTalk was used with ERICA the female android robot  at Kyoto University, Japan. The combination of ERICA and WikiTalk was demonstrated at IJCAI 2019 (International Joint Conference on Artificial Intelligence) in Macao, China.

DigiSami

The DigiSami project was part of a wider joint research project funded by the Academy of Finland and the Hungarian Academy of Science, intended to increase the visibility and use of small Fenno-Ugric languages in the digital world by applying language technology to support the development of digital materials. The goal of the Finnish part was to support digital content generation for the Sami languages, focusing mainly on North Sami. The DigiSami project at University of Helsinki investigated how modern language technology and corpus-based linguistic research can contribute to handling the digitalisation challenges faced by small Fenno-Ugric language communities.

DigiSami Corpus

The project collected the DigiSami Corpus of Spoken North Sami language in several North Sami-speaking areas of both Finland and Norway. The spoken dialogues were transcribed and carefully annotated using modern corpus linguistics techniques. The speech materials were made available to colleagues at Aalto University working on speech technology, who began to develop a speech synthesizer and a speech recognizer for North Sami.

SamiTalk

The project worked towards SamiTalk, a Sami-speaking robot linked to Sami Wikipedia. The aim was to demonstrate to young Sami speakers that the Sami languages of their grandparents will also play an active part in the digital future. WikiTalk had shown robots talking about many topics using English Wikipedia. The DigiSami project worked on localisation of spoken dialogue systems and made a Finnish localisation of WikiTalk using Finnish Wikipedia, as a step towards SamiTalk. A video of a SamiTalk demonstration prototype shows the robot talking about the situation of Sami languages in Finland using information from Sami Wikipedia. The robot speaks North Sami, but can only do speech recognition for Finnish as the North Sami speech recognizer was not available.

International Workshop

The project organised IWSDS 2016, the northernmost International Workshop on Spoken Dialogue Systems, in Saariselkä, Finland. The workshop attracted over 50 researchers from Europe, Japan and USA. Revised versions of the workshop papers were collected in Dialogues with Social Robots, edited by Kristiina Jokinen and Graham Wilcock and published by Springer in the Lecture Notes in Electrical Engineering series.

Best Paper Award

The project also worked on multimodal analysis of the videos and annotations in the DigiSami Corpus using machine learning techniques, in collaboration with University of Eastern Finland. This work was recognised by a Best Paper Award at IWSDS 2018 for Enabling Spoken Dialogue Systems for Low-resourced Languages: End-to-end Dialect Recognition for North Sami by Trung Ngo Trong, Kristiina Jokinen and Ville Hautamäki.

Robot Videos

ERICA and WikiTalk

ERICA and WikiTalk (2019)

ERICA the female android robot talks about ancient Greek, robots, artificial intelligence and androids using

Wikitalk with English Wikipedia.

CityTalk Tokyo 2020 (2018)

The robot tells about restaurants and hotels in Tokyo Waterfront near the Tokyo 2020 Olympics.

Mulitilingual WikiTalk

Multilingual WikiTalk (2016)

A multilingual robot talks about Shakespeare in English and Japanese, switching languages on request.

SamiTalk Sami-speaking robot

SamiTalk (2016)

The world’s first Sami-speaking robot talks about the situation of Sami languages in Finland using information from Sami Wikipedia.

Finnish-speaking robot at home

Finnish WikiTalk (2015)

At home with a Finnish-speaking robot who talks fluently about many topics using information from Finnish Wikipedia.

MoroTalk Finnish news-reading robot

MoroTalk first demo (2015)

A Finnish-speaking robot tells local news about Tampere city in Finland using online news feeds of the Aamulehti newspaper.

WikiTalk first demo

WikiTalk first demo (2012)

The first demo of WikiTalk on Nao robots at International Summer Workshop on Multimodal Interfaces in Metz, France in 2012.