LEGO robot making conversation

02 June 2015

Having a conversation with a talking LEGO robot has been made possible thanks to a collaborative project between UC's HIT Lab NZ and Japan's Osaka University.

LEGO robot making conversation

Having a conversation with a talking LEGO robot has been made possible thanks to a collaborative project between UC’s Human Interface Technology Laboratory New Zealand (HIT Lab NZ) and Japan’s Osaka University.

Masters student Shogo Nishiguchi, from Osaka University, has been working with HIT Lab NZ and has created The Robot Engine (TRE).

Nishiguchi says the result is a new approach to animating robots and programming their interaction with users.

“Enabling experts on human behaviour to design fluent animations and behaviours for advanced robots is problematic, since the tools available for such robots are often in their prototype stage.

“We have built TRE based on the Unity 3D Game Engine to control robots. Unity 3D allows non-programmers to use a set of powerful animation and interaction design tools to visually animate robots.”

Nishiguchi says that one of the greatest features of TRE is the conversational function.

“You can write the script for a conversation virtually on TRE and the robot will talk accordingly. However, it is almost impossible to write down all the possible questions and answers, as people sometimes give a random question that is out of context to the robot.”

Nishiguchi says that to solve the problem they embedded a function, called Chatbot, which is an online database of conversation, and gives proper answers to questions.

“The advantage of scripting is that the conversation is heading to a goal set by the user. On the other hand, Chatbot can answer any random questions. TRE makes use of both advantages,” he says. 

“The robot normally talks along the script with the user but if people ask it something unexpected, it connects to Chatbot to answer it.”

The LEGO robot was used to conduct field work at Christchurch’s Imagination Station, located in Christchurch, using TRE. The Imagination Station is a charity run, not-for-profit play and education centre and the first LEGO play and learn centre in New Zealand.


“The robot was put by a donation box and asked customers to donate. The Imagination Station completely relies on donations to keep it open and free-of-charge.

“The purpose was to investigate if the robot could naturally convince people to donate through conversation. The robot had to have a natural conversation without irritating people and making them enjoy it. 

“We believe that this unconventional integration of technologies has the potential to bring the expertise of integration designers into the process of advanced human-robot interaction projects.” 

The video of The Robot Engine can be viewed here: https://youtu.be/I0EkGOZKYtw

The results of this project will be presented at the 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN2015), in Kobe, Japan in September 2015.


HIT Lab NZ is a human-computer interface research centre hosted at the University of Canterbury. The HIT Lab NZ integrates research and education, providing students with a project-based learning environment.  

For further information please contact: 
Charlene Smart, Communications Advisor, University of Canterbury on 03 364 2260 or communications@canterbury.ac.nz.