Raquel Fernández: talking to your computer as if it were human
You may have seen one while shopping online: you scroll a bit and a pop-up suddenly appears in the corner of your screen, asking, ‘Is there anything I can help you with?’ These ‘conversational agents’ or chatbots are like digital assistants meant to make our lives easier. The question is, do they?
Raquel Fernández, full professor at the Institute for Logic, Language and Computation, is studying how to make computers communicate more like real people do.
‘We humans are constantly using language. This way of communicating is a fascinating thing,’ Raquel says with enthusiasm. ‘You produce sounds that another person hears and this allows you to transmit ideas and cooperate. I want to understand how humans do this and then capture it in computer models. That way, it will be possible to talk to a computer as if it were a person.’
Raquel praises the interdisciplinary nature of the field. ‘We’re working with Communication Sciences, for instance, to see whether conversational agents can adapt their language use to different age groups.
First, we looked at subjects of varying ages to see whether they used different styles or slang that could be detected automatically. That was indeed the case. The conversational agent was then able to both detect the variance in language use and adjust its own language use based on who it was speaking with, the older or younger user.
We're now at the point where it's time to let this adaptive agent communicate with real people and see whether people find it a more pleasant experience. Does the digital agent seem more human to them now? Did it feel like a conversation with a real person? That study is scheduled to take place this year. I’m extremely curious to see the results.’
"I want to understand how humans use language and then capture it in computer models. That way, it will be possible to talk to a computer as if it were a person."
Does the virtual agent also have to estimate the age of the person it's talking to? ‘In this case, the agent knows the age group the person with whom it’s communicating belongs to before the conversation starts.
A next step will be to have the agent estimate the age of the user based on their language use. But it doesn't have any eyes, so it can't see.’ She adds, laughing, ‘Otherwise, that could lead to awkward situations, if the agent was completely wrong about a person's age based on physical appearance.’
Human touch
Still, computer vision is a very real part of Raquel's work. ‘In conversations, people constantly refer to things we perceive around us. This is not a problem when you're talking to another person, but it could be if you were to ask your smart assistant: “what kind of flower is that?” Then the assistant would have no idea what you were talking about.
We still have a long way to go in this regard. Visual information could be really useful in giving conversations with your smart assistant that human touch.’ Raquel mentions an example: preparing a recipe together. ‘With visual clues, the smart assistant could not only tell you the ingredients, it could also warn you if you're going to need a bigger pan. Then it becomes a genuine cooperative effort.’
So how long until we achieve that genuine cooperation between human and machine? ‘We are nowhere near that point, but we are making rapid progress. There are already systems that are capable of describing images or other visual input. They still make mistakes, though. Sometimes, they “hallucinate” and describe a thing that is not in the picture. The system might see a car, for example, even when there's no car there.
The algorithm has no concept of what is true or correct; it only looks at probabilities based on large amounts of data. In such cases, the data used to train the algorithm probably did show a car most of the time. The systems we have created to describe images often work well in a controlled setting, but not yet in the real world. I see that as a challenge.’
Working in LAB42
What does Raquel expect working in the new LAB42 building to be like? ‘As a field of study, natural language processing and in particular ‘vision and language’ are booming and may potentially have a tremendous impact on society. Take, for instance, visual recognition for use in self-driving cars or to help visually impaired people.
Because I'm not originally from the Netherlands, most of my contacts are international. Through LAB42, I hope to grow my local network and pursue greater cooperation with other groups, such as industrial partners, social organisations and start-ups. More local contact leads to more impact.’
About Raquel Fernández
- Leading the Dialogue Modelling Group at the Institute for Logic, Language and Computation;
- More information on Raquel her personal page on the UvA site.