Nina the mobile accessibility assistant

I went to the opera last week with some friends; they bought the tickets and I paid for dinner. At the end of the evening I owed them over a hundred pounds (the opera was excellent but expensive!!). How was I going to pay them? I do not carry that much cash, nor would they want it. I do not carry a cheque book with me, nor would they want the hassle of paying it in. Ringing up the telephone banking service on my mobile would take too long, credit and debit cards could not help. My mobile banking app would do the trick but is fiddly, so I went home, logged on to my online banking system went through the menus and transferred the money.

What I would have liked to have done is say to my phone “please transfer one hundred and twenty pounds to my friend Joe Blogs from my current account now”. Obviously I would need to verify who I am and confirm that the banks automated assistant had understood me correctly.

The technology to do this is available from Nuance, the Dragon Naturally Speaking company, and is called Nina – Nuance Interactive Natural Assistant.

The bank trains Nina using a process similar to the training of call centre operators to understand the domain of online banking. The software is then intelligent enough to be able to parse my request into a standard money transfer request and then reply with a confirmation message “You want to transfer one hundred and twenty pounds to Joe Blogs today”. I could then just confirm that or say “Make that one hundred and thirty pounds on Monday” or “Can you confirm Joe’s bank details?”

If for some reason Nina could not understand what I wanted – for example I had forgotten that I had set up transfers with Joe’s wife Cynthia – then Nina could say “I am sorry I cannot find the account. I will transfer you to my supervisor”, Nina will then pass all the information she picked up and pass it to a human call centre operator and hopefully their superior intelligence will be able to resolve the situation “Did you mean Cynthia Blogs?”

Nina can carry out the whole transaction with just voice input and output but can also provide visual output and support typed input.

I think that I would be much more likely to use a Nina-based mobile app than a menu and forms based app. With my accessibility hat on the question is, would it help, or even be usable, by people with disabilities. People with a variety of disabilities such as vision impairment, physical dexterity, learning difficulties, English as second language (not exactly a disability but …), could all benefit. People who are deaf, hard of hearing or have vocal disabilities will be able to use a mixture of text and voice input and output.

In principle it would seem that Nina should be a boon to people with disabilities but this has not been tested in depth yet. There will be detailed design issues that will need to be thought through; for example how would a blind person want a statement of transactions to be presented? Would it be best to have all the transactions sent together and then let the text to speech software on the device read it out? This would enable the user to scan backwards and forwards quickly but they might prefer to say “next transaction” or “go back to the payment to Cynthia Blogs”.

Hopefully a Nina customer will soon step up to the accessibility issue and prove that it works or that it needs some extra functionality.

I have used a bank as an example but there are many other domains that could benefit. Requests like “How can my dog and I get from here to home?” or “I want to report that the street light outside my home is broken” should also be amenable to Nina’s charms.