Voice Recognition on Android & ProgramAB in MRL - Speech Recognition

LAST UPDATE: 05.01.2015 - New video!

24.04.2015 - renamed the service

12.05.2015 - source

19.11.2014

So this could be a step in chatting with our chatbots using our voice.

 

 

 

 

 

 

Using a custom App for Android I can chat to Alice using given voice (she gives text, but it's only an example).

Read further here -> http://myrobotlab.org/content/search-holy-grail-free-form-speech-recogni... .

Please excuse me for my bad English, the bad sound and the bad recording.

 

29.12.2014

After I got a little free-time, I thought the App could be easily improved. So I did, now it's got:

- settings

- > enter all required information

- - > IP

- - > service name

- > configure how the App should work

- - > mode

- - - > never: don't send anything to a remote instance (only recognition)

- - - > manual: select an entry yourself

- - - > automatic: auto-selecting the first entry

- - > display response: should the response be displayed?

- a bright design

- a small info

- all "Demo"'s & "Test"'s removed

- a (better) icon, I don't know, if it's better, but at least it is an icon

Downloads:

APK (Unsigned) -> ---[out of date, removed]---

ZIP (Sources)     -> ---[out of date, removed]---

 

31.12.2014

I finally decided to put a few data files in my GitHub repo. Now the App can check itself, if it's got an Update. In Addition I changed the signature. And the App is now hosted on GitHub, too.

New Download (with a few Bug-Fixes): https://github.com/LunDev/database/releases/tag/31.12.2014-AndroidVoiceR...

Sources will come soon (again)!

 

31.12.2014 #2

I implemented a simple SocketServer in a MRL service and added a Socket(Client) in the Android-App.

Routing the recognized text from Android to Python works. Now it can act fully like Sphinx (I think). Also you can send text from Python back to Android.

Starting the recognition from Python shouldn't be too hard.

Uploaded as a manual install here -> https://github.com/LunDev/database/releases/tag/31.12.2014-AndroidVoiceR...

 

31.12.2014 #3

The Recognition is now startable from Python, allowing a continous conversation, (sometimes you need to start it again, because it didn't understood you, but that's only sometimes).

I let him say, what I talked, allowing some really funny conversations, when Google didn't exactly get's what I was trying to say, for example:

I: kinds

G: Knights

I: I love them

G: I lost them

I: Really

G: Wheetly

I: Are you sure

G: I have to walk

I uploaded the build for tomorrow, so you can install it using the update-function, or here ->

https://github.com/LunDev/database/releases/tag/01.01.2015-AndroidVoiceR...

INSTRUCTIONS

Take this Python file as a example


#file : home/MaVo/AndSpeRec-Speech.py edit raw
#AndroidSpeechRecognition combined with Speech
from java.lang import String

#creating services
asr = Runtime.start("asr", "AndroidSpeechRecognition")
speech = Runtime.start("speech", "Speech")

#recognition-def
def heard(data):
 print data
 speech.speakBlocking(data);
 #asr.sendToClient(data);
 asr.startRecognition();

#adding listener
asr.addListener("recognized", python.name, "heard", String().getClass());

Start the Android-App, switch it socket mode, go back, there should be displayed Connection verified, if not, restart the App.

Feel free to speak!

Aviable from 1.0.96 up

TODO: Sources, details

 

01.01.2015

Fixed bugs I made with the previous version. Working from 1.0.97 up.

EDIT: Fixed one more bug in the App, updated it on GitHub

 

05.01.2015

I thought it would be an good idea to show off the AndroidVoiceRecognition-Service & Android-App and explain them a bit. So here is a (way too long) video:

Somehow I sound a bit sleepy, although I didn't was. Before you fall asleep, close the video!

 

The used script:


#file : home/MaVo/AndSpeRec-Speech-ProgramAB.py edit raw
#AndroidSpeechRecognition combined with Speech & ProgramAB
from java.lang import String
 
#creating services
asr = Runtime.start("asr", "AndroidSpeechRecognition")
speech = Runtime.start("speech", "Speech")
pab = Runtime.start("pab", "ProgramAB")
pab.startSession()
 
#recognition-def
def heard(data):
 print data
 resp = pab.getResponse(data)
 print resp
 #speech.speakBlocking(resp)
 #asr.sendToClient(data)
 asr.startRecognition()
 
#adding listeners
asr.addListener("recognized", python.name, "heard", String().getClass())
pab.addTextListener(speech)

24.04.2015

Renamed the service form AndroidVoiceRecognition to AndroidSpeechRecognition, because Google gave some small hints that they try to implement a Voice-Recognition ( & the service is recognizing speech not voice).

12.05.2015

Uploaded the sources of a version that will be (hopefully) published in the next few days.

-> https://drive.google.com/file/d/0B3Q0--r6Z_6Vc3VkejZSNWIwZlU/view?usp=sh...


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
GroG's picture

Great demo MaVo, lot's of

Great demo MaVo, lot's of potential here.  I'm impressed with the speed of Google's recognition, and curious how Gael might utilize it.  He uses a headset for noise cancellation.  He could use the same thing plugged into his phone. 

He requested and I put in the confirmations a long time ago.  But he could take them out with this.  The confirmations now are on the phone, and tapping validates them.  Then combine that with the new AIML oob driven commands...

Lots of fun !

MaVo's picture

Thanks! And your right in

Thanks!

And your right in every point.

The future looks good! ; )

GroG's picture

Hi MaVo Install Worky

Hi MaVo

Install Worky !

Setup Screen Worky !

Voice Worky !

 
 
Today I got a nasty surprise that Mcafee Firewall was installed without me knowing it on the laptop - its currently blocking any inbound conenctions GRRRR !
 
So I could not actually get it to work until I figure away around the obnoxious software...
 
But great work MaVo !
 
I might install MRL on a different machine so I may continue...
 
You mentioned you needed "call backs" ?  A simple socket server which gets returned MRL messages?
 
MaVo's picture

Hey,I'm glad you got the

Hey,

I'm glad you got the basic-function working!

The pictures look good, thank you, I thought already I have to take some ; ) .

McAffe is a problem for itself, I had a trial-version pre-installed, too. At least it informs you, that it blocked something. Then you try to disable it, McAffee says: "You need to reboot for this.", You reboot and have 1h to do what you want (without McAffee) and after an hour, McAffee automatically reboots. I still love blockers, which are "learning", what kind of programs it should block and what not. (It ended up that I removed it. I didn't liked it before, but now I hate it ; ).)

call-backs to Android: I think I'll use a custom socket(server) for now. A MRL intergrated server with "MRL's kind of messages" would be better, but also a lot more complex (at least I believe it is).

Anyway, good luck with your fight against the "nice virus scanner"!

GroG's picture

Another Very Cool Service MaVo !

It enjoyable to see you work so quickly and produce such useful services !
I haven't tried it yet, but I will ...

If I might offer a couple small suggestions - I would have submitted a pull request? if I knew how in git (is that right?)  I did not want to edit directly, because I find the merging in git a bit of a pain (more so than svn)

Anyway I send you some comments and we'll see what you think..

  • Use log and not System.out -  with log you log at a level so the amount and details of the logging can be modified during runtime... System.out can not .. Also the destination of logging can be modified during runtime for log (e.g. logfile, console, socket, etc...) 
  • you start the new listening thread in startService - you should probably shutdown the listening thread in stopService   - instead of while (true) {  - have a boolean var to shutdown gracefully while (listening) {
  • On exception catching I will catch everything with catch(Exception e) ... only if I handle different exceptions differently (very rare) will I break it out.  So, most of the time its just

    catch(Exception e) { Logging.logException(e); } .. this will dump the stack trace and log the error

    } catch (IOException | ClassNotFoundException ex) <- multi catch is not allowed on some JVMs
    this can be solved with the simple :

    catch(Exception e) { Logging.logException(e); }

 

I'm excited your making so many great services !  Keep up the excellent quality work !

MaVo's picture

Pull requests in git are, as

Pull requests in git are, as far as I know, mostly for people, who doesn't belong tho the project, but want to share a fix or something other code-based. Also it is used for commits/changes of project-members, who aren't completely sure, if their change is good.

Ya, I should be using log, but I mostly start the service in NetBeans as a new Project and migrate it later into a myrobotlab-service in eclipse and am just to lazy to change the message-out-line. I changed it now for all logs I did (every service I wrote).

Haven't thought of stopping the thread, since it is terminating when it found a client, but yes I should (and do now) and I shut down the client thread (if there is one).

Logging exception is the same topic as above, too lazy too change the outgoing-line (now it is for all projects). I didn't know that multicatch isn't supported on all Java Virtual Machines, learned something new today!

If you find more points in the code, just change it, there isn't much to break (yet).

I'm trying to follow the high quality you set with the framework, it just got a push-up today ; )

GroG's picture

Yeah I figured that is the

Yeah I figured that is the point for Pull requests ..    I'm still fighting with the way git handles merging, but I guess it will be better in time...

Thanks for the very high complement .. coming from you that means much !

hairygael's picture

Great work Ma Vo and super

Great work Ma Vo and super progress!!

This is very promissing, I will try to follow your tutorial to see if I can get results!

 

moz4r's picture

I really need this, perfect

I really need this, perfect inside noisy environment and/or without internet connection.
I will fork your project. Thank for your work MaVo