Peak-A-Boo Tutorial showing voice control and image template matching

Started this post to help us get an idea of what a very cool demo would be utiliizing voice control, servos, saving images, and image template matching in MRL.

The general tutorial (so far) should be a game of Peek-A-Boo 

The instructor has a scene of simple object.

The instructor tells the pan/tilt kit to look away

It looks away

The instructor asks which object is gone

the robot tells which is gone


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Alessandruino's picture

solved GroG :)

Problem solved :)

GroG's picture

Heh, we should do this ;)

Heh, we should do this ;)

Alessandruino's picture

test

 

from java.lang import String
from org.myrobotlab.service import Speech
from org.myrobotlab.service import Sphinx
from org.myrobotlab.service import Runtime
 
# create ear and mouth
ear = Runtime.createAndStart("ear","Sphinx")
mouth = Runtime.createAndStart("mouth","Speech")
opencv = Runtime.createAndStart("opencv","OpenCV")
opencv.addFilter("pdown","PyramidDown")
opencv.setDisplayFilter("pdown")
opencv.capture()
mouth.setLanguage("it")
# start listening for the words we are interested in
ear.startListening("hello robot|take photo")
 
 
# set up a message route from the ear --to--> python method "heard"
ear.addListener("recognized", python.name, "heard", String().getClass()); 
 
def heard():
      data = msg_ear_recognized.data[0]
      print "heard ", data
      if (data == "hello robot"):
         mouth.speak("ciao Alessandro.") 
      elif (data == "take photo"):
           opencv.recordSingleFrame(True)
           mouth.speak("foto scattata")
 
ear.attach("mouth")
GroG's picture

To get the file name you

To get the file name you should be able to do this..

 

photoFileName = opencv.recordSingleFrame(True)

Alessandruino's picture

It gives me None

 

Hey GroG, I've modified the script in this, but it returns "None" as file name, when i ask to print it..
 
 
 
from java.lang import String
from org.myrobotlab.service import Speech
from org.myrobotlab.service import Sphinx
from org.myrobotlab.service import Runtime
 
# create ear and mouth
ear = Runtime.createAndStart("ear","Sphinx")
mouth = Runtime.createAndStart("mouth","Speech")
opencv = Runtime.createAndStart("opencv","OpenCV")
opencv.addFilter("pdown","PyramidDown")
opencv.setDisplayFilter("pdown")
opencv.capture()
mouth.setLanguage("it")
# start listening for the words we are interested in
ear.startListening("hello robot|take photo")
 
 
# set up a message route from the ear --to--> python method "heard"
ear.addListener("recognized", python.name, "heard", String().getClass()); 
 
def heard():
      data = msg_ear_recognized.data[0]
      print "heard ", data
      if (data == "hello robot"):
         mouth.speak("ciao Alessandro.") 
      elif (data == "take photo"):
           opencv.recordSingleFrame(True)
           photoFileName = opencv.recordSingleFrame(True)
           print "name file is" , photoFileName
           mouth.speak("foto scattata")
 
ear.attach("mouth")

 

Alessandruino's picture

mmm....

 

here is the modified piece 
 
 
def heard():
      data = msg_ear_recognized.data[0]
      print "heard ", data
      if (data == "hello robot"):
         mouth.speak("ciao Alessandro.") 
      elif (data == "take photo"):
           opencv.recordSingleFrame(True)
           photoFileName = opencv.getLastRecordedFrameFileName
           print "name file is" , photoFileName
           mouth.speak("foto scattata")
 
 
this is python print : name file is <bound method org.myrobotlab.service.OpenCV.getLastRecordedFrameFileName of org.myrobotlab.service.OpenCV@280c7ab0>
GroG's picture

It's a method .. not a

It's a method .. not a field 

So it has to be with parens

photoFileName = opencv.getLastRecordedFrameFileName()

GroG's picture

Saving AVI problems

So when you stop, then close (make sure MRL is dead) - you should be able to play the file..   This is how it's always behaved apparently.   

The reason is the library I am using (JavaCV) is not thread-safe.   And the thread doing the capturing is not the same thread which stops the capture ;P

I can fix this.. just will take some time...

GroG's picture

Fixed !

No really I really really fixed it this time ;D

So its back to your original script - I got rid of getLastSingleFrame ...

Now 

photoFileName = opencv.recordSingleFrame(True)

blocks so that - it will continue when it has the correct filename