Third MRL tutorial -> take a photo (webcam) and send it by email using VOICE CONTROL (speech recognition and Text-to-speech)

Services: 

 

 

This tutorial shows how to take a photo (webcam) and send it by email using VOICE CONTROL...

WARN : There are 3 version of the python script

1) simple one - only take a photo using voice

2) full version - it requires a gmail address-take a photo + send it by email (SEE THE TUTORIAL)

3)full version ITA - for CIX, italian users, or who wants to listen the computer talking in italian :D

Good vision :D

 


#file : home/Alessandruino/Voice.takephoto.py edit raw
from java.lang import String
from org.myrobotlab.service import Speech
from org.myrobotlab.service import Sphinx
from org.myrobotlab.service import Runtime

# create ear and mouth
ear = Runtime.createAndStart("ear","Sphinx")
mouth = Runtime.createAndStart("mouth","Speech")
opencv = Runtime.createAndStart("opencv","OpenCV")
opencv.addFilter("pdown","PyramidDown")
opencv.setDisplayFilter("pdown")
opencv.capture()
# start listening for the words we are interested in
ear.startListening("hello robot|take photo")


# set up a message route from the ear --to--> python method "heard"
ear.addListener("recognized", python.name, "heard"); 

def heard(data):
      print "heard ", data
      if (data == "hello robot"):
         mouth.speak("Hi Alessandro.") 
      elif (data == "take photo"):           
           photoFileName = opencv.recordSingleFrame()
           print "name file is" , photoFileName
           mouth.speak("photo taken")

ear.attach("mouth")



#file : home/Alessandruino/OpenCV.Sphinx.takePhotoSendEmail.py edit raw
from java.lang import String
from org.myrobotlab.service import Speech
from org.myrobotlab.service import Sphinx
from org.myrobotlab.service import Runtime
import smtplib
from email.MIMEMultipart import MIMEMultipart
from email.MIMEBase import MIMEBase
from email.MIMEText import MIMEText
from email import Encoders
import os

gmail_user = "youremailadress@gmail.com"
gmail_pwd = "YOURPASSWORD"



# create ear and mouth
ear = Runtime.createAndStart("ear","Sphinx")
mouth = Runtime.createAndStart("mouth","Speech")
opencv = Runtime.createAndStart("opencv","OpenCV")
opencv.addFilter("pdown","PyramidDown")
opencv.setDisplayFilter("pdown")
opencv.capture()
# start listening for the words we are interested in
ear.startListening("hello robot|take photo|send email")


# set up a message route from the ear --to--> python method "heard"
ear.addListener("recognized", python.name, "heard"); 

def mail(to, subject, text, attach):
            msg = MIMEMultipart()

            msg['From'] = gmail_user
            msg['To'] = to
            msg['Subject'] = subject
 
            msg.attach(MIMEText(text))

            part = MIMEBase('application', 'octet-stream')
            part.set_payload(open(attach, 'rb').read())
            Encoders.encode_base64(part)
            part.add_header('Content-Disposition',
                        'attachment; filename="%s"' % os.path.basename(attach))
            msg.attach(part)

            mailServer = smtplib.SMTP("smtp.gmail.com", 587)
            mailServer.ehlo()
            mailServer.starttls()
            mailServer.ehlo()
            mailServer.login(gmail_user, gmail_pwd)
            mailServer.sendmail(gmail_user, to, msg.as_string())
            # Should be mailServer.quit(), but that crashes...
            mailServer.close()

def heard(data):
      print "heard ", data
      if (data == "hello robot"):
         mouth.speak("Hi Alessandro.") 
      elif (data == "take photo"):           
           global photoFileName
           photoFileName = opencv.recordSingleFrame()
           print "name file is" , photoFileName
           mouth.speak("photo taken")
           mouth.speak("if you want i could send the photo by email")
      elif (data == "send email"):
           mouth.speak ("Alessandro I am sending you an email")
           mouth.speak ("I will inform you when I am done")
           mail("ReceiverEmailAdress@gmail.com",
             "Hello from python!",
             "This is an email sent with python",
             photoFileName)
           mouth.speak("I am done")
           mouth.speak("photo sent by email")

ear.attach("mouth")



#file : home/Alessandruino/OpenCV.Sphinx.takePhotoSendEmailItalian.py edit raw
from java.lang import String
from org.myrobotlab.service import Speech
from org.myrobotlab.service import Sphinx
from org.myrobotlab.service import Runtime
import smtplib
from email.MIMEMultipart import MIMEMultipart
from email.MIMEBase import MIMEBase
from email.MIMEText import MIMEText
from email import Encoders
import os

gmail_user = "youremailadress@gmail.com"
gmail_pwd = "password"



# create ear and mouth
ear = Runtime.createAndStart("ear","Sphinx")
mouth = Runtime.createAndStart("mouth","Speech")
opencv = Runtime.createAndStart("opencv","OpenCV")
opencv.addFilter("pdown","PyramidDown")
opencv.setDisplayFilter("pdown")
opencv.capture()
mouth.setLanguage("it")
# start listening for the words we are interested in
ear.startListening("hello robot|take photo|send email")


# set up a message route from the ear --to--> python method "heard"
ear.addListener("recognized", python.name, "heard"); 

def mail(to, subject, text, attach):
            msg = MIMEMultipart()

            msg['From'] = gmail_user
            msg['To'] = to
            msg['Subject'] = subject
 
            msg.attach(MIMEText(text))

            part = MIMEBase('application', 'octet-stream')
            part.set_payload(open(attach, 'rb').read())
            Encoders.encode_base64(part)
            part.add_header('Content-Disposition',
                        'attachment; filename="%s"' % os.path.basename(attach))
            msg.attach(part)

            mailServer = smtplib.SMTP("smtp.gmail.com", 587)
            mailServer.ehlo()
            mailServer.starttls()
            mailServer.ehlo()
            mailServer.login(gmail_user, gmail_pwd)
            mailServer.sendmail(gmail_user, to, msg.as_string())
            # Should be mailServer.quit(), but that crashes...
            mailServer.close()

def heard(data):
      print "heard ", data
      if (data == "hello robot"):
         mouth.speak("ciao Alessandro.") 
      elif (data == "take photo"):           
           global photoFileName
           photoFileName = opencv.recordSingleFrame()
           print "name file is" , photoFileName
           mouth.speak("foto scattata")
           mouth.speak("se vuoi posso inviarti la foto per email")
      elif (data == "send email"):
           mouth.speak ("Alessandro ti sto inviando la mail")
           mouth.speak ("ti avviso appena ho finito")
           mail("youremailadress@gmail.com",
             "Hello from python!",
             "This is a email sent with python",
             photoFileName)
           mouth.speak("ho finito")
           mouth.speak("foto inviata per posta elettronica")

ear.attach("mouth")

RAJA SEGAR R S's picture

Hi need some basic ideas

Hi boss i saw this tutorial..need to clarify some basic ..concepts..were to start on with this nice project...wat hard wares are needed for this proj when i run this code i get some errors...can u post a detail tutorial on how to work on this proj :)

 

Alessandruino's picture

Hi man.... There are some

Hi man.... There are some steps to do in order to make it worky... Let's start !!

I advise you to start with the first script

HARDWARE needed : 

  • Computer
  • webcam
  • microphone
  • speakers

Software (MRL services required):

  • OpenCV
  • Sphinx
  • Speech

So in order to make it works, install and test that services...when we are sure that services are working properly, we'll start the script... Let us know how it proceeds!!

 

RAJA SEGAR R S's picture

Thank You

Hi boss :)

Thank you so much for ur guidance ...Got it perfect as u said..realy superb to work on with MRL...i want to learn more ..pls HELP me..were to start on with..??

THANK YOU for ur HELP :)

 

eddsjohn's picture

Hi buddy, I'm a starter on

Hi buddy,

I'm a starter on this framework and I'm still get in touch with the tool. 
I found your video but I can't find your source codes. 

Do you still have them?

 

Best Regards,

Edward

Alessandruino's picture

Done Edward...sorry for the

Done Edward...sorry for the delay !

Gingi's picture

domanda

salve,

volevo sapere se era normale che a volte  il computer risponda senza parlare o che ripeta la frase

"hi alessandro " in continuazione

grazie

 

 

Alessandruino's picture

Hi Gingi.... Adding the

Hi Gingi....

Adding the confirmation request would help with false-matchings...

Add these 2 lines to your script after the ear.startListening stuff to add confirmation and negations
In that way mrl will repeat the word
 
ear.addComfirmations("yes", "correct", "yeah", "ya")
ear.addNegations("no", "wrong", "nope", "nah")
 
Let us know
Alessandro
Gingi's picture

Hi Alessandruino, Grazie al

Hi Alessandruino,

Grazie al tuo consiglio adesso sembra fuzionare bene o meglio su un tempo prolungato di prova di domande e risposte solo una volta mi ha ripetuto "hi Alessandro",  un altro problema che riscontro

è che se dico hello world c'è la buona possibilità che mi risponda "hi Alessandro", ho pensato che forse il problema sia del microfono(ps3eye) secondo te con la kinect miglioro?

 

 

massimo's picture

MYROBOTLAB DOES NOT SPEAK

Hello

I used the script on this page, but I have problems with the voice, in fact everything works , except the sound.I do not hear the voice of myrobotlab.
if you can help me  !!!!!
Mats's picture

Speech service

The prefered Speech service now is MarySpeech. 

So if you replace the line:

mouth = Runtime.createAndStart("mouth","Speech")

with

mouth = Runtime.createAndStart("mouth","MarySpeech")

you should be able to hear the sound.

massimo's picture

Voice mail

thanks for the help all works well there is only one problem the sentences that feel are one on the other, i would like to delay the time between one sentence and another

Mats's picture

Speak blocking

Just replace

mouth.speak

with

mouth.speakBlocking

at several places.