Hi
I have recently begin to play with myrobotlab to use with my inMoov and I'm really impressed with all the possibility this program allow. So I bow to the conceptor/developer of this program, it`s really an amazing work
This week-end I found out about the face recognizer filter for opencv and that will be very cool when it do this or that. I understand this is new stuff and this is still a lot to do
So, with a goal to understand better how myrobotlab is working, I have try to use that face recognition filter with my inMoov.
First I have need to grab what the last name the filter recognize so I add this

public String getlastRecognizedName(){

      return this.lastRecognizedName;

}

 

in OpenCVFilterFaceRecognizer.java and recompile myrobotlab

 

then start my inMoov script, set up the faceRecognizer filter, train a few face in it then lauch this small script

 

name="christian"

 

def recognizeFace():

  global name

  fr=opencv.getFilter("fr")

  lastReconizedName=fr.getlastRecognizedName()

  if name != lastReconizedName:

    name=lastReconizedName

    i01.mouth.speak("hello "+name)

    pinocchio.startSession(name)

    i01.mouth.speak("your town is  ")

    sleep(5)

    pinocchio.getResponse("what is my town")

    sleep(5)

  t=Timer(2.0,recognizeFace).start()

 

recognizeFace()

So, with that scrip, my inMoov can reconize some people, great them when he saw them and told in wich town they lives (that info is save in a programAB predicate file)
 
But I was still not able to set up the face recognizer filter with a python script
whit my PC camera, this is working really good, but with my inMoov eye, the faceRecognizer have much harder time to recognize and train faces. I must be very close to it to get it work, that probably because of the fish eye lens I have on the camera
Sorry, I can`t do a video to show this in action
 
 

kwatters

8 years 8 months ago

Ahoy Calamity!

  Thank you for taking the time to post your progress and reach out.  As you may have noticed MRL is about the community contributing and building a better platform for those who follow.  It sounds like your on the latests and greatest build!  You may be right, the fish eye lens my be interferring with some of the face recognition.  I have noticed that the face recognition is very sensitive to the rotation of the face to pick it up.  

As for the the detection of the last recognized face,  we can definitely add the additional method to get the last recognized face.  Heck, if you want. feel free to commit it :)  we love people contributing code :)  especially when it helps by making new things easier to build!

  I've recently started in on some refactoring of how ProgramAB works so it will better support holding down many conversations with many people at once.  I see you're already going down that path with the startSession call.  Great work!  

  I believe the plan going forward with OpenCV is to make it a bit more generic so that the face recognizer will "publishRecognizedFace"  so you can just have a callback in your code that will update whenver a new face is recognized. 

  Let me know, I'm happy to add the "getter" for the last recognized face, or if you prefer, you can push it yourself :) and then you'll be a true commiter and contributor to MRL :)

Welcome and thanks!  It's people like you and posts like this take help keep us all inspired to make MRL better for everyone!

 

-Kevin

 

Hi Kevin

Thanks for your wecome

of course a publishRecognizedFace will be much more better, but that still way beyond my knowlege :)
 
As I said, my goal was mostly to dig a bit in myrobotlab to understand it a little. 
 
I don`t know how to commit changes, so feel free to add it if you think it worth it. I know it`s just simple temporary stuff before the filter gets implemented better, but it work and the results is quite spectacular and other may want to play with it.
 
 

kwatters

8 years 8 months ago

Great feedback @calamity!  I just pushed a change for you to add the method to get the last recognized name.  I also added a method for  publishRecognizedFace  that you can subscribe to .. anytime a face is recognized, it will be called. 

I'm sure we'll want to change the behavior of this in the future, perhaps to only publish the recognized face if it's a new / different face.. but lets see how it goes for now...  It's a start.

Thanks!

https://github.com/MyRobotLab/myrobotlab/commit/1c88fbefe0ae15a8c90a79dbd6bcc9e9277f1842#diff-84b47d90773f37fc1c9c756db9c916f7R643

calamity

8 years 8 months ago

I have updated my script to be able to use WebKitSpeechRecognition and to take advantage of the programAB multi-session fonctionnability

Now, my inMoov will be able to answer to the last person he recognize.

 

webgui=Runtime.create("WebGui","WebGui")
webgui.autoStartBrowser(False)
webgui.startService()
 
#start speech recognition and AI
wksr=Runtime.createAndStart("webkitspeechrecognition","WebkitSpeechRecognition")
pinocchio = Runtime.createAndStart("pinocchio", "ProgramAB")
pinocchio.startSession("default", "pinocchio")
htmlfilter=Runtime.createAndStart("htmlfilter","HtmlFilter")
mouth=Runtime.createAndStart("i01.mouth","AcapelaSpeech")
wksr.addListener("publishText","python","heard")
pinocchio.addTextListener(htmlfilter)
htmlfilter.addTextListener(mouth)
 
opencv=Runtime.start("opencv","OpenCV")
opencv.setCameraIndex(1)
opencv.capture()
fr=opencv.addFilter("FaceRecognizer")
opencv.setDisplayFilter("FaceRecognizer")
fr.train()# it takes some time to train and be able to recognize face
 
def heard(data):
    lastName=fr.getLastRecognizedName()
    if((lastName+"-pinocchio" not in pinocchio.getSessionNames())):
        mouth.speak("Hello "+lastName)
        sleep(2)
    pinocchio.getResponse(lastName,data)