FACE TRACKING USING :OPENCV, FACE DETECT FILTER, AN ARDUINO WITH A PAN TILT KIT

Face tracking using opencv + facedetect filter with my pan/tilt kit...

Facedetect filter is heavy, and webcam video flow delay from servos response...this leads to overshooting

GROG..i need a lighter facedetect filter.....video stream is  slow when i apply pirdown and face detect :(

Here is the video...

CHECK IT OUT !


#file : Tracking.face.py
from java.lang import String
from java.lang import Class
from java.awt import Rectangle
from org.myrobotlab.service import Runtime
from org.myrobotlab.service import OpenCV
from org.myrobotlab.opencv import OpenCVData
from com.googlecode.javacv.cpp.opencv_core import CvPoint;
from org.myrobotlab.service import OpenCV
from org.myrobotlab.service import Arduino
from org.myrobotlab.service import Servo
from time import sleep
 
 
xpid = Runtime.createAndStart("xpid","PID")
ypid = Runtime.createAndStart("ypid","PID")
xpid.setMode(1)
xpid.setOutputRange(-3, 3)
xpid.setPID(10.0, 0, 1.0)
xpid.setControllerDirection(1)
xpid.setSetpoint(160) # we want the target in the middle of the x
ypid.setMode(1)
ypid.setOutputRange(-3, 3)
ypid.setPID(10.0, 0, 1.0)
ypid.setControllerDirection(1)
ypid.setSetpoint(120)
 
arduino = Runtime.createAndStart("arduino","Arduino")
pan 	= Runtime.createAndStart("pan","Servo")
tilt	= Runtime.createAndStart("tilt","Servo")
arduino.setSerialDevice("COM3", 57600, 8, 1, 0)
sleep(4)
arduino.attach(pan.getName() , 12)
arduino.attach(tilt.getName(), 13)
global actservox
global actservoy
actservox = 90
actservoy = 90
 
 
# create or get a handle to an OpenCV service
opencv = Runtime.create("opencv","OpenCV")
opencv.startService()
# reduce the size - face tracking doesn't need much detail
# the smaller the faster
opencv.addFilter("PyramidDown1", "PyramidDown")
# add the face detect filter
opencv.addFilter("FaceDetect1", "FaceDetect")
 
def input():
 
    #print 'found face at (x,y) ', msg_opencv_publishOpenCVData.data[0].x(), msg_opencv_publish.data[0].y()
    opencvData = msg_opencv_publishOpenCVData.data[0]
    global x
    global y
    global sposx
    global sposy
    global posx
    global posy
    global servox
    global servoy
    global movex
    global movey
    if (opencvData.getBoundingBoxArray().size() > 0) :
     rect = opencvData.getBoundingBoxArray().get(0)
     posx = rect.x
     posy = rect.y
 
     w = rect.width
     h = rect.height
     sposx = (w/2)
     sposy = (h/2)
     x = (posx + sposx)
     y = (posy + sposy)
     print x
     print y
     xpid.setInput(x)
     xpid.compute()
     servox = xpid.getOutput()
     movex = (actservox + servox)
     global actservox
     actservox = movex
     ypid.setInput(y)
     ypid.compute()
     servoy = ypid.getOutput()
     global actservoy
     movey = (actservoy + servoy)
     actservoy = movey
     print 'x servo' , int(actservox)
     print 'y servo' , int(actservoy)
     pan.moveTo(int(actservox))
     tilt.moveTo(int(actservoy))
     return object
 
# create a message route from opencv to python so we can see the coordinate locations
opencv.addListener("publishOpenCVData", python.name, "input", OpenCVData().getClass());
 
# opencv.setCameraIndex(1)
 
opencv.capture()
 

Third MRL tutorial -> take a photo (webcam) and send it by email using VOICE CONTROL (speech recognition and Text-to-speech)

Services: 

 

 

This tutorial shows how to take a photo (webcam) and send it by email using VOICE CONTROL...

WARN : There are 3 version of the python script

1) simple one - only take a photo using voice

2) full version - it requires a gmail address-take a photo + send it by email (SEE THE TUTORIAL)

3)full version ITA - for CIX, italian users, or who wants to listen the computer talking in italian :D

Good vision :D

 

Helpful MyRobotLab Tips And Tricks

Installing, Starting and Releasing Services

"One Service to Rule them All ! "
The Runtime Service is the first service to start whenever MyRobotLab starts.  It has the capability of installing, starting and releasing all other services. You can see the services it can control under the runtime tab in the gui.  Some services require additional dependencies and have to be installed before they can be started.

Second MRL tutorial - tracking service with wiring diagram

Services: 

A tutorial that explain how to use tracking service in MRL...

Python Minimal script


#file : Tracking.minimal.py
# a minimal tracking script - this will start all peer
# services and attach everything appropriately
# change parameters depending on your pan tilt, pins and
# Arduino details
tracker = Runtime.create("tracker","Tracking")
 
# setXMinMax & setYMinMax (min, max)
# this will set the min and maximum
# values of the servos 
tracker.setXMinMax(10, 170)
tracker.setYMinMax(10, 170)
 
# set rest x,y
tracker.setRestPosition(90, 90)
# set serial port of the Arduino
tracker.setSerialPort("COM12")
# se the pan and tilt pins
tracker.setServoPins(13,12)
 
#change cameras if necessary
tracker.setCameraIndex(1)
 
# start the tracking service
tracker.startService()
 
# set a point and track it
# there are two interfaces one is float value
# where 0.5,0.5 is middle of screen
tracker.trackPoint(0.5, 0.5)
 
# the other is integers which are pixel location
# tracker.trackPoint(10, 120)
 
# don't be surprised if the point does not
# stay - it needs / wants a corner in the image
# to presist - otherwise it might disappear
# you can set points manually by clicking on the
# opencv screen
 


Python FULL AND SAFE script

#file : Tracking.manual.py
# we can create all the peer services for tracker manually
# giving them new names and different values
# then we can attach them. This will give us access to 
# change any of details of the other services. This must be 
# done BEFORE "starting" the tracking service.  
 
# create a tracking service
tracker = Runtime.create("tracker","Tracking")
 
# create all the peer services
rotation = Runtime.create("rotation","Servo")
neck = Runtime.create("neck","Servo")
arduino = Runtime.create("arduino","Arduino")
xpid = Runtime.create("xpid","PID");
ypid = Runtime.create("ypid","PID");
 
# adjust values
arduino.setSerialDevice("COM12")
eye = Runtime.create("eye","OpenCV")
eye.setCameraIndex(1)
 
# flip the pid if needed
# xpid.invert()
xpid.setOutputRange(-3, 3)
xpid.setPID(10.0, 0, 1.0)
xpid.setSetpoint(0.5) # we want the target in the middle of the x
 
# flip the pid if needed
# ypid.invert()
ypid.setOutputRange(-3, 3)
ypid.setPID(10.0, 0, 1.0)
ypid.setSetpoint(0.5)
 
# set safety limits - servos
# will not go beyond these limits
rotation.setPositionMin(50)
rotation.setPositionMax(170)
 
neck.setPositionMin(50)
neck.setPositionMax(170)
 
# here we are attaching to the
# manually created peer services
 
tracker.attach(arduino)
tracker.attachServos(rotation, 13, neck, 12)
tracker.attach(eye)
tracker.attachPIDs(xpid, ypid)
 
tracker.setRestPosition(90, 90)
 
tracker.startService()
tracker.trackPoint(0.5,0.5)
 

Wiring diagram

First MRL Tutorial in ITALIAN...

Services: 

A tutorial that explain how to install MRL and how to use Arduino and Servo 's services in order to move a servo...

Lucas Kanade Optical Tracking (OpenCV - LKOpticalTrack Filter)

Lucas and Kanade developed an algorithm which allows tracking of object through a video stream.  The algorithm requires less resources and is usually faster in response than many other forms of tracking.  

Pros :

  • low resource
  • quick response time
  • can track many points at once

Cons:

Using the MRLClient

Services: 

    

Hardware: 

No additional hardware is needed for this tutorial

The MRLClient is a client adapter which allows other Java programs to interoperate with MyRobotLab.  Although the myrobotlab.jar can be included and used in a Java program directly, it may sometimes be desirable to communicate over the network with an adapter.

The MRLClient jar is a small binary which uses the network to send and recieve messages to a running MyRobotLab instance.  Other network adapters could be created for languages besides Java, but currently only Java is supported.

Connecting an Android Phone to Arduino using Bluetooth

Services: 

Additional Software

Arduino IDEArduinoSerialBare.PDE, MyRobotLab.apk

Hardware: 
  • Smartphone running Android 2.3.3 or later
  • LED
  • 1K resistor
  • Arduino bluetooth board, or Arduino clone with bluetooth module

 

Legacy Video Tutorial

Video Tutorial #1 (Overview)