Tracking

javadoc

Tracking Service is used to find and track objects.  This service typically runs as a closed loop with feedback.  

 

The video above shows a Tracking service being started by a Python script.   Copy and paste the script below, adjust values correctly, then execute.  You may set tracking points through the GUI by clicking on the video, or set them programmatically in the script.

A common setup might include input such as LKOptical points from the OpenCV service.  The x and y position of the point would be sent two 2 PID controllers. The output of the PID controllers would be used to adjust the two Servos in a pan/tilt kit, which in turn would move the camera to follow the input.


#file : service/Tracking.py edit raw
# a minimal tracking script - this will start all peer
# services and attach everything appropriately
# change parameters depending on your pan tilt, pins and
# Arduino details
# all commented code is not necessary but allows custom
# options

port = "COM19"   #change COM port to your own port
xServoPin = 13   #change this to the right servo pin if needed, for inmoov this is right
yServoPin = 12   #change this to the right servo pin if needed, for inmoov this is right

tracker = Runtime.start("tracker", "Tracking")
gui = Runtime.start("gui", "SwingGui")
opencv=tracker.getOpenCV()

# set specifics on each Servo
servoX = tracker.getX()
servoX.setMinMax(30, 150)  #minimum and maximum settings for the X servo
# servoX.setInverted(True) # invert if necessary

servoY = tracker.getY()
servoY.setMinMax(30, 150)  #minimum and maximum settings for the Y servo
# servoY.setInverted(True) # invert if necessary

# changing Pid values change the 
# speed and "jumpyness" of the Servos
pid = tracker.getPID()

# these are default setting
# adjust to make more smooth
# or faster

pid.setPID("x",5.0, 5.0, 0.1)
pid.setPID("y",5.0, 5.0, 0.1)

# optional filter settings
opencv = tracker.getOpenCV()

# not for you, it's for test
#virtual=1
if ('virtual' in globals() and virtual):
  virtualArduino = Runtime.start("virtualArduino", "VirtualArduino")
  virtualArduino.connect(port)
  
# connect to the Arduino ( 0 = camera index )
tracker.connect(port, xServoPin, yServoPin, 0)

if ('virtual' in globals() and virtual):
  opencv.stopCapture()
  opencv.setMinDelay(500)
  opencv.setFrameGrabberType("org.bytedeco.javacv.FFmpegFrameGrabber")
  opencv.setInputSource("file")
  opencv.setInputFileName("resource/OpenCV/testData/monkeyFace.mp4")

gui.undockTab("tracker.opencv")
opencv.broadcastState();
sleep(1)

# Gray & PyramidDown make face tracking
# faster - if you dont like these filters - you
# may remove them before you select a tracking type with
# the following command
# tracker.clearPreFilters()

# diffrent types of tracking

# lkpoint - click in video stream with 
# mouse and it should track
# simple point detection and tracking
# tracker.startLKTracking()
tracker.faceDetect()
sleep(10)

# track and recognize faces
# you need to learn at least 2 faces to get it worky
if ('virtual' in globals() and virtual):opencv.captureFromImageFile("resource/OpenCV/testData/rachel.jpg")
tracker.faceDetect(True)
# scans for faces - tracks if found
# tracker.findFace()

 

Examples:


#file : service/Tracking.py edit raw
# a minimal tracking script - this will start all peer
# services and attach everything appropriately
# change parameters depending on your pan tilt, pins and
# Arduino details
# all commented code is not necessary but allows custom
# options

port = "COM19"   #change COM port to your own port
xServoPin = 13   #change this to the right servo pin if needed, for inmoov this is right
yServoPin = 12   #change this to the right servo pin if needed, for inmoov this is right

tracker = Runtime.start("tracker", "Tracking")
gui = Runtime.start("gui", "SwingGui")
opencv=tracker.getOpenCV()

# set specifics on each Servo
servoX = tracker.getX()
servoX.setMinMax(30, 150)  #minimum and maximum settings for the X servo
# servoX.setInverted(True) # invert if necessary

servoY = tracker.getY()
servoY.setMinMax(30, 150)  #minimum and maximum settings for the Y servo
# servoY.setInverted(True) # invert if necessary

# changing Pid values change the 
# speed and "jumpyness" of the Servos
pid = tracker.getPID()

# these are default setting
# adjust to make more smooth
# or faster

pid.setPID("x",5.0, 5.0, 0.1)
pid.setPID("y",5.0, 5.0, 0.1)

# optional filter settings
opencv = tracker.getOpenCV()

# not for you, it's for test
#virtual=1
if ('virtual' in globals() and virtual):
  virtualArduino = Runtime.start("virtualArduino", "VirtualArduino")
  virtualArduino.connect(port)
  
# connect to the Arduino ( 0 = camera index )
tracker.connect(port, xServoPin, yServoPin, 0)

if ('virtual' in globals() and virtual):
  opencv.stopCapture()
  opencv.setMinDelay(500)
  opencv.setFrameGrabberType("org.bytedeco.javacv.FFmpegFrameGrabber")
  opencv.setInputSource("file")
  opencv.setInputFileName("resource/OpenCV/testData/monkeyFace.mp4")

gui.undockTab("tracker.opencv")
opencv.broadcastState();
sleep(1)

# Gray & PyramidDown make face tracking
# faster - if you dont like these filters - you
# may remove them before you select a tracking type with
# the following command
# tracker.clearPreFilters()

# diffrent types of tracking

# lkpoint - click in video stream with 
# mouse and it should track
# simple point detection and tracking
# tracker.startLKTracking()
tracker.faceDetect()
sleep(10)

# track and recognize faces
# you need to learn at least 2 faces to get it worky
if ('virtual' in globals() and virtual):opencv.captureFromImageFile("resource/OpenCV/testData/rachel.jpg")
tracker.faceDetect(True)
# scans for faces - tracks if found
# tracker.findFace()

Alessandruino's picture

Here is my "NIGHT

Here is my "NIGHT TRACKING"...using a torch to make light on R2D2 :D

GroG's picture

Nice !

R2D2 will not escape !
I wonder if a laser would set a good tracking point, I imagine it would depend on camera and laser details...

GroG's picture

Tested them both on incubator

Tested them both on incubator ..  they worked but there was one "funny" part..

The incubator's hardware is horrible - very slow, over burdened, not optimized at all - very old....

Anyway I had some problems where the Servos would not be attached after running the script - I believe the issue was opening the serial port takes too long... 
 

I solved it with a sleep(3) after the tracking.attach(arduino) .. but there should be a better way to do this..

After I did that it all woked, but it is very WONKY when you glue the rest position at 5 instead of 90 :(

GroG's picture

And Now With Error Messages !!!

Tested the second one now .. It works (with the latest MRL but you probably already Auto-Updated :) ..
Hmm.. this should probably be a warning not an error..   Look what gluing it wonky gets you !  It's interesting to see the whole thing rotate - so that it tries to get the point centered ..  PID IS AMAZING !

GroG's picture

Strong Worky !

Ok Alessandruino see if you can break them !  
Or actually if you can get them to work with "only" changing configuration values.
Deleting the .myrobotlab is perfectly acceptable, should work if you delete it or don't

I tested both on my Win64 laptop..  and as soon as the build is done, I'll be testing on the incubator too..

I was really surprised how much "more" work this took on the Tracking service (not the script) .. but there is much good from it.

The Tracking service is a "Composite type Service"  in that it does not control or produce data from hardware...  instead it manipulates other common services to produce the desired function.  I have for a long time been trying to find a pattern of how to successfully initialize a Composite Service.  And now, I'm pretty sure I have figured out a general pattern to follow.  

Other composite services like (InMoov & Cortex) should benefit greatly from this....

Try the 2 scripts - adjust the minimal amount of config - and let me know if they finally WORKY FOR YOU !

Thanks !  Making it (HARDER, BETTER, FASTER, STRONGER) :)

GroG's picture

Alrigtht, I added an invert()

Alrigtht, I added an invert() method to pid to flip it .. and direct() to make it direct again...
Did some tweaks in the PID & Tracking services, enabled MinMax to really work, changed name of second script to "manual" versus "safe"... and more stuff..

AND NOW I FIND THE BUILD SERVER IS DOWN ..  I need a robot to kick it - so it will start and I can get a build out :P

until then I'll have to wait until I get home

Alessandruino's picture

GAEL'S SERVOS ARE FINALLY SAFE !!! :D

Here is some pics about safety ranges

# make safe with

tracker.setXMinMax(60, 120)
tracker.setYMinMax(60, 120)

# save servos !

GroG's picture

No Worky <- is No Worky !

Hey Alessandruino,
Your No Worky is No Worky !

The 2  No-Worky log files you sent this morning contain no Servo errors .   I suspect this is because you have a -logToConsole in the myrobotlab.bat ? 

You need to remove it or you can add a file appender (file log) by selecting System->Logging->Type (and checking File) before you do the test.

Alessandruino's picture

OK...thanks GroG..

Ya..i have a -logToConsole in my myrobotlab.bat

GroG's picture

This doesn't work right yet

This doesn't work right yet :)  but I wanted to post it to show you can work with the eye that's created by default from the Tracker directly .. just another way..  

this adds the face detect & an extra pyramid down after the lk .. interesting results :P

 

 

tracker = Runtime.create("tracker","Tracking")
 
tracker.setRestPosition(90, 90)
tracker.setSerialPort("COM12")
tracker.setServoPins(13,12)
 
tracker.setCameraIndex(1)
tracker.startService()
tracker.trackPoint(0.5, 0.5)
 
tracker.eye.addFilter("pd1", "PyramidDown")
tracker.eye.addFilter("FaceDetect1", "FaceDetect")
 

 

 

 

Jaykumar Vaidya's picture

Object detection problem.Please help URENTLY!!!

 Hello Respected Sir,

I am new to myrobotlab, its a sincere request to please help in 'Quality detection of bottle' project.

I am UG Electronics Engg student, working on a project 'Quality detection plant'. Its a PLC based control plant which fills, holds and caps the bottle accordingly. I want to check quality parameters such as:

1) One or more bottles are slipped down on belt or not

2) Cap fixed or bottle went uncap

3)Level of liquid

There are more parameters to be done but the&se are urgent

I tried alot with Visual Studio 10/12 plus Opencv. Which never opens my internal webcamera/external.

 

I want to do this with myrobotlab seeing its various functionalites, I tried tracking service in 1.0.119, but as its containing alot stuffs such as servos, pid,etc even though arduino uno is connected by version number displayed 21 still I amnot able to get what should be the connection betwwen tracking.opencv, tracking.arduino & python code. I dont want to tracki, so no need of servos, as my cam is stable at 1 position. I just wanna test whether there is cap fixed or not?(cap having different color from vicinity & bottle), and te level of liquid as bottle is white transparent. These will contribute to 3 ouput signals from arduino to PLC CB280 conveying  1)wheteher bottles slipped, 2)cap fixed, 3)level below threshold mark 

 

 

Please guide us as soon as possible.

Thanks in advance

Jaykumar Vaidya

vaidyajaykumar@gmail.com

Jaykumar Vaidya's picture

also a replica of hardware in

also a replica of hardware in preview to software, a SCADA like thing is needed. I saw Webgui i think thats web SCADA?or will have to use GUI tab in myrobotlab?

Hardware ready, to be inspected by opencv img procesing

 

Thanks

JV

GroG's picture

I'd recommend you look at the

I'd recommend you look at the OpenCV service page

http://myrobotlab.org/service/OpenCV

If your really intrested, I'd recommend you start a new post, "clearly" describe your goals, and post some sample data (pictures of what you are trying to identify)

Then maybe you can generate interest, and illicit more help from people.

Pasqui's picture

Tracking test..no work

I'm new in myrobotlab, I downloaded the MRL last release, 1.0.1758, and upload the entire code for service tracking above, but I've errors like these:

File string, line 16, in moduleAttributError: org.myrobotlab.service.Servo objact has no attribute setPin at.org.pyton.core.Py.AttributeError(Py.java:205)....

Could you help me understand the problem? I have to start a project for tracking objects, but I can not proceed because of these problems!