Tracking Results

2017.09.04 Update - 
I found a few small bugs, but as Juerg reports there probably is some sort of race condition which messes things up when doing the initial connect.  I have "seen" the publishPortNames null pointer exception - but its elusive.  I have a pan-tilt kit which was attached to my puter over night.  This morning, I was very surprised to see the port missing !  Arduino IDE could not see it either. Unplugging and plugging the cable back in fixed the issue.  But leaves me wondering .. hmmmm.  Smells of a hardware or operating system driver issue to me...



Tracking is worky for me ..  but I used a small pan-tilt-kit  (arduino + 2 servos + usb camera)

I also did not use a script but have been using the "main" method in the Tracking service, I created an equivalent script.  I could only get the Sarxos frame grabber to work with my usb camera - all the rest could only run my integrated camera on the laptop and not the pan-tilt kit.  (mileage will vary) ;)

#file : home/GroG/ edit raw
useVirtualArduino = True;

xPin = 9;
yPin = 6;
arduinoPort = "COM5";
cameraIndex = 0;
# using Sarxos for usb webcam, the other frame grabbers only worked on my integrated camera
frameGrabberType = "org.myrobotlab.opencv.SarxosFrameGrabber";

Runtime.start("gui", "SwingGui");

if useVirtualArduino:
  virtual = Runtime.start("virtual", "VirtualArduino");

t01 = Runtime.start("t01", "Tracking");
x = t01.getX();
# invert if necessary
# x.setInverted(True);

y = t01.getY();
# invert if necessary
# y.setInverted(True);

t01.connect(arduinoPort, xPin, yPin, cameraIndex);
opencv = t01.getOpenCV();
# noticed some swing display issues - I don't think Sarxos gets updated to display

# not sure if necessary - but get things to settle for 3 seconds
# before starting tracking
# do lk optical point tracking
# t01.startLKTracking();
# do face tracking

CheekyMonkey's (aka Acapulco Rolf) version for I2C

#file : home/CheekyMonkey/ edit raw
# A script to test tracking on the Raspberry Pi driving servos with the AdaFruit16ServoDriver service
# as at mrl development build version 2489
# a mashup of code taken from Mats:
# and also from Grog:

import time
from org.myrobotlab.opencv import OpenCVFilterPyramidDown

xPin = 0;
yPin = 1;

arduinoPort = "/dev/ttyAMA0";

cameraIndex = 0;

#start an AdaFruit16C I2C servo driver instance
adaFruit16c3 = Runtime.createAndStart("AdaFruit16C3","Adafruit16CServoDriver")

#start a Raspberry Pi instance
raspi = Runtime.createAndStart("RasPi","RasPi")

#attach the AdaFruit16C I2C servo driver to the Raspberry Pi

#set the frequency for the AdaFruit16C I2C servo driver to 50 Hz

#start and connect a Virtual Arduino instance
virtual = Runtime.start("virtual", "VirtualArduino");

#start a tracker instance
tracker = Runtime.start("tracker", "Tracking");
x = tracker.getX();
# invert if necessary
# x.setInverted(True);

y = tracker.getY();
# invert if necessary
# y.setInverted(True);

tracker.connect(arduinoPort, xPin, yPin, cameraIndex);

#small delay here to resolve servo/attach glitch

#detach x and y servos from Virtual Arduino

#attach x and y servos to AdaFruit servo driver


#adjust PID values to suit
pid = Runtime.start("","Pid")"tracker.x", 5.0, 1.0, 0.1);"tracker.y", 20.0, 1.0, 0.1);
#pid.setPID("x", 10.0, 1.0, 0.1);
#pid.setPID("y", 50.0, 1.0, 0.1);

#opencv = tracker.getOpenCV();

opencv = Runtime.start("tracker.opencv","OpenCV")
pid.setPID("x", 5.0, 1.0, 0.1);
pid.setPID("y", 5.0, 1.0, 0.1);


# additional PyramidDown filter for improved framerate on the Pi (~15 fps)
PreFilterPyramidDown = OpenCVFilterPyramidDown("PreFilterPyramidDown") 


# do lk optical point tracking
# tracker.startLKTracking();

# do face tracking

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
moz4r's picture

My assistant testing inmoov

My assistant testing inmoov things !

Core seem worky

I use setMinMax instead of map(0,180,x,y) because I had a lot of pid errors.

I notice this issue :


no worky, no errors, but tracking did not start


this is ok. seem we cannot use 2 tracking at same time ?


GroG's picture

Thanks for investigating

Thanks for investigating moz4r .

Cute girl detector works ! :)

In the past 2 trackings worked at the same time.

When faceDetect was running, the pid controllers of each tracking had different values, such that, the eyes QUICKLY would move, and the head would more SLOWLY track ... 

It would be nice to get this to work as it did ...

I'll start trying with the InMoov service and see what happens..  I'll start thinking on how to make a pan-tilt inside a pan-tilt emulating head/eyes tracking in InMoov

GroG's picture

Posting Comment on behalf of Juerg

tried again to follow program flow during the connection and saw, that some events only get logged with activated Debug log level.
I am still a bit confused that we have a log entry
  waiting for boardInfo lock.....
then we get a number of onByte events with values
  10 19 8 0 12 5 29
next is in my opinion a valid version message
  170 9 3 56 1 255 255 25 83 0 0
which is however logged as version null (might be a timing issue and we are faster with the log then with setting the version number)? 
and therafter the message
  waited 885 ms for Arduino ArduinoCom3 to say hello
the next version message received logs then "goodtimes" and we are fine.
not sure but could we - while having version == null - wait for the first onByte with value 170 before trying to interprete message content?
It would at least get rid of the somehow ugly message errors during the connection phase.
Acapulco Rolf's picture

with the Raspberry Pi

with the Raspberry Pi 3

code  here :



GroG's picture

Mona ! .. "can't take my eyes

Mona ! .. "can't take my eyes off of you ... it just too good to be true..."