Ro-Bot-X and I are working together even though we are 2500 miles away !

Hey Look... I can see Ro-Bot-X's living room through the eye of the bot he's currently working on ! (Hopefully he won't mind me posting it :) )

It's a Chumby 1 and were getting MyRobotLab to run on it. 

It was a challenge and learning experience getting the OpenCV running on the Chumby.  It required cross compiling binaries on a Linux X86 PC --to--> ARM 9 with a tool chain.

So far, we have MRL running on it with OpenCV and Serial which means potentially it can us Ro-Bot-X's uBotuino to control Motors, Servos, read Sensors, etc.

I'm making two new Services in MRL.  A Chumby service which will interface with all of the hardware a Chumby has to offer and simplify/standardize it so other Services can use it.  And a ChumbyBot service which will contain the behaviors and personality of the little bot.

His board handles Motors a little differently than the 2 way's I am familiar with, so well figure that out and put it in the Motor service with configuration.  

I'm beginning to like the little Chumby.  It runs Linux and has support for USB, I2C, Video, Microphone, Stereo Speakers, Wifi, Touch-Screen, FM Radio, even an Accelerometer.. and for just $90 on Amazon or lower on ebay

We are just starting to connect the wires and pipes together, but I think the hardest part was the OpenCV support... 

What current works

  • Remote communication from 2500 miles away attached to my MRL instance
  • OpenCV webcam and remote control of vision filters (facedetection, color isolation, LKOptical tracking, etc)
  • Web control through an MRL applet
  • Remote GamePad control
  • TextToSpeech with sexy Audrey voice

Things to test

  • MRL/Sphinx4 Speech recognition
  • Audio Recording and PlayBack through MRL
  • HTTP getting info on the intertubes - jokes, daily news, shoutbox data etc.
  • Fancy Sound generation (Drums, Sax, Guitar etc through MRL & JFugue)
  • Play Chess? (Thanks Patrick :D)
  • Servos

Things to develop

  • Encapsulate Chumby's hardware in a easy to use Service
  • Behaviors and Personality for ChumbyBot
  • Augment Motor to handle uBotuino's method of control

We will make sure all the lower level stuff works well (Motors, Servos, Sensors, etc) then I believe Ro-Bot-X was interested in grabbing colored objects around the house (cleaning? dunno)

Wheeeee !

The frame rate of the camera is pretty lousy, but he said he got it for $5.. To reduce the lag as its shipped 2500 miles you can drop the color.

Here I tried isolating the red square out of his carpet...  Unfortunately, he has not mounted the thing on a base or camera on a servo.. but that will come shortly...



RoboScope - monitors IR Sharp sensor with 2500 mile test leads !

I was able to log the data coming from the IR Sensor in the SensorMonitor Service ..   Here are a few snapshots.  It works as  a great diagnostic tool, but the real nice part will be the ability to set "Alerts" where a sensor reaches a certain pre-defined (or even dynamic) level and fires off an event.  For example, 50 cm = Stop and look for open area.

The IR Sensor was on a servo.  The servo swings back and forth and in turn the range changes.  As the range changes I can see the value of the IR sensor coming back as a graph.  The dip is the change of range since part of the time, the IR sensor is aimed at a nearby wall.  The SensorMonitor can graph (and soon Alert) on multiple sensors at once.



The Detailed Nerdy Part:

The first step in this process is to get Java running on Chumby.
A great wiki is provided here 
However, It contains references to older versions of JamVM and GNU Classpath.  I do not believe that these versions support all the JNI necessary to communicate successfully to the hardware.

Our objective then becomes install the "latest" JamVM and GNU Classpath

Some other lessons I have learned through this process :

  • Sun embedded JVM for Arm (headless) - worked great and it was an incredibly easy install, but it is not open source and THE BIG strike against it - is there was no javac (compiler) which would make the compilation of rxtxSerial difficult !
  • OpenJDK - seems like a great project - it was originally just targeting PCs - it is huge and cross compiling it for the Chumby seemed like a exercise in extreme torture (please prove me wrong!)
  • Don't compile the wrong versions of JamVM and GNU Classpath - they are different projects, so only certain versions of each are compatible - the target we are going for is JamVM 1.5.3 and GNU Classpath 0.98
  • Don't compile install JamVM first ! - this only causes problems in the long run - JamVM will not run without a valid GNU Classpath already in place - yes I burned time with this.
  • Don't cross compile - another exercise in torture (please prove me wrong!)
  • Use JamVM 1.5.3 - at this time JamVM 1.5.4 is available, however, it will not compile on the Chumby - because it needs 40 megs of memory at least to do it - if your an expert in cross compiling you can get around this.  I am not an expert in cross compiling - so I'm sticking to 1.5.3
  • Don't attempt to bootstrap or compile GNU Classpath on the Chumby itself

The basic strategy will to get an ARM tool chain installed. This will allow us to compile code which runs on the ARM from a regular PC. Thankfully, the very helpful Chumby wiki has such information in step by step instructions (


A very popular way to send serial information through Java uses the RXTXComm.jar from .   MyRobotLab's Arduino Service uses this code to talk to Arduinos, and theoretically any microcontroller attached via USB-FTDI or serial port. Since Java must interface directly with the hardware in this situation, it will be necessary to build Native code with the cross compiler toolchain we just installed.


Unfortunately, there is a bug in the configure/build at the moment for rxtxcomm.

src/RS485Imp.c: In function 'Java_gnu_io_RS485Port_Initialize':
src/RS485Imp.c:96: error: 'UTS_RELEASE' undeclared (first use in this function)
src/RS485Imp.c:96: error: (Each undeclared identifier is reported only once
src/RS485Imp.c:96: error: for each function it appears in.)
Fortunately - someone has posted the solution (love that) here
In addition the configure and scripts need to be corrected to search for |1.6* Java to correctly set the path - otherwise a UnsatisfiedLinkError will occur.
mkdir rxtx
cd rxtx
./configure  --target=arm-linux --host=arm-linux --build=i386-linux

After all that labor I have successfully built a for Chumby -  which means Java programs can now use it to communicate over Chumby's serial port or USB-FDTI to a microcontroller.  There is also a librxtx - Parallel I2C Raw and RS485 libraries.  I have attached them to this node.

OpenCV + JavaCV + JavaCPP + MRL
This was rather "educational", and successful in the end.  
First a toolchain was necessary to compile all the parts on a Linux machine running a X86 processor (my home box).  Chumby has a convenient download for this here.  It's "real" name is arm2008q3. 
OpenCV was challenging to compile, in that the cmake build wants to get stuff from /usr/include (not a good idea).  I have had mixed feeling regarding cmake... but at this point I am beginning to like the gui manager.  After symbolically linking a new /usr/include and taking care of all the other dependencies and compiler specifics, I built it with very few options.  No GTK or GSTREAMER as I don't believe those are on the target Chumby.

JavaCV is a great interface and MRL uses it to get to OpenCV.  JavaCPP is the method which Samuel Audet uses to access the natively compiled OpenCV.  Samuel Audet manages both JavaCV & JavaCPP.  Although JavaCV supports a wide range of platforms, it does not support Chumby hardware.  With some help from Mr. Audet I compiled the necessary interface binaries.  I gave him a copy so that JavaCV will have another platform it can successfully talk too.  Isn't open source great?

The OpenCV binaries and JavaCV/JavaCPP are attached to this post as well.

JamVM 1.5.4

JamVM 1.5.4 is the latest version.  This version needs about 40 Megs of memory to compile, so compiling it on the Chumby is not an option.   The cross compiler will be necessary.
Zlib is a popular compression library which is a necessary dependency for the JVM.
tar zxvf jamvm-1.5.4.tar.gz
cd jamvm-1.5.4





12 years 11 months ago


Cool man! Thanks for your efforts!!!

I guess my daughter moved the uBotino board out of the camera range... But the carpet looks fine. BTW, what time was this picture taken? I'm just trying to figure if it was before or after Roomba cleand the place... :D

Anyway, I'll install the camera on a pan servo (no tilt at the moment) and a Sharp sensor on top of it, mounted on the test robot. Chumby is powered by a wall wart, if I connect it to a battery, it will deplete it before I'll be back from work. Chumby powers the USB hub and the hub powers the uBotino (and the Sharp sensor). But for the servo, I need to switch the battery on... So, let me know when you want to do tests and I'll turn it on for you. 

You're talking about my different method of controlling motors. This does not affect MRL or Chumby. Only the Arduino firmware. There we need to make the custom config, perhaps selectable by a variable that MRL can flip True. Also, the motors speed will be a byte instead of a int, but that can be also dealt with inside the SetSpeedLeft and SetSpeedRight functions. Going further, after we work with encoders, the motor functions can accelerate to the SetSpeed, then the Stop command will decelerate until a full stop is achieved. PID control can be used for this. But let's go small steps first.



12 years 10 months ago

Hello Ro-Bot-X,

Could you say which webcams you used?  Some Chumbarian were asking....

If I remember correctly you had one which worked and had very slow frame rate ...

And one which did not work and had very fast frame rate ...

I would think that the OpenCV list of supported cameras would work...  I have used Logitec webcams in the past (although with PCs not Chumbys)

Logitec C600 and they have worked great...


12 years 10 months ago

In reply to by admin

The webcam with the low refresha rate is similar with this one:

The webcam that needed a driver is similar with this one, but it is 1.3MP:

I would get a camera that has over 1MP real, not software. Heck, I could get this one and replace the mic with a ultrasonic sensor, looks cool, like Wall-e's eyes:


I'm done with the move, things get back rolling, I'll try again to revive my Chumby. As soon as I'm done I'll let you know!



12 years 9 months ago

This is a breif descriptio of how to compile the OpenCV sources for a Chumby on a Linux x86 box.
The specific versions use for the description:

Chumby : Chumby 1
OpenCV : OpenCV-2.3.0 
Linux box : Fedora 13 running on an x86
CMake : cmake & cmake-gui

On the Linux box

  • Download and unzip the latest OpenCV sources - in this case I've chose to unzipp the tree into /home/gperry/chumby/OpenCV-2.3.0
  • Install cmake and its dependencies using the command
    # yum -y install cmake cmake-gui
  • Install the chumby tool gcc 4.3.2 chain from this guide -
  • The guid does an excellent job of setting up all the cross compiling tools in /usr/arm-2008q3/bin
  • Run cmake-gui - this will bring up the gui 
  • Browse to the OpenCV-2.3.0 sources
  • Choose an output directory - in my case I have chosen /home/gperry/chumby/ 
  • Coose "Specify options for cross-compiling" when the configuration button is pressed

    here are the cmake options I chose

    Some additional configuration will be needed. The Chumby does not have jpeg, tiff, png  or python support so I removed these.  Strangely, the build will not complete because of some linker options where are not supplied.  These extra linker flags must be added to a variable CMAKE_SHARED_LINKER_FLAGS -lrt -lpthread . The list of useful cmake variables is here
    The variable  CMAKE_SHARED_LINKER_FLAGS is already defined but only viewable in "Advanced" mode.  The flag entries -lrt -lpthread need to be added.

    Here is a complete list of changes

    Commandline options:




    Cache file:













  • Somehow the build for OpenCV has some references to /usr/include - the compiler does a nice job of warning during the build
    "cc1plus: warning: include location "/usr/include" is unsafe for cross-compilation"
  • We will want to take care of this - it may seem a bit heavy handed but as long as you remember to switch some symbolic links back when your done...  it should be ok
  • if you were to compile at this time - warnings would come up

    To compile binaries with improper headers at best would create compiling havoc, at worse nasty runtime errors. So, I moved the include and linked the appropriate include so that the ARM includes are now under /usr/include

    # cd /usr
    # mv include include_gcc
    # ln -s /usr/arm-2008q3/arm-none-linux-gnueabi/libc/usr/include include
  • Now as an extra precaution, we will start changing the pathing of a shell, so that it will "think" it's using the native gcc compiler but in actuallity it will use the arm compiler for the chumby with the following command
  • $ export PATH=/usr/arm-2008q3/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
  • The point of this is to have the /usr/arm-2008q3/bin at the front - so that any gcc or ld commands will get handled from the executables in that directory first
  • The export persists for the life of the terminal, so you will want to do all your building in that terminal
  • Go to where the Makefiles were generated $cd
  • $ cd /home/gperry/chumby/
    $ make clean
    $ make
    [100%] Building CXX object samples/gpu/CMakeFiles/example_gpu_surf_keypoint_matcher.dir/surf_keypoint_matcher.obj
    Linking CXX executable ../../bin/surf_keypoint_matcher_gpu

    Yay! But wait there's more !

    Grrr ! - OpenCV 2.3.1 gives flann/dist.h:63: error: 'fabsl' was not declared in this scope
    which means opencv started using floating point in its flann library, and the current gcc 4.3.2 does not have this function.

    It's defined as a template but not used, I know how to solve that - comment it out, and pray to the linker :)
    in dist.h
    //inline long double abs<long double>(long double x) { return fabsl(x); }
    building building building ....
    [100%] Built target example_gpu_surf_keypoint_matcher
    Yay !
    Bundle the .so's and move them onto the chumby - the following shared object files (.so) where created and are attached as
    But wait there's more !
    JavaCV will need JavaCPP generated files to be compiled on this system

    The G++ command from hell !  Location of your dependencies will vary :P
    $ g++ -I/home/gperry/opencv/OpenCV-2.3.1/modules/flann/include  -I/home/gperry/opencv/OpenCV-2.3.1/modules/calib3d/include  -I/home/gperry/opencv/OpenCV-2.3.1/modules/features2d/include  -I/home/gperry/opencv/OpenCV-2.3.1/modules/imgproc/include  -I/home/gperry/opencv/OpenCV-2.3.1/modules/core/include -I/home/gperry/opencv/OpenCV-2.3.1/modules/legacy/include -I/usr/lib/jvm/java-1.6.0-openjdk- -I/usr/lib/jvm/java-1.6.0-openjdk- jniopencv_legacy.cpp   -Wall -O3 -fPIC -shared -s -o -L/home/gperry/opencv/ -lopencv_legacy -lopencv_video -lopencv_features2d -lopencv_flann -lopencv_calib3d -lopencv_highgui -lopencv_imgproc -lopencv_core  2> out.txt
    This one is for core
    g++ -I/home/gperry/opencv/OpenCV-2.3.1/modules/flann/include  -I/home/gperry/opencv/OpenCV-2.3.1/modules/calib3d/include  -I/home/gperry/opencv/OpenCV-2.3.1/modules/features2d/include  -I/home/gperry/opencv/OpenCV-2.3.1/modules/imgproc/include  -I/home/gperry/opencv/OpenCV-2.3.1/modules/core/include -I/home/gperry/opencv/OpenCV-2.3.1/modules/legacy/include -I/home/gperry/opencv/OpenCV-2.3.1/modules/core/include -I/usr/lib/jvm/java-1.6.0-openjdk- -I/usr/lib/jvm/java-1.6.0-openjdk- jniopencv_core.cpp   -Wall -O3 -fPIC -shared -s -o -L/home/gperry/opencv/ -lopencv_legacy -lopencv_video -lopencv_features2d -lopencv_flann -lopencv_calib3d -lopencv_highgui -lopencv_imgproc -lopencv_core 2> out.txt
    all cpp files generated by javacv/javacpp need to be compiled for the Chumby etc...

    Don't forget to move include_gcc back into place !

On the Chumby

Try to keep things "clean" by looking for old versions of your new libraries.
# find / -name libopencv*

Get the latest release of MRL

# wget

go into the myrobotlab-0014/bin directory and delete everything ! - yep all of it is for windows or Linux x86 - it won't work on a Chumby

# cd myrobotlab-0004/bin
# rm *

Now get the Chumby OpenCV 2.3.1 binaries

# wget
# tar zxvf opencv.2.3.1.chumby.tgz

Get the interface so's
# wget
# tar zxvf

Got this error after exporting LD_LIBRARY_PATH and PATH with everything ../../lib/ cannot open shared object file: No such file or directory
if you look at it carefully , it's looking for it in a specific ../../lib location

# cd myrobotlab-0014/bin
# mkdir arm7
# mv libjni* arm7
# mv libopencv_* to myrobotlab-0014/lib

have nearly everything except the kitchen sink exported

export LD_LIBRARY_PATH='/mnt/usb-E0FD-1813/myrobotlab-0014/lib:/mnt/usb-E0FD-1813/myrobotlab-0014/bin/arm7:/mnt/usb/myrobotlab-0014/bin:/lib:/mnt/storage/lib:/mnt/storage/local/lib:/lib:/usr/lib:/usr/local/lib:/mnt/storage/lib:/mnt/storage/local/lib'

Change of error ! - UnsatisfiedLinkError: no jniopencv_core in java.library.path

So the ../../lib reference is relative to where the cwd is of program execution... NOT relative to the libjnicore so

# cd myrobotlab-0014
# mkdir ../../lib
# cp lib/libopencv*.so ../../lib

Now the next error

ERROR org.myrobotlab.service.OpenCV  - using com.googlecode.javacv.OpenCVFrameGrabber
7334 [OpenCV_videoProcessor] ERROR org.myrobotlab.service.OpenCV  - ------
java.lang.Exception: cvCreateCameraCapture() Error: Could not create camera capture.
    at com.googlecode.javacv.OpenCVFrameGrabber.start(
    at org.myrobotlab.service.OpenCV$ Source)

Unsupported Camera?


12 years 9 months ago

Tried running ldconfig - got error of read only file system

chumby:/mnt/usb-E0FD-1813/myrobotlab-0012# ldconfig
ldconfig: Can't create temporary cache file /etc/ Read-only file system
After - # mount -o remount,rw /
ldconfig worked
Completely forgot needed the libjniopencv_*  binaries attached to the ticket here
uncompressed them in the bin directory.
Now :
Exception in thread "OpenCV_videoProcessor" java.lang.UnsatisfiedLinkError: /mnt/usb-E0FD-1813/myrobotlab-0012/bin/ cannot open shared object file: No such file or directory
the libjniopencv_ were compiled for 2.2 - attempting to use them for 2.3 - WARNING interface is probably broken somewhere 
cp  Got the same error !

The generated cross-compiled library is Explicitly looking for

I thought Samuel had a filtering policy - I'll ask him about it and strategies of loading.
Now - 
Exception in thread "OpenCV_videoProcessor" java.lang.UnsatisfiedLinkError: /mnt/usb-E0FD-1813/myrobotlab-0012/bin/ cannot open shared object file: No such file or directory

Remote Command Line
# chumby:/mnt/usb-E0FD-1813/myrobotlab-0012# /mnt/usb-E0FD-1813/java/ejre1.6.0_25/bin/java -classpath ":myrobotlab.jar:./lib/*" -Djava.library.path=./bin org.myrobotlab.service.Invoker -service OpenCV opencv RemoteAdapter remote

Quick OpenCV Only
# /mnt/usb-E0FD-1813/java/ejre1.6.0_25/bin/java -classpath ":myrobotlab.jar:./lib/*" -Djava.library.path=./bin org.myrobotlab.service.OpenCV
More cross compiling - need to "re-compile" this ( ) for the latest version 



12 years 8 months ago

I am now under the suspicion that the cmake-gui will mess things up.

With OpenCV 2.3.1 if you specify cross-compile the option WITH_V4L disappears !

I put it in the cmake-gui manually, but I have a feeling that it's being ignored.

Sooooo... (running out of options here)  I'll create native Makefiles with cmake-gui and select the WITH_V4L but build in an environment that is configured to cross-compile, with gcc linked to the arm7-gcc

Another option is to use the FFMPEG frame grabber, I cross-compiled ffmpeg without any errors.. will be moving it up to the chumby shortly.  Potentially, this might be faster anyway.....

This command should test that ffmpeg is working on the chumby correctly

# ffmpeg -f oss -f video4linux2 -i /dev/video0 out.mpeg

The libjniopencv_* files appear to be looking for their corresponding libopencv in a hardcoded relative path location !  I'm guessing javacv unloads the jars into a tmp directory and expects to find the libopencv in /lib (maybe a few other locations?)

I have moved all the libopencv_* into /lib and the libjniopencv_* into myrobotlab-0014/bin  this should line up things.

No more unsatisfied link errors YaY !


12 years 8 months ago

This command worked - ffmpeg was streaming video - just didn't have anything to recieve it !

chumby:/mnt/usb-E0FD-1813# ./ffmpeg -f video4linux2 -s qvga -pix_fmt yuyv422 -i /dev/video0 -vcodec mpeg
4 -f rtp  -r 15 rtp://

I'm a noob with ffmpeg - and got the command line from this chumby forum -


12 years 8 months ago

Up again ....
This nearly kiled me, I still have to collect all the pieces and make more notes, check stuff in....
Part of the issue (took me a while to realize it) OpenCV 2.3.1 has decided NOT to support Video For Linux (V4L) and is supporting V4L2 only, which the chumby does not have ...



12 years 8 months ago

This is what your chumby looks like without MRL running

Mem: 41388K used, 18724K free, 0K shrd, 472K buff, 25016K cached
CPU0:  0.9% usr  2.9% sys  0.0% nic 95.4% idle  0.0% io  0.3% irq  0.1% sirq
Load average: 1.14 1.19 1.08 1/48 565
32070  1134 root     S     5960  9.8   0  0.9 sshd: root@pts/0
25763  1134 root     S     5960  9.8   0  0.0 sshd: root@pts/1
 1134     1 root     S     3276  5.4   0  0.0 /sbin/sshd
  944     1 root     S     3264  5.4   0  0.0 /sbin/syslogd -C200
32489 32078 root     R     3068  5.0   0  0.9 top
32078 32070 root     S     3068  5.0   0  0.0 -ash
25776 25763 root     S     3068  5.0   0  0.0 -ash
  831     1 root     S     3068  5.0   0  0.0 -/bin/sh
The driver uses at "least" 1 buffer @ 5 Megs - probably more - then MRL, the Java VM + several more video Buffers + OpenCV = not enough memory
As an experiment I tried to get ffmpeg to stream.  FFMPEG is an extremely efficient headless program which can do all sorts of cool things with video, including streaming it.
chumby:/mnt/usb-E0FD-1813#  ./ffmpeg -f video4linux2 -s qvga -pix_fmt yuyv422 -i /dev/video0 -vcodec mpeg
./ffmpeg: /lib/ no version information available (required by ./ffmpeg)
ffmpeg version N-32154-gcaf63eb, Copyright (c) 2000-2011 the FFmpeg developers
  built on Aug 29 2011 22:45:07 with gcc 4.3.2
  configuration: --enable-cross-compile --target-os=linux --cross-prefix=arm-linux- --arch=arm
  libavutil    51. 14. 0 / 51. 14. 0
  libavcodec   53. 12. 0 / 53. 12. 0
  libavformat  53. 10. 0 / 53. 10. 0
  libavdevice  53.  3. 0 / 53.  3. 0
  libavfilter   2. 37. 0 /  2. 37. 0
  libswscale    2.  0. 0 /  2.  0. 0
[video4linux2 @ 0xd10420] The driver changed the time per frame from 6629316/13700208 to 1/30
[video4linux2 @ 0xd10420] ioctl(VIDIOC_STREAMON): Cannot allocate memory
/dev/video0: Input/output error
Mr. Chumby either needs more memory or another camera.....


12 years 8 months ago

It turns out that OpenCV 2.3.1 does not support Video For Linux (V4L) instead they support V4L2 (ish).  This was a problem as the Chumby uses V4L. 

The module responsible for camera capturing is highgui.  The file for capturing V4L cameras is cap_v4l.cpp. Within it some #defines might need to be changed. To get the file to actually be included in the build CMakeLists.txt in modules/highgui needs to be changed.


I tried with all sorts of new flags to get this change - but it did not work, you have to hack the file.
I have included my CMakeLists.txt & CMakeCache.txt which should be enough for anyone to successfully cross compile.


12 years 8 months ago

Wondering if the upgrade is using more memory :P
Ro-Bot-X tried a 5Mbit camera - and it would not start.  Now the old/slow camera will run out of memory (Grrr.)
Possible solutions:
1. Use the ffmpeg frame grabber - I've already cross-compiled ffmpeg and I have heard it is supposed to be faster than OpenCV's frame grabber.

2. Run headless - without streaming the images - not easy to debug or remotely control - but a possibility which will save memory.