Connecting (N) MRL Video Streaming Instances - Network Updates

2013.04.22

When your hunting tricky network bugs, its helpful sometimes to draw a map of where the bugs might be hiding. I had to do this to get the (last?) networks bugs which were hiding out in an involved service registration process. Added new feature to MRL too - the capability to spawn other MRL instances.

Hopefully I can put together a tutorial soon to show how to use all this new functionality.

Things still left to do :

Dynamic network protocol assignment like UDP for Video & Audio streams
Heartbeat for connectionless protocols like UDP
Platform and type checking
RoutingHistory implemented with different data structure - HashSet - and performance metrics added
Switch for detailed Message logging
SendBlocking timeout - must log error
GUI shows duplicate in running services - for any new service created
Ability to robustly shutdown an instance
Multi-cast protocol - discovery !
Add auto-connect option
Ability to spawn new instance from the GUI (with options)

Finally squashed a very big network bug - I was not reseting the ObjectOutputStream after each message - this lead to the stream caching objects and no data transfered to remote clients. Finally ! This morning I tested 40 Clock clients - and it appeared to work ! Then for grins I went to 100 remote clients in the same star topology I previously tried. The "servers" gui was slow and crashed after all 200 tabs came up. I think this is progress ? :) . There are still a couple threading issues in the gui... I'll get them one of these days. On a different test I decided to try something much more data intensive - video streaming (no control over this). I had a few problems getting a system up initially - it seems the streams become corrupt at some point. But I did have success in one test with 5 clients ! 5 simultaenous video feeds to remote clients !!! - It lasted 5 minutes before one of the streams became corrupt.

Next on the agenda would be to add UDP video streaming (TCP is not as proficient). Also, I'll have TCP & UDP stream corruption issues reconnect & recover !

There are still some lingering bugs in networking, but it's getting a lot better. The above screen show's 10 instances of MRL all connected in a Star Topology to a single 11th Instance. This proof of concept might be for multiple raspberry pi's connected to a single instance so they cuold "work together" in a swarm.

In this example I created 10 instances, each with its own gui - (which is probably not what you would want on the raspberry, but it made it easy for me to see what was happening)

All these clients can be controlled by the 11th hub / master instace - To see cause and effect each rasPi instance has a clock, and I turned some of them on from the master instance.

Or this could be (N) number of Instances on (N) number of computers with (N) number of Arduinos, whatever :)

I still know of some bugs - but I'm getting closer to squashing them too !

Above you can see 2 instances of MRL connected together. One MRL instance will run on a RasPi - but currently I have it running on a laptop with another MRL instance.

Some things to note:

I've changed the names via the startup scripts so they don't clash - Remember, its one very imporant thing in MRL - "no 2 services can be named the same thing" .. You can change the names on the command line.

This is the command line in the myrobotlab.bat file

start javaw -Djava.library.path="libraries/native/x86.32.windows;libraries/native/x86.64.windows" -cp "libraries/jar/*;libraries/jar/x86.64.windows/*;libraries/jar/x86.32.windows/*;" org.myrobotlab.service.Runtime -service gui GUIService python Python

The highlighted part is the service name & start parts... so what this does is start a GUIService named "gui" and a Python service named "python" on the RasPi I made a different command .. Its like this

I started them both up then I connected the PC's MRL to the RasPi

What happens when they connect -
They initially tell one another about themselves - and "register" their services - When a service gets registered in a system which contains the GUIService - a GUI is created.. and Viola - you can control one from the other..

It's not perfect.. there are occasionally serialization problems and some other issues. Some services don't lend themselves to be exported to another instance. An example is the GUIService to and Android (smartphone) system .. the GUIService is written in Swing and Android does not support it.. but I'm working on those issues...

I've actually been pretty suprised how little work I had to do things like remotely control and Arduino..

Multiple MRL instances

HI Grog, Good progress! Is the reason for multiple MRL instances so that differnet processes can be run and manipulated concurrently? That is like - InMoov - the motion, vision and some other operations would be different instances instead of one large process which could crash and stop the entire robot?

A single MRL instances

A single MRL instances support many "Services".
The design is referred to as "Service Oriented Architecture"
So within a single process many Services can run and operate concurrently. If one Service crashes, it should not effect others in the same process, they are all operating on different threads.

The concept of having multiple processes, allows MRL to step out of being contained on one hardware system. There are several important reasons to have this ability.

Process isolation - as you mentioned, you could isolate a very unstable Service which potentially takes the whole process down. Although I dont think this is very likely.
Remote control - remote systems can connect and merge with other remote systems providing remote control and remote data logging
Harnessing the best hardware for the best job - Although you may have a raspi controlling several swarm bots, they might not be the best systems to process video streaming. A possible senario is for the raspi's to send the video to a high powered computer which is not mobile for processing then the results (of what to do or where to go) - come back from the big computer
Swarm - since MRL instances merge resources when they connect, potentially a large swarm of robots can be controlled centrally, or the control itself can be distributed among all the bots, but with the ability of connecting - one bot can very easily effect others
The whole is greater than the sum of the parts - to make extremely complex systems, a single computer is may not be sufficient.

There's probably more reasons too, but I havent had enough coffee to write them down ;)

MRL Instances

OK! I will keep thinking, planning and collecting. You will definitely have the features to make a lot of things work.This MRL IS COOL!

Rasberry PI

I'm getting ready to compete in the Red Bull 2013 Creation Contest, and I wanted my Rasberry PI to control an Arduino remotely using MRL. After 8 hours of fiddly crap and mega frustration, I can finally say it works! An important tip for others is to use the same MRL version on both computers, that will save some trouble.

Hi Raver1975

Ya - I'd like to make the initial registrations more smart, such they say ...

"No Way Dood ! Your trying to connect to different versions !" :)

Congratulations for working it out though and Good Luck on the competition...

And before you spend another 8 hours of mega frustration - you might want to ask .. or you might be into that kind of thing :)

I just got the idea of what

I just got the idea of what your are doing. Neural net. Collaborative computing. A cloud, sharing information and resources. As you shared with birds and your Swarm. I saw some one reference it - I now see the Borg coming.

Now of course, you will be

Now of course, you will be assimilated ! :D

But, yes - this is in preperation of say all the pan / tilt kits, and or InMoov instances connected together, sharing and distributing information and processing

YES !