Maybe a bit confusing but I was thinking about the following.

I am going to do a live build of an inmoov during the maker faire in Eindhoven, end of September 2018.

And programming the inmoov with myrobotlab. Also during the build I want to make tutorial videos on how to do certain things. Because there are going to be many who can help that won't be an issue. But what I was thinking, a normal webcam with say a tablet does not give much fps. My thinking was that a raspi 3 with a dedicated raspicam will give you decent fps. I also know Grog once told (and Ray has it in his inmoov) that you can run multiple instances of MRL to do dedicated things (can't find the right word). And have them controlled by the main  MRL program.

That would give the raspi the freedom to do opencv service and use less resource on the main program. At least in my thinking.

 

To summarise , is it possible to use a raspi with the raspi cam for tracking and other fun stuff in a client MRL instance, and let a tablet or, in my setup, a laptop run the main inmoov services, programAB.

If possible and if the fps would be reasonable could we make a how to (I really don't know where to start, I'm just restarting) so I could implement it during the live build and make a tutorial about it.

 

I talked to Gael about making video tutorials on inmoov and myrobotlab, especially for newbies and for people who like to tinker but are just starting in trying to program. It is going to be like a book, chapters and starting with downloading, editing the config files, installing arduino and so on.

Hope you (the MRL gurus) can help me , and wanted to make it a blog post so anything good added is findable in the future.

 

Gr. Wilco

GroG

6 years 4 months ago

Wvantoorn !  Welcome back, really excited to see you in Eindhoven !

Think of it this way ... Video takes alot of compute power, after 6 billion years we have a very fancy visual cortex.

Its a relatively big chunk of your brain dedicated to pretty singular tasks.  Processing video just takes lots of cpu, so you usually want your biggest, fastest computer hooked up to the eyes.

Contrastly, sending signals to muscles it turns out you don't even need a brain, you can do with with just a pair of electrodes .. so this is certainly "low power compute" - your dumbest hardware should be responsible for this.

Distributed computing (which we continue to try to improve in mrl) is where multiple computers fast & slow can work effectively on the same problem.  They do this by communicating to each other events & data (this is the whole publish/subscribe framework in mrl)

I don't have a raspi3 - and others would know if they have enough horsepower to crunch video effectively. But the overall model I describe is a pattern which holds true.  ProgramAB doesn't really take that much power - it's a great program for reducing input and responding with canned or static randomized responses.  Tensorflow, DL4J, WaveNet, Yolo and all the stuff that learns needs the "bigger" brain/computer..

I hope this helps !

See you in Eindhoven ;)

In Fred#1 I installed 3 Raspberry pi 3 computers is the head.

This had limited success.

The main control Raspi 3 ran the ProgramAB, Servo Drivers, Servos and speech recognition with no trouble at all.

The second Raspi 3 has a Raspi Camera attached an used OpenCV to process the images, the very first filter i had to use was PyramidDown (PD), without out, then you run at between 1 and 2 frames per second on any other filter.

With only PD running I get as much as 15FPS.

Face detect works, but that as far as i went with that part as i was having trouble getting reliable communications between the Raspberry Pi 3. I never got around to programming the third one.

You can use the Raspi to simply encode the video and send it over the network to a more powerful computer to process the video with YoLo and the like.

I haven't yet tried YoLo on the Raspi, Last attempt I had, YoLo was not yet supported on the Arm platform, although i hear that it may now work.

If YoLo is working on the pi then communications is the next big thing to work out.

In this case, you could create using a much larger computer two YoLo models, one for object detection and one for face recognition.

That detected information can then be used by the AI to make decisions on who to say hello to and what profile to run in ProgramAB.

The advance communications functions we need for this may yet be a feature in the next version of MRL

hairygael

6 years 4 months ago

Hello Wilco,

I have tried using a raspi2 a while back and a OdroidU3, but the result regarding video input was so poor that I dropped that option for the tablet.

Recently I was building a new sort of robot called Bubble, and thought of using the raspi3 for that purpose, hoping it had improved enough. After testing, it turns out the fps is way too low to do a proper face tracking.

During my research, I came accross an interesting camera device which does all the post processing before sending the results to the PC. Searching on the net, it turns out someone in England already tried it on his InMoov and mentions that it's working very well. That person didn't seem to use MRL though.

The device contains a camera and a quad core cpu. It is called JeVois. I showed it to Grog during our Paris meetup and it seemed we could easily use it with MyRobotLab.  I wanted to test one before summer, but didn't have the time to do so.

https://www.jevoisinc.com/

Hi Gael, 

Ok that is dissapointing. Did you use the raspi and its dedicated camera (with the flatcable) or a regular webcam? Anyway, i may not give up on the raspberry but for now i think other things may be started first.

I have looked into that camera, and it looks like it may really be what we want, all processing done on camera and sending the good stuff to the tablet/laptop.

We could try to look at it at the faire, seen that there are so many brilliant minds together, it may be a succes! Anyway the price isnt too bad, if you consider the raspi is say 40 euros and the camera is also 20-30 euros, depending on mp's. It is about the same and if framerate is much better, i would call that worky!!

Anyway i will keep the project in the back on my mind for now but maybe this post will get the Jevois-A33 update.

 

Maybe we should make a how to put mrl on raspi and maybe even like ray did on multiple. So people who want to start with raspis (they are not to expensive and pretty powerfull) have a solid base to begin with.