Fun with OpenCV

If your gonna refactor the service - might as well make it "better" "stronger" "faster" ..

So I made it capable of input from youtube videos - cuz our robots will want unsupervised training of neural networks by watching tonnes of youtube !!

The above stream was done by a single line :

opencv.capture("https://www.youtube.com/watch?v=2STTNYNF4lk");

 

I'm also making the capture method easier, where it will try to choose the best framegrabber based on input, so noone needs to be bothered with such details ...

 

There was a problem with Nixie and a javacpp version clash with BoofCv which prevented the FFmpegFrameGrabber from working - But now since I incremented BoofCv's version - we can watch Bucky the Big Rabbit - When you have ears like these it makes yolo see it as a bird :)

Locally I'm polishing the heck out of the OpenCV service ... 
No more filter crazyness
No more VideoProcessor
No more pass-through functions 
Big reduction of code
Big simplification of OpenCV service
You can actually start stop and restart a capture Yay!

I never played with yolo filter - its quite fun 

Things Left To Do :

  • FrameGrabber maddness - find out what the deal is with framegrabbers not being successfuly released - may be javacv code...
  • Crazy big unit test .. perhaps test all filters
  • DL4J filter stuff seems to break on my test

Let me know what other things bother you, or possible enhancements ...  I know Gael found it difficult when it resized to BIG size even though no capture was going on.


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
moz4r's picture

Wahoo Nice ! Happy ! some

Wahoo Nice ! Happy ! some headhaches less, soon with opencv. So, more things to experiment !

Can you check something crazy also, with yolo filter if you add / remove couple of times, memory increase until the implosion off the memory module :) and it hurt.

Kill the ghostly process seem not enough

Also I remember the "input" from UI for filters was no worky.

kwatters's picture

dl4j borked?

curious, what are you seeing with the dl4j filter? it should just be doing vgg16 classification...

GroG's picture

Maybe I don't know how to set

Maybe I don't know how to set it up or use it ... I'll poke around more 

GroG's picture

I might have found a way

I might have found a way where the yolo thread is not released, but now since the moving of filters is much more simple there is no possibility of not releasing  filter when your done with it.

 

hairygael's picture

Hello Grog, Opencv +yolo

Hello Grog,

Opencv +yolo filter gets my CPU (i7-7700-3.6Ghz 16GoRam) run like crazy until I get warnings for over temp cpu. Temps gets up to 95 degre if I do not shut down MRL...

Using 1.1.315 Nixie.

 

Do you have similar effect?

GroG's picture

I can hear my fans spin up

I can hear my fans spin up ... 

Adjusting the little sleep variable for pending helped a little, probably a parameter of the filter we'd like to expose, the trade off appears to be less detection, but I'm sure their are probably other and better strategies for being less computationally intensive.

hairygael's picture

Aah that would be great,

Aah that would be great, specially if we want to be able to run the robot added with yolo in real time detection, otherwise the cpu will die very fast.

:)

GroG's picture

Since the restructuring I

Since the restructuring I think adding & removing filters are more robust, so  there's also the possibility of
 

  • adding yolo filter to do an initial detection
  • cropping and storing what was found
  • tagging the images in the field and use a lower intesive filter to do tracking once identification with yolo has been done
  • repeat at some interval

I have to say thanks to kwatters, this filter was a nice easy worky treat to find and begin to play with :)

hairygael's picture

Is it possible to have in

Is it possible to have in Yolo filter  the text in the bounding box color for a better reading view?

GroG's picture

Yup ;)

Yup ;)

GroG's picture

Oh .. and I think I found the

Oh .. and I think I found the root cause of the Gui getting huge ...
It might have to do with a stack trace being displayed - now for the display, the super long error msg with be truncated.

hairygael's picture

Hey cool! Much better to read

Hey cool!

Much better to read the the displayed text, specially on the tablet!

Oh yes because when Opencv change the swinGui size everytime you ask for a filter or to close a filter was a bit annoying and disturbing for beginners.

hairygael's picture

Currently the yolo filter is

Currently the yolo filter is using coco, correct? So 80 object recognized max. (The tomatoes behind you are not detected)

Maybe if we had also the option for the 9000, for high computing robots, it could be good. Though the results are pretty messy, but maybe some good folks will work on it for better results in the futur.

data/combine9k.data yolo9000.cfg yolo9000.weights

GroG's picture

are you doing darknet python

are you doing darknet python examples ?

hairygael's picture

Yes I use currently the

Yes I use currently the darknet python with Manticore, but when testing Nixie yolo filter in Opencv

it seems to be the coco version.

kwatters's picture

coco / yolo9k and beyond

When I originally added yolo as a filter,  I think only coco was supported.  In theory we should be able to swap out the model for the yolo9k model..  I've also been working on taking the yolo recognized objects and then passing the recognized result on to a dl4j transfer-learned model for sub-classification. 

This is one of the things that I plan to be showing & talking about at the Activate conference in Montreal next week.

hairygael's picture

Very cool for the dl4j

Very cool for the dl4j transfer!!

I tried this morning to play between OpenCV and dl4j filter and dl4jtransfer filter but didn't figure how it works and what is suppose to happen for two reasons

1-My cpu ramps up to a heat point a bit scary.

2-I get an error whenever i try with dl4j saying "there is no image in opencv..."

moz4r's picture

About you CPU  , the yolo

About you CPU  , the yolo gesture using yolo filter will work a like what you have today, the filter is ON few seconds only, time to analyze the environment. And stop ( or should ).

 

GroG's picture

I don't trust the current

I don't trust the current OpenCV filters to do the right thing ... (they "should" unload)

I trust new OpenCV - which I'm thinking we might want in nixie...

Spawn32's picture

What version of OpenCv are

What version of OpenCV are you thinking of inplementing Grog ?

GroG's picture

What I mean is manticore's

What I mean is manticore's version of the OpenCV service in MRL had several issues.
I trust Nixies version of the OpenCV service (as soon as I push my changes)

The version of the opencv libraries in Nixie is 3.4.2

GroG's picture

People and traffic on a busy

People and traffic on a busy street in London

opencv.capture("https://www.youtube.com/watch?v=q71MCWAEfL8")


All our cat videos belong to us !

Still needs some polish - lots of nulls can come back from the (very capable) FFmpeg frame grabber..
Its "all" nulls when it gets to the end of the video, but was going to look for a cleaner way to determine end.
Might be nice to  default "loop" when the end is reached...
Also playback speed is sometimes wonky.