status of openCV

Hi Grog

Just saw in the shoutbox that you are working on distributed computing now.

Do we have a status of openCV?

At least the swing gui in version 2398 does not work for me,. Looks like a problem with window sizing as I can not get to the filter list on the screen.

GroG's picture

Appears worky for me locally

Appears worky for me locally - and I have not made any changes to it recently.

I don't know the context in how your displaying it .. is the panel undocked ?  Perhaps there was an error with the filter ?  - you might want to try deleting the (dot)myrobotlab directory.  Or start with no filters, and see if the script or thing adding the filter got an error.  Start with small worky steps - then of course send a noWorky when it goes BOOM.

juerg's picture

I will try to explain what I

I will try to explain what I encounter (w10)

I start the whole robot. I either need to resize the window or switch to full screen to get all items shown. For this post I used a resized window to show my issues.

The layout after having the script adding filters. No capture is active

Pressing capture changes the layout and it cuts of a piece at the right

I can again manually resize the window but clicking on each of the filters reverts the layout back to the only partial visible window. In this window I can not scroll to the right, I need to resize the window and then drag it to the left to see everything.

I have 4 filters active: Transform, PyramidDown, Gray and Face Detect. Selecting the Transpose filter shows me the camara capture transformed - nice!

Clicking the PyramidDown filter shows me the original camera pic, no transform, no scaling down. Clicking on the Gray filter does not result in a visible change, still the original cam pic (with the exception of the "fillter parameters" section where it shows Gray).

Clicking the FaceDetect filter also only changes the "filter parameters" section, the picture shown is the unfiltered camara view.

So with a lot of clicky clicky I can get a layout that shows all items but I don not get the filtered output in the preview frame.

As I said I tried it with version 2398

Do not know about a noworky, it's not throwing an error. I however see in the java log that the "pre" and "post" messages of the filters do not match


[i01.opencv] [INFO] pre setDisplayFilter displayFilterFaceDetect
[i01.opencv] [INFO] post setDisplayFilter displayFilterinput
[i01.opencv] [INFO] pre setDisplayFilter displayFilterinput
[i01.opencv] [INFO] post setDisplayFilter displayFilterTranspose
[i01.opencv] [INFO] pre setDisplayFilter displayFilterTranspose
[i01.opencv] [INFO] post setDisplayFilter displayFilterPyramidDown
[i01.opencv] [INFO] pre setDisplayFilter displayFilterPyramidDown
[i01.opencv] [INFO] post setDisplayFilter displayFilterGray
GroG's picture

Curious if you tried

Curious if you tried "undocking" the opencv window?

on opencv color tab, right click -> undock -> size to what size you want ... press filters, capture etc. - resize to what size you want manually.

juerg's picture

nice trick with the detached

nice trick with the detached window - layout is fine now and does not change by selecting stuff..

I still have however the issue that the preview only works for the transpose filter (selected Transpose filter in the right list) when set by the script. If I manually add the Transpose into the empty list it also does not show the rotated pic.

selecting the other filters show the unfiltered pic in the preview and not the filter output.

Is there another workaround for this?

I have also tried if the preview only works for the top filter in the list but this is not the case. If I have only the Gray filter in the list selecting that does not show a gray-coded preview picture.

There is a long list of filters and I did not try them all to see which ones impact the preview and which ones not. I could however go through all of them and post the result if it is any help to you.

And another thing I noticed - fps!

I get with my 2017 lenovo notebook (VideoInput) 10-12 fps for the unfiltered pics. Adding the transpose, gray or pyramidddown filter drops the fps to 3, with combined filters I get only 1 fps - not good enough for tracking?

In resource monitor cpu is only at 6..7 % and does not significantly increase with added filters (only a small peek when adding a new filter)

GroG's picture

Ok, worked around 1 issue and

Ok, worked around 1 issue and now we have 5 or 6 others ?  Adding gray or pyramid down should not increase processing time.   I don't know your complete filter set at any given time.  So it would help if you posted a video which clearly describes the problem(s) you are having.

I currently get 31~32 frames per/second using my laptop and an embedded camera (no usb).
a single FaceDetect filter operating on this raw input I get about 30~31 which I was suprised at the good speed.  Adding gray & pyramid down (for testing purposes) adds no noticable delay. 

Adding transpose adds no noticable delay.

However this is a silly pipeline (I just constructed it from reading your post)

You always want to reduce the data before you send it to a search filter.  
The preferred pipeline would be (Input -> PyramidDown -> Gray -> Transpose -> FaceDetect)

juerg's picture

my first issue - mrl locks in

my first issue - mrl locks in undocked mode when I activate capture

juerg's picture

starting capturing first and

starting capturing first and afterwards undocking prevents the lock up.

I also see this time a much better frame rate than yesterday - can't see why as I have not rebooted but anyway - big progress! Maybe activating this w10 "game mode" for capturing the screen helped?

I also get the required filters in place and the preview shows the filtered output - so I do not know what made that not happen in the many other trials I did. Also the frame rate does not drop as dramatically as I had experienced it.

So I got carried away and started to remove the filters. I then reapplied the filters again and when I added the transpose again it crashed - but that is not a problem to me.

In total I can perfectly live with this state of opencv - to me a much improved situation. 

Thanks for your standing by

GroG's picture

That's good to hear

That's good to hear Juerg,

There is a large refactoring of the OpenCV filtering system in the near future.  It's a lot more robust (should be no crashes).. in addition to the bonus of having "Input" as a filter - which will allow the easy combining of multiple video-feeds to do stereo-scopic ranging and other fun tricks

juerg's picture

here the result of trying to

here the result of trying to set the filters by this script:

def startTracking():
  i01.headTracking = Runtime.createAndStart("headTracking","Tracking")
  i01.headTracking.attach(i01.opencv)"x", 10.0, 5.0, 0.1)"y", 30.0, 5.0, 0.1)
  i01.mouth.speakBlocking(u"ich beginne Gesichter zu verfolgen")

filterSettingWithScript.mp4 in my google drive folder, I am down to 0 fps :-(