Hi

I'm testing RemoteAdapter with several services. I have tested AcapelaSpeech, Arduino and Servo. I think they are worky now. I have only made very light test yet, so more testing is needed.

The service that I'm testing now is the OpenCV service. It starts up nice and I can see it on both hosts, but as soon as I start capture it fails. 

I think that the problem is that java.awt.image.BufferedImage can't be serialized. I found a note in OpenCV.java at line 111

// FIXME - don't return BufferedImage return SerializableImage always !

Have you seen this problem before, or is the comment related to some other problem ?

Any advice on how to fix this issue ?

/Mats

 

 

GroG

8 years 3 months ago

BufferedImage has a long history with Java - its part of awt ...  there are good parts and bad parts to it.

One bad part is - its not serializable, so I created SerializableImage.

The good part of SerializableImage .. is .. well .. its Serializable . the bad part is - it takes time to to convert from a BufferedImage to a byte array with a known image type which can be decoded.

OpenCV at its core is JNI - and within the JNI there are several low level image classes which are used to in many of the OpenCV methods .. IpImage is a commone one 

JavaCV does a lot of great work simplifying access to these C & C++ classes .. but they are definately not serializable either.

I have methods in the OpenCV service which will conversion to display in Java land often goes like this :
Frame -> lpImage -> BufferedImage -> SerializableImage ----(serialization)---> ---(network)---> ---(deserialization) ---> SerializableImage --> BufferedImage --> Display in different process / different host

So if your sending an OpenCV service over the wire - there better not be anything which doesnt serialize, or if it is, it needs to be marked as transient 

Long term, as far as the process of serialization - its not optimal, we could :

  • look into more proficient means to serialize
  • look into BoofCV - which uses very little JNI code (Yay!)

Short term :
look for a member which is of the following type and not transient in OpenCV
Frame, lpImage, BufferedImage  
and mark it as transient...

I will try the short term solution. It will probably cause that no video wil be shown on the remote host, just on the local, but I think thats fine. Performance would suffer otherwise. Thanks for the advice. 

OK.

It's semi working now. OpenCV can execute with RemoteAdapter, but no images passes between the two instances.Need to do more testing to see if results from for example faceDetect and faceRecognizer still will work across multiple instances. 

/Mats

 

Hey Mats,

When you start open cv, it also starts the streamer service. The streamer service takes the video stream and publishes it as a video stream in the MJpeg format.
A while back I added a new frame grabber to OpenCV service. It can attach to a remote MJpeg feed. This is how I was able to get the video stream to a remote instance of OpenCV.
Perhaps , rather than serializing each image/frame, you could just attach to the feed that is produced by the streamer service. (The webgui uses this method to display the video in a browser)
Hope this helps!
Kevin

Getting Started