There's been some refactoring of OpenCV to expose a method on it that allows you to pass in a custom frame grabber directly when you tell opencv to start.  This means we can programmatically control the settings on the grabber 

 

Here's a small python example of creating a frame grabber, intializing it with the filename and the api preference and then 

 

from org.bytedeco.javacpp.opencv_videoio import CAP_GSTREAMER
from org.myrobotlab.opencv import OpenCVFrameGrabber
 
filename = "nvcamerasrc ! blah blah blah.."
 
grabber = OpenCVFrameGrabber(filename, CAP_GSTREAMER)
opencv = Runtime.start("opencv", "OpenCV")
 
opencv.capture(grabber)
 
 
I have no idea if this will work.. but it's an example for Spawn32 to try out..   I don't know what the proper string is to pass in for the filename for the jetison tx2 .. so take the above just as an example..
 
 
 
 

Spawn32

6 years 3 months ago

But we are still not there...

this line will in a terminal windows deafult to the lowest res and open a capture window at 820x616

DISPLAY=:0 gst-launch-1.0 nvcamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM)' ! nvegltransform ! nveglglessink"

so i tested with:

from org.bytedeco.javacpp.opencv_videoio import CAP_GSTREAMER
from org.myrobotlab.opencv import OpenCVFrameGrabber
 
filename = "nvcamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM)' ! nvegltransform ! nveglglessink"
 
grabber = OpenCVFrameGrabber(filename, CAP_GSTREAMER)
opencv = Runtime.start("opencv", "OpenCV")
 
opencv.capture(grabber)
 
and also with:
filename = "gst-launch-1.0 nvcamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM)' ! nvegltransform ! nveglglessink"
 
but i can see with dmesg -wH that it doesnt try to power on the cam at sensor-id=0 (/dev/video0)
If i select cam 0 or use filename /dev/video0 in the gui it will power that cam, but not using this python test..
 
it accepts the py script and say's "starting capture" but after a few seconds:
[opencv_videoProcessor] ERROR c.myrobotlab.opencv.VideoProcessor - stopping capture 22:23:10.415 [opencv_videoProcessor] ERROR class org.myrobotlab.logging.Logging - ------ org.bytedeco.javacv.FrameGrabber$Exception: retrieve() Error: Could not retrieve frame. (Has start() been called?) at org.myrobotlab.opencv.OpenCVFrameGrabber.grab(OpenCVFrameGrabber.java:268) at org.myrobotlab.opencv.VideoProcessor.run(VideoProcessor.java:281) 
 
as expeted since it dosent power / select the cam..

Spawn32

6 years 3 months ago

So first i added your commit to my local javacv 1.4.1 and buildt it for arm64, but dont think there are any cpu spesific code in javacv, it cam out at the original javacv in mrl, only a few bytes longer becuse of your commit, tested it and dident do anything differently :/

looking inside /lib in mrl ther is also just one javacv jar, not any other for x86, armhf etc...

Don't think the javacv is the probelm ?

 

i just looks like nvcamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM)' ! nvegltransform ! nveglglessink parsed to the tx2 since it's not even finding sensorid=0 (no power on/off of cam)

 

filename = "/dev/video0" finds the cam and powers it, but there we are back to the orginal problem, no nvcamerasrc to convert from bayer to yuv..

also tested for fun:

filename = "/dev/video0 nvcamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM)' ! nvegltransform ! nveglglessink"

but that again dident power the cam :)

scratch all that,  WOOORKEY :D

filename = "nvcamerasrc ! video/x-raw(memory:NVMM), width=1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink"

did the trick, dont know why, but seems that the other line i tested maybe carshed the nvcamersrc deamon even if it was ok from a terminal window ?

the way i start it now is to lunch mrl with python and opencv service enabled, undock opencv so i can see it,

then i past my py script into python and execute it.

But,it's a bit buggy, i need to execute the script and stop it a few times and when it starts it work niceley for a few min's and crashes mrl, this last thing, didnt i read somewhere that someone else was having the same problem ? crashing / stoping after a few min's ?

could you test a py scrip against you own pi cam and test this two things kwatters ? :)

with the same script, but tuned to your raspi :)

and aslo scratch that, no crash when aplying the same capture sting directly in filebox of opencv, and no lanecy to speek of with just a finger test :)

GroG

6 years 3 months ago

Super progress guys !

I think its great we can externally define a frame grabber and pass it in !

If this is the only environment that this works in perhaps the "default" grabber for the jetson tx2 should be the opencv framegrabber with the string ?

Is the enviorment currently defined as arm.64.linux ?