OculusRift

javadoc

This page (will) describe the OculusRift service to get head tracking information and to attach to a stereo camera system .

Description

The OculusRift service consists of 2 cameras in addition to the head tracker.  The OculusRiftGUI will display the video feeds in a side by side fashion as required by the OculusRift.  When this display is maximized , it will work in the OculusRift.    The cameras are mounted rotated by 90 degrees.  One camera is rotated clockwise and the other is rotated counter clockwise.  Orientating the cameras on their side (portrate vs landscape)  provides the proper aspect ratio for the oculus rift gui display.

The video feeds and cameras must be perfectly level otherwise stereo convergence might not occur for the wearer of the Oculus Rift.  

After starting the OculusRift service, open the GUI.  Pick a point near the center of the image that is easy to identify.  Click the same point as closely as possible in both video streams.  Upon clicking the second point, the two video streams will vertically align automatically.  

 

Example Script for the InMoov Oculus Head

Below is a script that will show how to create the OculusRift service.  Then create 2 servos that will attach to a pan/tilt assembly.  (pan is the "rothead" servo in the InMoov.    tilt is the "neck" servo in the InMoov)

For this example, the left and right cameras are index 0 and 1 respectively.  The cameras are mounted and rotated by 90 and 270 degrees respectively so the video feed is on it's side.   

 


#file : home/kwatters/OculusRift.py edit raw
#########################################################
# Example Script to show using OculusRift
# with the InMoov head, having 2 cameras
# It will display the camera video streams in the Rift (Side-By-Side)
# It will also subscribe to the head tracking data to update servo positions
# The neck servo tracks pitch
# the rothead servo tracks yaw
# roll is implemented with the OpenCV Affine filter by couter-rotating the video stream
#
# TODO: add leap motion & inverse kinematics :) woot!
# 
# JVM args ? maybe not necessary
# -Djna.library.path=myrobotlab\libraries\native  (needs to find the OVR_C.dll native library)

# Com port for arduino
comPort = "COM19"
# neck servo pin
neckPin = 12
# got head servo pin
rotheadPin = 13

# Create the Rift 
rift= Runtime.createAndStart("rift", "OculusRift")
# create the arduino
arduino = Runtime.createAndStart("arduino", "Arduino")
# create the 2 servos for pan/tilt
neck = Runtime.createAndStart("neck", "Servo")
rothead = Runtime.createAndStart("rothead", "Servo")
# connect to the arduino
arduino.connect(comPort)
# Attach the servos 
neck.attach("arduino", neckPin)
rothead.attach("arduino", rotheadPin)

# Define a callback method for the oculus head tracking info
def headTracking(data):
    # Debugging info
    print(data)

    # amplify the pitch recorded from the rift by a factor of 3 
    # To account for gear ratio of neck piston in inmoov (adjust as needed) 
    pitch = data.pitch * 3
    # the center position for the neck is 90 degre3es
    neckOffset = 90
    # update the neck position
    neck.moveTo(pitch + neckOffset)
    # track the yaw
    yaw = data.yaw
    # center position (yaw = 0 / servo = 90 degrees)
    rotHeadOffset = 90
    # turn head left/right to track yaw
    rothead.moveTo(rotHeadOffset + yaw)
    # Track the Roll in software
    rollgain = 1
    roll = data.roll * rollgain
    # left camera is 180 degrees rotated from the right camera
    # as you roll clockwise, this counter balances that 
    # by rolling the camera counter clockwise
    rift.leftOpenCV.getFilter("left").setAngle(-roll+180);
    rift.rightOpenCV.getFilter("right").setAngle(-roll) 
    # TODO: track the affine filters for roll

# add the callback to python from the rift.
rift.addListener("publishOculusData", "python", "headTracking")





 

The OculusRift service exposes 3 useful methods.  getPitch()  getRoll() and getYaw()

These return an angle in degrees.    

 

Native Library Dependencies

Note:  OculusRift service relies on a JNA binding and native dll.  This should work automatically when the OculusRift service is installed, but if not, you may need to add the -Djna.library.path argument to the JVM.

TODO: update/remove this section of this page.

The oculus rift service relies on native libraries that are exposed via java's JNA interface.  When using the OculusRift service, be sure to start MRL (java) with the command line argument for 
 

At this time the OCR_D.dll file is not pulled down properly with the install of the Oculus service.  It's on the TODO list to fix it.  In the mean time, you'll have to manually download add the DLL file to your MRL install's native libraries.  The same goes for the java JAR file  ( jovr-0.3.2.4.jar and OVR_C.dll )

Work Around:

1. downlaod OVR_C.dll file from github : https://github.com/MyRobotLab/repo/blob/master/org.saintandreas.jovr/0.3.2.4/win32-x86-64/OVR_C.dll?raw=true

2. save that file into your MyRobotLab/libraries/native/win32-x86-64 directory

3. download the jar file https://github.com/MyRobotLab/repo/blob/master/org.saintandreas.jovr/0.3.2.4/jovr-0.3.2.4.jar?raw=true and save it into  MyRobotLab/libraries/jar 

3. start myrobot lab from the command prompt using the following command:

javaw -Djava.library.path=libraries/native/x86.64.windows -Djna.library.path=libraries/native/x86.64.windows -cp ./;./myrobotlab.jar;./libraries/jar/* org.myrobotlab.service.Runtime -service gui GUIService python Python

NOTE: replace the path with the appropriate path for the system you are running on.  (I have only tested with windows 64 bit..)  

 

DIY Oculus Rift References

Examples:


#file : service/OculusRift.py edit raw
# start the service
oculusrift = Runtime.start("oculusrift","OculusRift")

kwatters's picture

Moving in stereo...

Life is strange.. moving in stereo...  Starting to work on the gui for the rift service.. adding two openCV subservices so I can see in stereo ...  woot!

 

Left and right Eyes in the Rift

GroG's picture

There You Are ! Nice work

There You Are !

Nice work kwatters ! .. must have missed this - are we looking at your 3d self with 2 hercules cameras + fish eye lens?  Excited to see where this goes :)

kwatters's picture

telepresence realized..

yup.. that is a stereo view into my offfice.  the images are aligned so they converge into stereo 3d vision in the rift..  I'm hoping that maybe next weekend, I'll have the time to finish up the gui to pull the cameras into a single gui that can be maximized..  that plus the head tracking from the rift, makes for quite an out of body experience...

kmcgerald's picture

Nice shirt!

Good album. I like rusty cage.  :)

 

Need to record the two streams and export them as side by side so people cat watch on 3DTV.

GroG's picture

FPS ?

What's the FPS like ?

kwatters's picture

stereo hercules cameras

The USB2.0 hercules cameras were only getting about 11 to 12 FPS..   not great, but functional.  I just got a GoPro...  but I don't think it's really cost effective to put 2 go-pros in the inmoov head.

kwatters's picture

Oculus Rift GUI powered by GroG!

With some (a lot!) of help from GroG  we got the Rift GUI working.. this is huge step forward!  

Oculus Rigt MRL GUI v0.1

Herzeh's picture

Any update on this ? Im

Any update on this ? Im wondering if its possible to display two camera stream on the oculus rift and how does one even get Oculus Rift service in first place, i don't have it in the service list.

diegocdl's picture

Which Oculus Runtime do you use?

I'm trying to use the oculus service with MRL version 1.0.119 and Oculus Runtime v0.8, and I follow the instructions for the native library dependecies, but I get an error like the "video was null" in MRL. Which version of the oculus runtime is supoused to work with the oculus service ?

 

kwatters's picture

Use the latest

Hi diegocdl,

  Verion 1.0.119 is SUPER old!  Use the latest build.  We're up to build 1.0.1117 now. The link for it is at the top of the in the shoutbox on this page.  I updated the oculus service last time it was using Oculus Runtime 0.7.  It should work with both the DK1 and DK2.  I have not tried the actual consumer version of the oculus yet.

The  "video was null" error message means that there was no video from the camera.  I suspect you'll need to do some tweaking to get it working for your setup.  See how far you get, let me know.  I'm happy to help work through the issues to get it working with you

Good luck!

 -Kevin

diegocdl's picture

Still not working

Sorry for answer too late, Now I clone the development branch from the repository of github and run My robot lab with eclipse to use the last version and becouse I'm working in create a new service to control the InMoov, and have the oculus runtime 0.8 becouse the 0.7 don't recognize my oculus DK2 and I find the runtime 0.8 has compability with the 0.7, but still get an error, the specific error is "The Oculus Display was null". And before that the Oculus Service generates an Exception in the hmd.create() that says  "Unable to initialize HMD"

Do you have any clue about what can be wrong?

I really apreciate your help