It took "too" long, but I got virtual Joysticks working ! ...  
To be precise, any device (joysticks, gamepads, keyboards, touchpads, .. etc) which the Joystick service can see can now be "virtualized".

Ok, so what the hell does that mean ? :)

It means we can test all kinds of stuff without "real" hardware.  I can make a navigation application attempt to use this virtual joystick to "steer" a robot platform through a house.  I can analyse its move and adjust - oh wait that sounds like Training .

I was inspired by the dexterity video, of the hand and cube - but I want the virtual hand to steer a robot !

https://blog.openai.com/learning-dexterity/

You can see in the top UI - the controller selected is the virtual RumblePad 2.  It was created by just "cloning" a reall controller.

joystick.cloneController(2)

once the controller is cloned it can be set

joystick.setController(4)

and button, axis, etc can be set programmatically - the output is the same and fools the UI too as you can see I "programmatically/virtually" pressed buttons and moved the axis ..

joystick.pressButton("3")
joystick.moveAxisTo("x", 0.754)

When cloning a json file is created and re-loaded, which describes the controller. This file can also be "tweaked" to create new not-yet-existing joysticks .. like the 53 axis joystick with 100 buttons.. if desired.

 

wvantoorn

6 years 2 months ago

I know we already can, but could you use a imu to steer those 'joysticks' to move a robot through a maze or house?

And again i know it can already be done the normal way but it could be usefull to train headtracking or eyetracking

 

Sure .. its possible - although for the initial testing I'll basically be "simulating" myself using the joystick programmatically...

Work-E is pretty big and at the moment hard to control without inflicting chaos & damage... so .. if I simulate myself as a crazy tele-operator, I can evolve Work-E's software so that its more forgiving, and "does the right thing" when I accidently push full forward with both joysticks into a wall :)

I think I remember the F-16 fighter jet saying it was "fly by wire" - where the control system was very new in that it all was sent to a computer before getting to the ailerons and flaps ... and if it didn't do this the design of the plane would make it tumble uncontrollably

Yeah i know about the F-16, had the honor to be able to fly in the backseat a couple times. And the simulator on the airbase here(sadly closed now, miss the birds flying over)

I understand, lets get the basics right and after that do the fun stuff, making get a beer out the fridge while you sit in your lazy with the oculus on your head.