Virtual InMoov in WebGui (a start)

 

 

 

Virtual InMoov currently works in Blender, but wouldn't it be great if we did not need Blender to display ?
Three.js is an great library for using webgl and displaying 3d models and environments.

Some benefits for using the WebGui to display Virtual InMoov.

  • No dependency on Blender - it would be 'all' MRL -  meaning end users just need to install MRL and would not have to install Blender
  • Speed?  The current Blender service drives Blender, but I'm wondering comparitavely if the WebGui would be faster as its sole purpose is to display, & Blender has much more complex functions than display (e.g a big editor)
  • Clean interface - currently the Blender service uses  a slightly hacked up interface over sockets.  The WebGui virtual InMoov service could use the identical interface as all other current services - (JSON over websockets)
  • Fewer open ports - all communication would be handled through the same ports which WebGui does all its other communication.
  • Portable - Yes this would work on any mobile device too, while getting Blender to run on a mobile device will probably not happen

Some of the details :

  • Three.js project has written many 'loaders' 
  • They have also written plug-ins for Blender, this library is called io_three, it comes in the three.js-r73 release.
  • We already have three.js-r73 in the webgui supporting the Inverse Kinematics service.
  • There are a variety of possibilities for exporting importing, I have tried several, but currently am marginally successful 

I'm working on the process / matrix below, but I've had limited success.  At this point I have not been able to get 'shading' working at all.  I'll keep updating with progress.

Blender Export Three Import Notes
Collada .dae  ObjectLoader ColladaLoader - this potentially seems like the most direct, and the preferred method according to this .   In this senario, no Blender plugin is needed.  Instead Blender exports to a 'common' file format, and a Three.js loader loads it directly
Three exporter to Json   JSONLoader Scenes always crash - individual objects seem to work

Rotating WebGl Ear of InMoov .. We want more !! 

 

References :


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
Gareth's picture

Yeah thats the way to go....

I have checked out the three.js editor (still unable to run the http-server yet...patience..), though i was able to test out the three.js editor.

Indeed the Blender plugin is able to export to a .json format that three.js can accept.

First glance it only exports a single object at a time (so trying to select all of vinmoov as one batch does not appear to work - will persue this further), also it does not recognise bones.. again need to dive deeper.

Re:- the shaders ..... to get the above image I added a light pointing into vinmoovs face ... without it you will only see a black mass as in your video....try adding a light and vinmoov will "see the Light".

Gareth's picture

ahaaa... it preseves spacial data....

This is good news....

Although blender appears to be able to only export single objects at a time....

.... it does preserve the spacial position of each object..( often the case is that an objects position is zeroed around 0,0,0 after export...haha  .json exporter rocks ) ..so reconstructing Vinmoov in three.js would be possible....

... now the question lingers about bones.......

Gareth's picture

Bones

ok in the msgpack of the three.js blender modules there are a bunch of python scripts relating to bones.

(as repsac indicates "First pass at adding support for configurable bone influences")

Gareth's picture

scrub the json..... coolada.dae

Checked the Coolada export (I should have observed your  plugin_fu advise ).. and it appears to export the whole works.... including the bones (though it does not display them to screen).

This is the current export :-

Yes_yes we all know that this is the normal first birth of our vinmoov... its a start...

GroG's picture

Lost in Translation

Hmmm..  What you did with the editor looked amazing, so I started clicking down the road, but seem to have taken a wrong turn.

I can't seem to get the editor to work with any of my Collada exports :P

Would you believe a InMoovHead looks like this?

Fading back 20 yards and punting .. I exported the omnipreset "Cube" ..
And what did I get ?

I don't think its there ...

I'm using the latest Blender 2.76b  .. perhaps I'm in version hell ?

Gareth's picture

Blender scale factor

Firstly Latest Blender works fine....

Two things come to mind here :-

(1) with three.js  there are only three mouse controls :-

  • Zoom with middle wheel
  • Left click whilst dragging mouse around gives rotation of whole field of view
  • Right click whilst dragging translates the whole field of view

     Zoom & home on into the middle axis...... when I first imported the Coolada I could not see the meshes either.

(2) By default blender scales most things to a very small level (I have to scale all my stl's by 1000 before I post to thingiverse for example). I would suggest adjusting the scale between (50 to 150) before exporting to Coolada.

    

GroG's picture

Ahahahah, that helped ! ...

Ahahahah,

that helped ! ...  Now I have a Big Head ! 

Gareth's picture

Just for info >>75% ready.....

Sorry for delay.......Another 25% of tweaking to get new three.js Vinmoov into its full Java jacket.....

kwatters's picture

wow..

Wow, this is amazing!  Great work so far, I'm excited to see where it all goes :)

Gareth's picture

Them bones.... git three.js resource

Github document/examples resource for three.js

The boney part :- https://github.com/josdirksen/learning-threejs/blob/7469a6010b0a19949d1c3ea402a604d28f6a1905/chapter-09/12-bones-manually.html

and stackoverflow seems to hit the nail on the head if its true :-

http://stackoverflow.com/questions/19031198/three-js-attaching-object-to-bone

hairygael's picture

Woouiii, I am going to follow

Woouiii, I am going to follow this very closely!

I don't know if this can help but, Blender can export OBJ files  keeping all the components and bones. At least it is what I use when I need to 3D print in colors and to conserve all the textures and meshes in place. I also use the OBJ exporter to create my 3D PDF files.

GroG's picture

Wooouiii ! :D Ahoy Gael !

Wooouiii ! :D

Ahoy Gael ! :)

Ya one click worky, all in one, Virtual InMoov would be a huge benefit !

Right now its sooo many options of export & so many options of import ...

Three.js is the library to target, now its a matter of finding the language between the two systems (Three.js & Blender) 

This is what I've found so far.

With Gareth's exported finger + bones I have successfully put a collada loader and loaded the file in Three.js.  Got the Chrome Dev Tools to work too, so now I can inspect what exactly is loaded by the loader..

The collada gave me a scene with 6 children :

0 - Spot_001 
1 - i01_right_hand_pinky7_000
2 - i01_right_hand_pinky5_000
3 - i01_right_hand_pinky3_000
4 - Armature
5 - Spot
 
I assume 0 & 5 are lights
 
Gareth wrote me the following map :
Here is the pinky finger isolated , with the rest of the whole skeleton (thought it better to keep the skeleton intact)

so the pinky parts and the pinky bones associated are:-

i01.right.hand.pinky3.000   needs to be connected to bone   i01.rightHand.pinky.base

i01.right.hand.pinky5.000   needs to be connected to bone   i01.rightHand.pinky.mid

i01.right.hand.pinky7.000   needs to be connected to bone   i01.rightHand.pinky.tip

digging deaper into i01_right_hand_pinky7_000 I can see it has a THREE.Matrix4 - 4 matrix is typical for tranformations and a bunch of nice big values, so perhaps its as easy as changing the values or using a Quaternion to modify then update the position ...  

We'll push and pull on it a bit...