Finally... Yay !
This morning I finally got this work end to end with a streaming 3D point cloud. It was slow, but it worked ! The kinect is fast but the display is painfully slow. If a robot was using the 3D cloud, it wouldn't need a display anyway and there is probably display acceleration software I could be using like JOGL, but its a start !