nVidia recently announced a whole line of system of a chip boards (similar to the raspi) that help showcase the nVidia GPU. These boards are based on an ARM 64 bit CPU architecture and coupled with an nVidia GPU. (The Nano has a 128 core maxwell GPU.) It's got 4 GB of ram, and uses a standard microSD card for it's main system storage (by default.)
There are some challenges with working on this new board. The biggest challenge is in terms of support for the arm64 processor. Most open source software out there really hasn't caught up to development on a linux based arm64 processor. (Good news is, that much of the support actually does exist, but for arm64 android phones.. no ubuntu linux based.)
So.. that means it's on us to recompile any of the native libraries that we need to to get MRL to be fully worky. MRL uses a lot of platform native libraries. This means we need to have a version of each of those binaries, one for each CPU (and GPU) that we intend to support.
The Nano, at it's onset is actually a very impressive board. I had been used to using the RasPI and it was very clear to me that the RasPI is not a very fast or powerful computer, the nano on the other hand is actually quite nice. I've had it using HDMI providing 4k output to my desktop monitor while working with it, the graphics are crisp. The cuda examples to demo the graphics capabilities of the nano are impressive to say the least.
All that said, the Nano looks like a very viable, and affordable ( ~ $99.00 cost) option for building robots. In comparison to eh raspi, the Nano looks like a super computer, and rightly so. Ok.. enough about that.. onto what we have done and need to do.
Issues to address with MRL on the Nano.
1. opencv needs to be built and packaged and released for arm64 ( I have this done. a nice to have would be to compile in the cuda support by default.. but I think this maybe a topic for another post as I think code changes might need to happen to take advantage of the gpu, if available.)
2. deeplearning4j needs to be built for the linux-arm64 platform. dl4j depends on a lot of packages from the javacpp-presets open source project. (And they have been very active and helpful (@saudet) in helping me though issues building the code. Building DL4j on the nano is going to be more challenging because it depends on an Intel library called MKL and as it would seem that library doesn't work on ARM based processors, it only works on intel x86 based processors. Anyway, I have a locally hacked up version of DL4j that excludes the "MKL" and the "MKL-DNN" dependencies.. In theory, it should be ok to do this for the arm builds.. but we'll see, i suspect more work to be done here.
3. serial ports! The library that we ues for handlig opening, closing, reading and writing is a native library. I am fairly certain, though I have not yet tested, that serial I/O will also need some work to get the arduinos working.
4. others stuff that we care less about now includes things like the leap motion, perhaps some local speech services, the myo band.. etc..
Ok., so here are my running notes for compiling and building various parts of javacpp-presets like OpenCV, and OpenBlas.. ..
Notes On Building JavaCPP & DL4J On The Jetson Nano
1. add some swap space - This should be done on a USB thumb drive that is plugged into a USB3.0 port on the Jetson Nano. When you plug in your usb stick a few things.
* Make sure you format the usb stick as an ext4 file system
You should first umount the usb stick before doing the following steps:
Assuming your usb stick is mounted in /media/user/XXX
umount /media/user/XXX
At that point the usb stick is no longer mounted and you can repartition and format it.
Assuming your USB drive is showing up as /dev/sda , you can run the command
"fdisk /dev/sda"
At that point, you'll be in the menu to add or delete partitions.
"p" will print the current partitons.
"d" will delete a partition ( it will ask you which partition to delete.)
"n" (I think. is a "new partition" which by default will fill the full usb stick and default to a linux partition.)
Delete all existing partitons, and create a single big partition on the usb stick of type "linux"
Next: Format that new partition: (assuming partition 1 on device /dev/sda )
mkfs.ext4 /dev/sda1
If you unplug the usb stick and plug it back in. it will mount automattically in a directory similar to before (but perhaps slightly different.
Let's assume it gets mounted automatically in
/media/user/YYY
then.. you are ready to create your swap file and mount it.
The Jetson nano will automatically mount
dd if=/dev/zero of=/media/user/YYY/swapfile bs=4096 count=1048576
chown root:root /media/user/YYY/swapfile
chmod 600 /media/user/YYY/swapfile
mkswap /media/user/YYY/swapfile
swapon -a /media/user/YYY/swapfile
Note: Do not use the SD card for swap.. this will lead to bad stuff happening to the sdcard.. too many writes occur. I started noticing problems using an SD card for swap space about 2 weeks after starting to use it like that.
TODO: add it to fstabs so it mounts on reboot.
2. apt-get update ; apt-get upgrade
3. apt-get install maven
4. apt-get install default-jdk
5. apt-get install curl
6. apt-get install gfortran
6a. apt-get install cmake
7. get a decent little usb stick.. plug it into the nano and use it for most of the work.
9. make sure you have the proper file permissions on the mounted usb stick. (worse case, just run a chmod 777 ... maybe try to run as root. ) I formatted a new usb stick ( fdisk / mkfs.ext4 ; mounted it in my home dir .. i had problems making a symbolic link on the usb stick i used , perhaps because it was a windows filesystem (fat32?) )
10. in order to build some stuff you'll need more than the 4GB of ram. I added a swapfile to have some temporary memory. Problem is using the sd card for swap memory is not a good thing to do. so I created the swap file on the usb with
cd (to where you have the usb stick mounted.. /media/username/foo or something like that.)
dd if=/dev/zero of=./swapfile bs=4096 count=1048576 (create a 4GB file of zeros.)
mkswap ./swapfile (mark the file that it can be used as swap memory.)
swapon -a ./swapfile (attach the swap file as memory)
Of note, you don't want to do a lot of swapping if you can avoid it.. so . i saw it was recommended to edit
/etc/sysctl.conf and add the line vm.swappiness=10 (maybe some lower value.) This should tell the operating system to be less aggressive about using swap memory. I guess..
after setting that, reboot..
when you run
cat /proc/sys/vm/swappiness you can see the value it is set to. (by default 60 on my setup)
Another random note. I was experiencing lagginess issues with the mouse and keyboard, and sometimes it would even start dropping keystrokes and mouse movements all together.
When I plugged in the thumbdrive, it was next to the bluetooth wireless keyboard/mouse adapter. I think there was actually some RF interference between the two devices when there was a lot of read/write activity on the thumbdrive. Long story short, I added a small usb extension cable to physically move the mouse/keyboard adapter farther away from the thumb drive and the problems seemed to go away.
11. build OpenBLAS , cd javacpp-presets/openblas;
mvn clean install -Djavacpp.platform=linux-arm64 -DskipTests -Dmaven.javadoc.skip=true
... big money.. no whammys..
12. hopefully you see build success..
.... ok.. other random notes as i continue down this path..
for compiling libfreenect
apt-get install libudev-dev libsystemd-dev
...
More to come.. good news is, I do have opencv worky with yolo on the jetson nano now. It was doing 30 frames per second, but the yolo classificatoin was considerablly slower (it's not using the GPU yet.. )
