3D sound localization

Some weeks ago we discussed about an interesting project that is called HARK. 

It's an open source project created by Honda Research Institute in Japan.

A few videos that describes what can be done: 

Don't worry if you don't speak Japanese, I think you still will be impressed: Watch it and be impressed.

https://www.youtube.com/watch?v=bi4ACLfaWy0

https://www.youtube.com/watch?v=rhBnLySsmJk&t=286s

https://www.youtube.com/watch?v=9v5RUOrkyhw

The software can be downloaded from here if you se Linux ( perhaps also Raspbian )

http://www.hark.jp/wiki.cgi?page=Softwares#p1

or the Windows installer: 

There are several prerequisites, so make sure that ypu download and install them first.

http://www.hark.jp/wiki.cgi?page=Softwares#p9

http://www.hark.jp/wiki.cgi?page=SupportedHardware

What's interesting is that you can use a cheap PS3 eye microphone array so cost to get started is very low.

I received a PS3 eye in the mailbox today, so I haven't done any experiments yet. I made this blog post to keep track of where to find the information about HARK. 

First step is to download and install the Hark Software from the localization above. I tried the Windows version and that went OK. The software need a Transfer function before it can be used. The transfer fuction is generated by the software, using a process of playing a sound several times from different locations in the room. This process is described here: 

https://www.youtube.com/watch?v=9v5RUOrkyhw

After listening to this video, I continued to read the documentation and found that several different transfer functions already exist for the PS3Eye. So I will try to use the Sound source localization and separation files that have been crated by using the TSP process. I.e based on actual recordings, not geometric calculations.

To learn how to use Hark the HARK Cookbook is useful. 

http://www.hark.jp/document/2.3.1/hark-cookbook-en/

However it assumes that you already know how to use HARK Designer. Since I'm new to Hark, I started with reading the Help files for Hark Designer combined with the instructions from the Cookbok to be able to make a build a Sound recording network. Hark designer is used to build a network that describes what you want to do, using a visual programming tool. 

http://www.hark.jp/document/hark-designer-en/harkdesigner.html

 

Stay tuned. Updates will come when I have more info. 


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
GroG's picture

Great stuff Mats !

Great stuff Mats ! ...  
Thanks for posting.  

Speaker localization and speaker identity is another great exciting frontier of learning !

Excited to see what you discover.

So far,
I see the links to source and a windows installer is here - http://www.hark.jp/wiki.cgi?page=Softwares 

 

From the windows installer (which appeared to install quite a lot of stuffs)
I got 3 program links

 

The harktool4 came up with a ui

A profile creation tool for specific microphone array setups ?

The harkjulius gave me this error

and the hark designer gave me this error

Both likely to be pathing problems ...    looks like exciting exploration !

Mats's picture

Installation instructions for Windows

You need to download and install this also:

  •  
  •  

As described in the documentaion:

 http://www.hark.jp/wiki.cgi?page=Softwares#p9

GroG's picture

You mean I should RTFM it!

You mean I should RTFM it!

Mats's picture

LOL

Yepp.