Supervised Training and Local Classifiers

Ahoy !
I have been recently busy refactoring the OpenCV service, and in the process I decided to make a "Training" filter.  
 
Kwatters made a great demo with interactive re-training, where he interactively presented pictures of different people he wanted the computer to recognize.  
 
The training system I am working on, constantly collects samples, and attempts to categorize them.  The computer tries to identify things it sees and puts the images into classifier directories which are prefixed by an underscore "_"  (e.g. _faces).  I can then, at my convienence, review how the filter did.  Pictures which I feel the filter guessed correctly, I will put in the class directory without the underscore (e.g. faces). Moving the files automatically, causes the filter to retrain on the new material.  Images for which the filter got wrong, I will delete or leave, since the underscore directories are only for guesses, and the ones without underscore are for training.  Hopefully this will cause the filter to get better through positive reenforcement. 
 
Looking at the thumbnails, I may decide to create a new class.  This is as simple as creating a directory with the desired name and dumping the appropriate pictures previously captured into it. 
 
If the method of training is easy enough, it will lead to very localized training.  For example, you may recognize images of certain doors, walls, floors etc from your halway when the robot travelled to another room.  You could grab all the images and dump them into a halway directory. Your collection of images is not an individual object, but "your" hallway is a group of very specific parts (your hallway floor, your hallway ceiling, your hallway doorway, etc), perhaps the robot can determine what room of your house it is in from this training.
 
 
I'm hoping to make the filter recursive so that many subclasses can be created. Faces will have specific people's faces, Room will have specific rooms, rooms could have directions (north wall etc..) - you would just need to make the directory and dump the pictures collected (typically manually sorted from the directory above marked with an underscore).
 
Kwatters sent his images to a search engine, which I completely agree is the better location in future, however, there are certain aspect of just being able to move files around on the filesystem which allow easy interaction and training.  I think a better solution than my file system process, would be to have the filter or service "send" you the data in the form of a large photo tiled webpage, where you could easily supervise and train in a similar manner.
 
time to get back to work on it ;)
 

One challenge was resizing and cropping appropriately.  Most "training systems" want all there images same color, format, size.  OpenCV takes care of the format and color (grey) is trivial to handle - but size was a little more tricky than I originally thought.  Since this system is potentially getting images from a variety of sources, the sizes may be very different.

But its close to the same algorithm which makes thumbnails. 

0001353-000.jpg is big (1752x790) - the top set is medium (215x215) and the bottom set is very small (67x67)
And here is after normalization - all images 256x256


Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.
hairygael's picture

Wow great project! Will you

Wow great project!

Will you make this available in Nixie or further release!

The training by vocal via big pictures could also help to classify info in a faster form.

Changing the underscore via vocal, that is what I mean.

GroG's picture

I think because all of the

I think because all of the opencv fixes - it will be a requirement to add it to Nixie.
Ya . if InMoov tiled a bunch of pictures and said ..

"Are all these pictures of your rabbit ?"

You could say "yes" or "3 7 & 4 are not" or "none are"

kwatters's picture

fingers are people too!

I think the idea of the manual review before assuming the training image is good.  Of course, you can probably guess what I'll say next, is that , we should store all this data in solr :) 

Great work Grog!  Sounds ilke we're getting closer to a merge of the opencv fixes, i'm very excited about that.  I have a DNN face detection opencv filter ready to add also.. it seems a lot more accurate than the haar classifier that we currently have.

keep it up!

GroG's picture

Totally agree - they need a

Totally agree - they need a search engine and transitive references, and lots of meta-data, and .. and.. and ... more stuff !   

I know I know ... "Its a search engine" :)