according to watson, it would seem my test image of a twinkie uploaded with the filter of "food" returned with a 60% probability of being a picture containing food. not sure what to think about that...
Where the ROS team is focusing on creating "new services", rolled from scratch in an open source fashioin, MRL has taken a different approach, focusing on creating proxies/shims to packages others have created.
The MRL approach could work if the focus shifted to a "qualitative analysis of existing packages". For example, what are the metrics MRL is using to evaluate speech rec solutions? What is the output for the ones evaluated / integrated? The Darwinian approach is a great option, but it means that the shift to benchmarking 3rd party solutions is taken seriously.
Jeff
ps - I'm not sure if a Twinkie is actually a food either ;-)
I really appreciate your posts, views, and insights. You hit the nail on the head with this observation.
I have been working on a MRL self testing system for a month. I had to drop the virtual InMoov in order to focus on the test harness, which was a difficult thing to do, since vInMoov is fully rigged by Gareth at this point waiting for itegration.
The challenges of evaluating the "quality of existing packages" are many fold, but in order to have any form of success, it MUST be automated. Initially I'm sure the Test service & supporting system will be a bit brittle, but evolution will prevail ! :). After the data begins to flow, we will shape and and tweak occordingly to provide more meaningful feedback, reports, and metrics.
Some other important goals in MRL's roadmap are:
Remain as widely multi-platform as possible. Java has this covered well for non-native/hardware related code - but for the JNI & JNA bindings this must be tested on multiple platforms. This means the Test service must be distributed - and tested on as many platforms as possible in an automated fashion.
Easy of use - this is very important and something which never seemed to be a priority with ROS (although perhaps its changed now) - I like to believe that the concept of robotics is so large and forthcoming, that anyone/everyone could potentially have something to contribute - not just acedemics in narrow projects. MRL does not exactly excel in this area, but I can see potential in community learning and better documentation .. we try to keep it a priority, it certainly effects the evolution of a complex system.
A small point, MRL is not adverse to creating "new services" - but I reliquish the fact that for inumberable fields of robotics I am very far from being a domain expert. Being that so many talented people have created amazing open source resources on the web, we find ourselves using these tools rather than re-inventing them. Our priority will be to interface open source with open source, but the framework will allow "best of breed" systems to integrate. Our plan is to deliver licensing information and infrastructure similar to the open source Eclipse (framework) and its plugin Marketplace.
I offer no answers but only some insights from having done this TOOOO long.
I saw some of the chatter on Java/Scala, serialization methods, etc. We both know this stuff comes and goes. The beauty of interfaces is the ability to hide the language/mechanism. So, more than anything, I'd encourage you to think API first, and continue to diminish the importance of the implementation. As for the JNI issues, I can only offer my sympathies. Personally, I believe that you can kill a project by trying to please too many masters (or platforms). Be careful.
I just finished watching the latest video from Virginia Tech's robotics group. And once again, another humanoid robot that couldn't stand up without a tether; how disappointing. We, the builders, are responsible for doing what others could not. We'll never get there if we don't break the problem into lots of little parts, and distribute the tasks. From that perspective, I fully embrace the idea of MRL being the "integration layer for best of breed components". All good programmers want to build new services; it takes will power to avoid it. Perhaps the one service that MRL should take on is what I call, "The Outer Loop". That is, the control loop that tasks all of the controller services, cognition services, etc. {aka, public static void main} It seems like a simple service but IMHO, it isn't.
Ease of use is a tough one. If you look at the IBM Watson services, they did a great job of giving a non-technical user the ability to test them out (UI+docs, call their API, etc.). Naturally, their more complex tasks require additional knowledge/effort. The decision to focus on ease of use boils down to this: Am i interested in users or contributors? Users (who sing your praises, and curse your failures) are leaches. Contributors join your war march and bring victory. My suggestion is, at an early stage, to focus on contributors who will advance your cause.
twinkies have a 60 % probability of being food
according to watson, it would seem my test image of a twinkie uploaded with the filter of "food" returned with a 60% probability of being a picture containing food. not sure what to think about that...
FAIL
FAIL
MRL - success approach
Where the ROS team is focusing on creating "new services", rolled from scratch in an open source fashioin, MRL has taken a different approach, focusing on creating proxies/shims to packages others have created.
The MRL approach could work if the focus shifted to a "qualitative analysis of existing packages". For example, what are the metrics MRL is using to evaluate speech rec solutions? What is the output for the ones evaluated / integrated? The Darwinian approach is a great option, but it means that the shift to benchmarking 3rd party solutions is taken seriously.
Jeff
ps - I'm not sure if a Twinkie is actually a food either ;-)
Sir, I really appreciate your
Sir,
I really appreciate your posts, views, and insights. You hit the nail on the head with this observation.
I have been working on a MRL self testing system for a month. I had to drop the virtual InMoov in order to focus on the test harness, which was a difficult thing to do, since vInMoov is fully rigged by Gareth at this point waiting for itegration.
The challenges of evaluating the "quality of existing packages" are many fold, but in order to have any form of success, it MUST be automated. Initially I'm sure the Test service & supporting system will be a bit brittle, but evolution will prevail ! :). After the data begins to flow, we will shape and and tweak occordingly to provide more meaningful feedback, reports, and metrics.
Some other important goals in MRL's roadmap are:
You're Right...
I offer no answers but only some insights from having done this TOOOO long.
I saw some of the chatter on Java/Scala, serialization methods, etc. We both know this stuff comes and goes. The beauty of interfaces is the ability to hide the language/mechanism. So, more than anything, I'd encourage you to think API first, and continue to diminish the importance of the implementation. As for the JNI issues, I can only offer my sympathies. Personally, I believe that you can kill a project by trying to please too many masters (or platforms). Be careful.
I just finished watching the latest video from Virginia Tech's robotics group. And once again, another humanoid robot that couldn't stand up without a tether; how disappointing. We, the builders, are responsible for doing what others could not. We'll never get there if we don't break the problem into lots of little parts, and distribute the tasks. From that perspective, I fully embrace the idea of MRL being the "integration layer for best of breed components". All good programmers want to build new services; it takes will power to avoid it. Perhaps the one service that MRL should take on is what I call, "The Outer Loop". That is, the control loop that tasks all of the controller services, cognition services, etc. {aka, public static void main} It seems like a simple service but IMHO, it isn't.
Ease of use is a tough one. If you look at the IBM Watson services, they did a great job of giving a non-technical user the ability to test them out (UI+docs, call their API, etc.). Naturally, their more complex tasks require additional knowledge/effort. The decision to focus on ease of use boils down to this: Am i interested in users or contributors? Users (who sing your praises, and curse your failures) are leaches. Contributors join your war march and bring victory. My suggestion is, at an early stage, to focus on contributors who will advance your cause.
Just my 2 cents; Jeff