SmartArm’s AI-powered prosthesis takes the prize at Microsoft’s Imagine Cup

A pair of Canadian students making a simple, inexpensive prosthetic arm have taken home the grand prize at Microsoft’s Imagine Cup, a global startup competition the company holds yearly. SmartArm will receive $85,000, a mentoring session with CEO Satya Nadella, and some other Microsoft goodies. But they were far from the only worthy team from the dozens that came to Redmond to compete.

The Imagine Cup is an event I personally look forward to, because it consists entirely of smart young students, usually engineers and designers themselves (not yet “serial entrepreneurs”) and often aiming to solve real-world problems.

In the semi-finals I attended, I saw a pair of young women from Pakistan looking to reduce stillbirth rates with a new pregnancy monitor, an automated eye-checking device that can be deployed anywhere and used by anyone, and an autonomous monitor for water tanks in drought-stricken areas. When I was their age, I was living at my mom’s house, getting really good at Mario Kart for SNES and working as a preschool teacher.

Even Nadella bowed before their ambitions in his appearance on stage at the final event this morning.

“Last night I was thinking, ‘What advice can I give people who have accomplished so much at such a young age?’ And I said, I should go back to when I was your age and doing great things. Then I realized…I definitely wouldn’t have made these finals.”

That got a laugh, but (with apologies to Nadella) it’s probably true. Students today have unbelievable resources available to them and as many of the teams demonstrated, they’re making excellent use of those resources.

Congratulations to Team smartARM from #Canada, champion of today's #ImagineCup! Watch the live show on demand at https://t.co/BLxnJ9FGxJ pic.twitter.com/86itWke2du

— Microsoft Imagine (@MSFTImagine) July 25, 2018

SmartArm in particular combines a clever approach with state of the art tech in a way that’s so simple it’s almost ridiculous.

The issue they saw as needing a new approach is prosthetic arms, which as they pointed out are often either non-functional (think just a plastic arm or simple flexion-based gripper) or highly expensive (a mechanical arm might cost tens of thousands). Why can’t one be both?

Their solution is an extremely interesting and timely one: a relatively simply actuated 3D-printed forearm and hand that has its own vision system built in. A camera built into the palm captures an image of the item the user aims to pick up, and quickly classifies it — an apple, a key ring, a pen — and selects the correct grip for that object.

The user activates the grip by flexing their upper arm muscles, an action that’s detected by a Myo-like muscle sensor (possibly actually a Myo, but I couldn’t tell from the demo). It sends the signal to the arm to activate the hand movement, and the fingers move accordingly.

It’s still extremely limited — you likely can’t twist a doorknob with it, or reliably grip a knife or fork, and so on. But for many everyday tasks it could still be useful. And the idea of putting the camera in the palm is a high-risk, high-reward one. It is of course blocked when you pick up the item, but what does it need to see during that time? You deactivate the grip to put the cup down and the camera is exposed again to watch for the next task.

Bear in mind this is not meant as some kind of serious universal hand replacement. But it provides smart, simple functionality for people who might otherwise have had to use a pincer arm or the like. And according to the team, it should cost less than $100. How that’s possible to do including the arm sensor is unclear to me, but I’m not the one who built a bionic arm so I’m going to defer to them on this. Even if they miss that 50 percent it would still be a huge bargain, honestly.

There’s an optional subscription that would allow the arm to improve itself over time as it learns more about your habits and objects you encounter regularly — this would also conceivably be used to improve other SmartArms as well.

As for how it looks — rather robotic — the team defended it based on their own feedback from amputees: “They’d rather be asked, ‘hey, where did you get that arm?” than ‘what happened to your arm?’ ” But a more realistic-looking set of fingers is also under development.

The team said they were originally looking for venture funding but ended up getting a grant instead; they’ve got interest from a number of Canadian and American institutions already, and winning the Imagine Cup will almost certainly propel them to greater prominence in the field.

My own questions would be on durability, washing, and the kinds of things that really need to be tested in real-world scenarios. What if the camera lens gets dirty or scratched? Will there be color options for people that don’t want to have white “skin” on their arm? What’s the support model? What about insurance?

SmartArm takes the grand prize, but the runners up and some category winners get a bunch of good stuff too. I plan to get in touch with SmartArm and several other teams from the competition to find out more and hear about their progress. I was really quite impressed not just with the engineering prowess but the humanitarianism and thoughtfulness on display this year. Nadella summed it up best:

“One of the things that I always think about is this competition in some sense ups the game, right?” he said at the finals. “People from all over the world are thinking about how do I use technology, how do i learn new concepts, but then more importantly, how do I solve some of these unmet, unarticulated needs? The impact that you all can have is just enormous, the opportunity is enormous. But I also believe there is an amazing sense of responsibility, or a need for responsibility that we all have to collectively exercise given the opportunity we have been given.”

SmartArm’s AI-powered prosthesis takes the prize at Microsoft’s Imagine Cup was first posted on https://techcrunch.com/gadgets/