The incidents, all taking place in 2016, are some of the early growing pains of autonomous robots entering into more real-world environments. For decades, increasing autonomy has been the trend in the controlled environments of manufacturing. On factory lines, where there is no variation in the environment or in the task the robot needs to perform, human workers have been replaced by machines as often as possible. In recent years, advances in machine learning have enabled robots to wheel out of those fully controlled environments into semi-structured realms that are mostly predictable but where things move around just enough to throw some doubt into the equation.
New workplaces for today’s autonomous machines are predominantly warehouses. There were about 4,000 warehouses that counted robotic workers among its staff in 2018, projected by ABI Research to ramp up to about 50,000 warehouses hosting 4 million commercial robots by 2025. Retail environments are also becoming populated with robots. In April, Walmart announced a rollout of Bossa Nova Robotics units to 5,000 of its stores, mostly in the domestic US market. The steel workhorses will replace lower-level jobs: taking inventory, unloading trucks, sorting merchandise, and cleaning up messes.
If that sounds like your job, this could be bad news – robots are getting their resumes ready to compete with you for your next position. The good news is that you might be able to get a new job as the commander of a robot squadron – if, that is, Silicon Valley–based startup Formant is right about how humans and robots will collaborate in these semi-structured workplaces in the future. Founder and CEO Jeff Linnell is an ex-Googler, and he wants to allow humans to use their natural intuition to enhance robots in environments that are sometimes difficult for a machine to predict. The idea behind Formant’s robot observability platform is that when some variable arises and the robot isn’t sure if it’s about to zoom gracefully down a hallway or tip into the fountain, it can just call for help.
“The next decade of robotics will be driven by semi-autonomy. We’re at least a decade away from fully autonomous. So the way to bootstrap with robotic applications is to rely on human assistants,” Linnell said in a phone interview. “We’re going to rely on human assistance and human intervention to help robots do their jobs.”
Today’s robots can do about 90% of their tasks in workplaces such as construction sites or retail stores, Linnell estimates. Unless the gap on that last 10% is closed, however, it’s still useless, as the robot could get stuck not knowing how to align its forklift with a pallet or, worse yet, risk the safety of a human colleague. Until robot applications can offer 99.99% reliability, they’re not good enough for industry. But what if a human were to fill in that last 9.99% for a fleet of 20 robots? Or 100?
“It’s not actually the one-to-one relationship that’s interesting but the one-to-many relationships,” Linnell says. “If you’re digging a foundation for a house by hand, you can dig one. Or you could dig 100,000 by being a commander of a robotic fleet. We can use our intellect to enhance robots. We’re very unreliable in some ways, but we’re incredibly intuitive. We can be context aware in a way that a robot can’t account for in all the edge-case scenarios. The repeatability of robots and the intuition of humans is like cheese and macaroni.”
Formant’s solution works by running a small client or a robot that broadcasts information from its sensors and log files back to the cloud. The software parses the data in a way that helps a human operator make sense of it – almost as if they were there experiencing what the robot was experiencing. The approach isn’t complicated from a technical standpoint, but it is bringing modern cloud tooling to an industry that’s preferred an on-premises approach thus far.
The platform could enable a remote style with flexibility similar to that of an Uber driver, Linnell imagines. Robots in need of assistance could put out the call, and a qualified operator working from home could connect to complete the task and get paid for it. Or the time-series database it keeps could be fit into developer workflows, such as a Slack alert when the battery temperature is getting too hot.
Formant’s software could not only help robots get out of a jam but also help train the robots to be better at their jobs. Linnell describes a robot that isn’t quite sure if its forklift is aligned with the pallet it needs to lift. It takes a picture of the pallet and sends it to its human operator. The person draws rectangles on the picture, identifying the proper place to insert the forks. The robot can now continue its job, and its algorithm is reinforced.
“Having your operators label your own data, experts on your team answering your own questions, those are golden datasets for machine learning operation,” he says.
Whether Formant’s software-as-a-solution model will be embraced by robot manufacturers in the same way it’s proven popular elsewhere is a big question for the startup’s success. It’s possible that firms like Bossa Nova and Knightscope will build their own “observability” solutions. But Linnell’s bet is that they’ll prefer to buy it instead. It follows the general wisdom of other industries going through these digital transformation moments.
“If you build a company right now, you won’t stand up that server rack, you just go to the cloud. We’re providing that same level of reliability and we’re relying on the same back ends,” he says. “Whether the world is caught up to that or not, we’ll see in two or three years.”
Formant promises its clients that they’ll always know where their robots are and how they’re doing thanks to its edge-to-cloud solution. Source: Formant.
In the late eighteenth century, a machine known as the Mechanical Turk toured Europe and amazed crowds as it defeated challengers at chess. The world thought it was a fully automated machine that could beat almost any human challenger for its entire life. But it was later revealed to be a hoax, with different chess masters hiding in the machine, using it to read what was occurring in the game, and then operating it to make moves. In a way, it was the original robot observability platform.
With platforms like Formant putting human intellect into robots’ actions today, people will once again look at robots and wonder if they’re seeing algorithmically deterministic calculations or the intuitive guidance of the man hiding under the hood.