Blended Reality With Robots

We are already living with robots. The future is here, but as William Gibson says, it’s not evenly distributed yet. Or I believe that we often don’t recognize the future when we see it.

How do we recognize robots? We usually look for humanoid robots, the stuff of science fiction. Even the classic robot ‘arm’ is part of a ‘human’. But technically, a robot is simply a machine that ‘senses, thinks and acts’. Even the ISO for industrial robots – the international standard describing industrial robot arms – is somewhat broad in definition. A robot is “an actuated mechanism, programmable in two or more axes, with a degree of autonomy, moving within its environment to perform intended tasks.”

Is a car a robot? Yes. Even without full autonomy, a car consists of many autonomous systems. Elon Musk called the Tesla S ‘a computer in the shape of a car’. But really, it’s a robot.

Is a washing machine a robot? Visually, we would find it hard to think of it as a robot. All the ‘humanoid’ bits are hidden inside a box. Yet the modern washing machine, soon to be a washing-drying-folding machine, is a very sophisticated piece of machinery, sensing, thinking and acting in the environment.

Because we see the world through human eyes, it is very hard for us to see things outside human categories. We divide the world into humans, and things. Robots change everything. As we build robots, we are really reshaping what it means to be human. What does it mean when our devices start to look like humans? What does it mean when they don’t? And what does it mean when we use so many different devices to communicate with other people?

Our technologically blended reality is asynchronous, mediated and indirect. Our technologies allow us to communicate across distance and time, and expand our scale, creating a larger richer world. This is nothing new. Civilization is the story of technology taming space and time.

Since we invented writing, we’ve been able to communicate with other people at a distance, at different times and at larger scale than direct communication. And as we invented reproduction technologies, like the printing press and photography, the scale of our communications increased. This has had a huge impact on the world, reshaping our cultural, religious and political structures.

The last 200 years has seen the introduction of many new communication technologies, telegraph, telephone, radio and television. But one thing they’ve all had in common. Until very recently, we’ve been able to see who is ‘pulling the strings’. The subject or object of communication has been visible or known.

In the last decade, we’ve seen an explosion of information and communication technologies and we’ve gone wireless and unplugged. Internet technologies in the 80s and 90s were supposed to usher in an era of anonymity, but in reality they largely just increased the scale of known communications. And our connections to the devices of communication were much more obvious.

As social beings, our reality is very much defined by our communication technologies. These days, even when we are in the same physical place as other people, we are no longer sharing the same reality. We are experiencing different worlds, as if we were in our own reality bubble.

And even when we are communicating, we are no longer certain to be communicating with other people. Ray Kurzweil predicts that in the future we will mainly communicate with machines and not other people. We will experience this technologically blended reality as an extension of ourselves, as a proxy for other people and has its own ‘alien’ identity.

Sharp’s new Robohon phone, created by Tomotaki Takahashi, is the epitomy of a blended reality device. It acts as an extension of ourselves. It provides a proxy for other people, and it has its own very distinct identity. Our categories of ‘you’, ‘me’ and ‘it’ are more fluid than we think. Robots are blending the me and you into the it.

Heidegger was one of the first to describe technology as an invisible extension of our identity. Heidegger’s hammer is ‘present’ when we look at it and think about it. But when the hammer is in the hand of a builder, then it becomes invisible. The hammer is ‘ready at hand’ when the builder thinks of building, not hammering or the hammer. The tool is well known and the focus is on the task instead. The hammer becomes an extension of our identity, an expression of our intent in the world.

Our technological extensions also augment our senses. A lady with feathers on her hat, as described by Merlau-Ponty, has enhanced her spatial awareness. She has increased her sense of the whereabouts of walls and doorways. Just like the whiskers on a cat, we are augmenting our world with technological whiskers.

Similarly, as technology acts as an extension of others, or a proxy, it also becomes invisible to us. Telepresence robots offer an illusion of real presence and become transparent as technologies. Our focus shifts from the tool to the task. In  this case the task is the social interaction. Suitable Technologies even prefer that we don’t call their telepresence devices robots because they want our focus to be on the experience not the device.

Robots are becoming popular and as more of them enter our world, they bring their very own personalities and appearances. But any device with a screen, or speakers and connectivity, is capable of being a gateway for many other people. We can have relationships that are indirect, asynchronous and at scale. Our relationships can be with you, me and it and many mixtures in between.

We are going to see more and more social robots in the service industry, including health, manufacturing and logistics, and in the consumer end, including the home, retail and hospitality. And we are just starting to understand the scope of this blended technological reality with robots.

People enjoy meeting Savioke’s Relay, the robot butler now at 4 hotel chains in California. You can communicate with Relay, although the robot behaves more like R2D2 than C3PO. Relay is functional too. Relay is designed to deliver small items to guest rooms when the front desk staff are busy.

After collecting a lot of feedback, Savioke find that as well as people enjoying their communication with Relay, they also appreciate not having to communicate with a person at a time when they are not feeling social, ie. late at night. The robot starts to become an extension of their wishes, but still has just enough personality to improve the experience.

A robot like Mabu from Catalia Health is acting as a proxy for a doctor or primary health care physician. Mabu will stay in the home of patients on a specialty pharma treatment where Mabu’s AI engages the patient directly in conversation and it’s only the data that is communicated to the doctor. And while Mabu the robot may sit at home, Mabu the app can travel with the patient anywhere.

And Fellow Robots OSHBot is really mixing all our relationships up. OSHBot can act as a simple extension. When you enter the hardware store you can ask the robot for directions and then simply follow the map. Or the robot can autonomously guide you to the correct location inside the store. You can engage the robot in conversations about the parts you’re looking for.

Robots are great at remembering 10,000s of SKUs and where on the shelves they all are. But people are really great at problem solving and understanding complex communications. So if you ask questions like “What sort of glue should I use on a roof tile like…”, then OSHBot can call an expert in for a video call with you. So you can be talking to both the robot and another person.

For the customer, this is just a great shopping experience. But this could change the nature of daily work for the store associates, leaving them free to focus on solving the things they enjoy, with their social and expert knowledge, rather than walking miles of aisles, tracking thousands of small items.

So robots are really augmenting our reality in a multitude of ways. Robots are the embodiment of information. And in our new blended reality, they extend and augment our senses, they are the proxies or avatars for others. And they also have their very own alien identity.

As a child, I wanted to be an astronaut, to explore the universe and to meet aliens, but it turns out the aliens are here, and they can teach us a lot about what it is to be human. In research areas from neuroscience, to biomechanics and psychology, we’re using robots to better understand humans.