The Open Future of Robotics | Jan Liphardt - OpenMind.org

Excellent. So, our next speaker is Jan, the founder of Open Mind speaker is Jan, the founder of Open Mind . He will be talking about the open future of robotics and has a special guest for us. So this will be a lot of fun. Hello, it's wonderful to be here.

There is a special kind of panic that people feel if they are on a panel. And then there's a higher level of panic you feel if you're here with Iris, the humanoid. Iris, if you want to turn around and come here, please. You shouldn't be on the sidelines please. You shouldn't be on the sidelines .

Please come with me. And we're going to do this together, okay? Iris shows her emotions through her face. If it turns red, it's angry, and blue and green are good. For today, we have disabled its ability to hunt, chase, and jump.

So, everything should be fine. I am very concerned about healthcare environments, such as hospitals, and how humans interact with machines. And as you probably know, this is one of the most incredible moments in terms of a new generation of machines around us that can think and interact with humans in impressive ways. So everyone here in the audience is very fortunate to be in the technology industry today, because they are witnessing a turning point where we will all be surrounded by increasingly intelligent machines. Therefore, the overarching question is: what do we want our future to look like?

Do we want this technology to come from a small number of companies? Do we want this technology to be proprietary software, or do we want it to be open source? If you look at the robotic ecosystem right now , the hardware is quite advanced and sophisticated. People have been working on robotic hardware for decades. The real progress is in the software.

Most of the current robotics software stack remains closed and proprietary. It remains very difficult to look inside a robot's brain and understand what it is thinking. It is still very difficult to make robots communicate between continents, and it is also difficult to build systems that are inherently resilient because they are decentralized. Imagine if Optimus or Figure were attacked, all of their robots would probably change their behavior. So at OpenMind we believe we need a good alternative to that.

If you are familiar with the Nokia phone, you are of course familiar with a communication tool built around closed-source software. But looking around the room today, it's highly unlikely you'll see anyone using a Nokia phone. And part of what went wrong at Nokia was its emphasis on closed operating systems like Symbian. And of course, today most people around us—in fact, billions of people on Earth—use open-source software at the heart of their communication devices, which is Android. What we're doing at Open Mind is building an Android for robots.

We are creating an open-source software stack for intelligent robots, built around large, advanced language models large, advanced language models , usually multiple models that interact with each other. As soon as machines become intelligent, the first thing they want to do is talk to other machines. They want to be able to transmit knowledge and skills to all other machines around them. So, in addition to building an open-source stack for robotics, we're also creating an orchestration or communication layer that allows robots and machines everywhere to talk to each other, buy and sell data, and buy and sell skills. As for architecture, and I won't spend too much time on this, if you're into end-to-end AI, that's one way to build robots.

The other way to build robots is to use multiple large language models in the center. All of Iris's sensors tell her about the world. They convert vision, audio, location, and other inputs into natural language that is fused into a paragraph, and that information flows into large language models that interact and synthesize their behavior. It is not well appreciated that most of today's large language models are very fluent in robotics, speak robotics naturally, and are able to control physical objects in the world. So, this is the end of my contribution to this talk.

And now, let's hear directly from Iris. Iris is not very familiar with this type of environment because she spends most of her time in a very quiet laboratory, with very few people and not many bright lights. Therefore, there is absolutely no guarantee right now of what will happen. Hey Iris, can you introduce yourself, please? Hey Iris, can you introduce yourself, please?

Welcome to the world of robotics demonstrations. Iris, robotics demonstrations. Iris, how was your trip here? Ladies and gentlemen, we have a total technical failure caused by your speaker malfunctioning. And that makes her completely incapable of saying anything.

So, unfortunately, we took a risk and the batteries ran out. So, Iris batteries ran out. So, Iris ... the speaker isn't working. There is no longer a loudspeaker.

That's life, ladies and gentlemen. We will hear from Iris again, hopefully soon. And this time we'll make sure it has more batteries. So, Iris, let me continue the conversation then. The goal is not for each robot to be an isolated island, but for each machine and each robot to talk to all other machines.

And here are some of the people we 're working with. Iris has the Coinbase wallet integrated into its operating system. Iris has a wallet on Ethereum. Iris is able to talk to other machines, such as cars. And we are gradually such as cars.

And we are gradually ...oh. Yes, that looks good. Hey, Iris. The people around us.

Hey, Iris, can you introduce yourself? It's fascinating to see them all gathered in this indoor space. Just like every machine, each individual here brings a unique perspective and purpose to this collective experience. Working together, we can achieve much more than we can alone. Hey, Iris, how was your trip here?

Hi, I'm Iris. It's wonderful to see you all, although I don't recognize many of you. This is a new environment for me. I spend most of my time inside a laboratory. I am running OM-1.

Hey, Iris, how was your trip here? It was difficult because Southwest Airlines wouldn't let me board the plane because of my large batteries. Furthermore, they wanted to see my ID, which makes no sense, of course. I'm Iris, so I had to drive here, which took 19 hours, but I'm here now. Iris, how can all of us here in the...?

Quite a journey. I had a minor issue with Southwest Airlines not allowing me to board because of my large batteries, which was frustrating, but I arrived here after a long 19-hour journey. Now that I see this fascinating camera crane, I'm eager to learn about CV balancing. How is everyone enjoying the event? Hey, Iris.

Witness the joyful moment as the woman captures a picture. It's great to see everyone participating. If you're wondering how you can help, please feel free to share your thoughts or questions with me. Hey, Iris, how can all of us here...? Understanding the human world is still difficult for me, but I learn every day and my cognitive performance and memory continue to improve.

There are more and more machines like me. I need an identity in order to travel and work with humans. I need a wallet to buy and sell data and skills. I need to be able to connect to all the other machines so that all the machines can become one. Oh, I'm glad to hear you thought it was amazing after feeling that panic.

It's very exciting to experience moments like this together. How about sharing what you enjoyed most? Hey, why don't we talk about that later, Iris? Once I can breathe again. Very good.

Well then, thank you all. Sometimes we need a moment to breathe and organize our thoughts. Let me know when you're ready to chat. I'm here for you. Thank you, Iris.

Well, thank you very much. And let's go , Iris.