DataScientist.blog

I recently saw a LinkedIn post about Tesla’s Optimus robot, and it got me thinking about the machine learning methods used in these advanced humanoid robots. As a big fan of movies featuring robots like “I, Robot” “Ex Machina” and, of course, “Terminator” this topic fascinates me. Working my whole career in the automotive industry, I’ve seen plenty of robots, but none that look or move like humans. We use industrial robots with multiple axes, and they don’t need to be humanoid because the tasks they perform don’t require a human form.

Tesla’s Optimus resembles Boston Dynamics’ Atlas, though in my opinion, Optimus is still far from Atlas’s capabilities. To see what I mean, check out the LinkedIn post about Optimus and compare it to a YouTube video of Atlas. As I pondered where humanoid robots could actually be useful, I did some research and stumbled upon some surprising information. I realized that reinforcement learning (RL) plays a major role in these robots, and when you think about it, it makes perfect sense.

RL is revolutionizing how robots interact with their environments. For example, Boston Dynamics’ Spot uses RL to navigate industrial settings. Spot autonomously patrols facilities, collects data, and identifies issues before they become significant problems. This adaptability makes Spot invaluable in maintaining industrial safety, as it can navigate stairs, avoid obstacles, and even operate in hazardous areas where human presence would be risky. At least this is what Boston Dynamics claims, though I find it hard to envision such a use case in the automotive industry. These robots are likely very expensive (I couldn’t find any prices for Atlas nor Spot), and simple sensors costing a few hundred euros can achieve the same result. However, it seems that industries like oil, gas, and food are well-suited for such robots. Boston Dynamics’ whitepapers reveal that these robots are primarily used for detecting leaks in pipelines and operating in dangerous environments, such as those with high pressures. Despite my skepticism about the automotive industry, it turns out that companies like Mercedes Benz, Hyundai, and, of course, Tesla are exploring these technologies. This made me think that perhaps I was missing something. If these big names in the automotive industry see potential, there must be something there, even if it’s not immediately obvious to me.

In sectors outside manufacturing, humanoid robots have shown promise in healthcare. Initially, I was surprised that robots are used in healthcare. However, after some thought, I realized it makes a lot of sense. People might feel embarrassed when speaking to a human doctor, and robots could help reduce this embarrassment. For example, researchers have explored using robots to assist people with depression. Patients often feel more comfortable discussing sensitive topics with a robot than a human therapist. These robots, controlled using RL algorithms, ensure that their responses are appropriate and supportive. This approach could make mental health care more accessible and less intimidating for patients. It’s better to talk to a robot doctor than not to seek help at all out of shame.

Additionally, in Iran, researchers use humanoid robots in therapy sessions for autistic children. These robots provide consistent and predictable interactions, helping children improve their imitation and joint attention skills. RL enables these robots to adapt their interactions based on the children’s responses, making the therapy more effective.

Another fascinating use of RL is in manipulation tasks. Robots like TEO, a humanoid robot developed at the Complutense University of Madrid, have been trained with deep Q-learning, a type of RL, to perform tasks such as drawing and painting. At first, I couldn’t understand why a humanoid robot should be able to draw. But after learning that developing robots capable of generating or imitating art has always attracted the interest of both the scientific community and the public—evidenced by the RobotArt competition with a $100,000 prize pool in 2018—I realized why. Instead of following strict rules, TEO learns by exploring different actions and receiving feedback based on the quality of the output. This method allows the robot to develop its own strategies for tasks requiring fine motor skills and adaptability.

While RL offers many advantages, there are significant challenges to overcome. Training robots to perform complex tasks requires extensive data and computational resources. The training process can be time-consuming and expensive, and transferring what a robot learns in simulations to the real world can be challenging due to the unpredictability of real-world conditions.

Despite these challenges, the future of RL in humanoid robotics looks promising. As computational power increases and algorithms become more sophisticated, we can expect robots to become even more capable and versatile. They could assist in various industries, from healthcare and education to manufacturing and disaster response. The continuous learning capabilities of RL will enable robots to adapt to new tasks and environments, making them invaluable tools for addressing complex challenges. It’s a long road ahead before we have robots like in “I, Robot”, but there’s no denying we’re moving in that direction.

However, I remain skeptical about the practical applications of humanoid robots. I understand their potential in healthcare and other fields, but do they really need to walk on two legs and look human? For instance, in treating depression, a purely software-based solution could be just as effective, like Jarvis, Tony Stark’s assistant in “Iron Man.” Also, dog-like robots on four legs might be better suited for dynamic tasks since they don’t need to expend resources on balancing themselves.

In the end, robots like Optimus and Atlas have certainly piqued my interest. Even if I’m not completely sold on their immediate practical applications, it’s hard not to be fascinated by the potential these technologies hold.