Summary
Generative AI is quickly becoming a key part of the next generation of robots, including humanoid robots designed to work around people. But as robots become more dependent on AI models, ensuring their safety is becoming a major challenge.
Unlike traditional software, generative AI systems do not always behave in exactly the same way for every situation. That makes it difficult to guarantee that a robot will always make the correct decision when interacting with humans.
A new startup called Safeworld is working on this problem. The company has emerged from stealth with more than $12 million in seed funding and is developing simulation technology to test how AI-powered robots behave in difficult and unexpected situations.
What Happened?
Safeworld was founded by Dr. Ding Zhao, who leads the Safe AI Lab at Carnegie Mellon University, along with startup executive Kyle Wong and machine learning engineer Simo Rachidi.
The company is focused on a simple but difficult question: How can developers prove that a robot controlled by generative AI is safe enough to operate around people?
Safeworld believes traditional testing methods may not be enough for robots powered by modern AI systems.
The startup has raised more than $12 million in seed funding. The round was led by Shine Capital and a16z Speedrun, with additional investment from Box Group, Carnegie Mellon University Endowment, Innovation Endeavors and SV Angel.
What Is Safeworld Building?
Safeworld is developing a simulation platform that allows companies to test robotic control systems before deploying robots in the real world.
The platform creates virtual environments containing realistic human models. A company’s actual robot software can then be connected to the simulation and tested against thousands of different situations.
The goal is to discover dangerous situations that might be difficult, expensive or unsafe to reproduce with physical robots.
For example, a robot working inside a factory might approach a blind corner. The simulation can test questions such as:
- How fast should the robot move?
- How much stopping distance does it need?
- Can it detect a person approaching from the other side?
- What happens if the person is carrying boxes?
- What happens if someone suddenly falls in front of the robot?
These types of scenarios can be tested repeatedly in a virtual environment.
Why Is Robot Safety So Difficult?
Traditional software can often be tested against clearly defined rules. Generative AI is different.
AI models can produce different outputs depending on the situation, which makes it harder to predict exactly how an AI-controlled robot will behave.
Robots also operate in environments that are much less controlled than traditional software systems.
A factory, warehouse, construction site or home can change constantly. People can walk, run, fall, crouch or carry objects. Lighting and surroundings can also change.
This creates a huge number of possible situations that robot developers need to consider.
Testing Humans in Simulated Environments
Safeworld’s approach involves creating digital versions of real-world environments and placing simulated humans inside them.
The company can use robotics simulation systems such as Genesis or MuJoCo to recreate an environment and then test the robot’s real control software.
Thousands of scenarios can be run to see how the robot responds.
One example is a person falling near a robot.
Testing such an event repeatedly in the physical world would be difficult and potentially dangerous. In a simulation, however, developers can reproduce the same situation many times and change different variables.
The testing can include people with different body sizes, positions, movements and appearances.
Robots Must Handle Unexpected Human Behavior
One of the biggest challenges is that humans are unpredictable.
A person working near a robot may be standing, walking, running, kneeling or crouching. They could also suddenly fall or move in an unexpected direction.
Even physical differences between people can matter.
A safety system may need to account for differences in height, body shape, clothing and other visual characteristics when detecting people.
This is particularly important for robots that work alongside humans in factories, warehouses and construction sites.
Why Third-Party Safety Testing Could Matter
Safeworld believes robot manufacturers may eventually need independent companies to evaluate the safety of their systems.
Robot companies can perform their own internal testing, but an outside organization could provide an additional layer of verification.
Independent testing could also help establish common safety standards across the robotics industry.
a16z Speedrun partner Jonathan Lai argued that the time to establish such standards is now, before robots become widespread in homes and workplaces.
The idea is to identify difficult safety problems before large numbers of robots are deployed rather than waiting for serious incidents to reveal them.
Gritt Robotics Is Already Working With Safeworld
Safeworld is already working with robotics companies, including Gritt Robotics.
Gritt Robotics is developing AI systems for robots used by workers installing photovoltaic panels at large solar farms. The company plans to expand its robots into more complex construction tasks.
Its CTO, Vishal Dugar, said proving robot safety mathematically can be extremely difficult.
Instead, the company believes that many safety questions need to be tested empirically by putting robotic systems through a large number of possible scenarios.
For robots operating next to workers, even a relatively simple problem such as preventing a robotic arm from hitting someone can involve many different situations.
Why It Matters
The robotics industry is moving toward AI-powered machines that can perform increasingly complex tasks.
Humanoid robots and other autonomous machines could eventually work in factories, warehouses, construction sites and homes.
But wider adoption will depend partly on whether people trust these machines.
A robot that works perfectly during a controlled demonstration may behave differently when placed in a busy environment with unpredictable people.
That is why safety testing could become an important part of the robotics industry.
Safeworld is trying to create a way for companies to test these difficult situations before their robots interact with real people.
What Comes Next?
Safeworld is still in the early stages of developing its business.
The company is considering how it will deliver its technology to customers. It could operate primarily as a software platform that robotics companies use themselves, or it could provide more hands-on safety testing services.
As generative AI becomes more common in robotics, demand for these types of safety evaluations could increase.
The bigger challenge will be determining whether simulations can accurately represent the unpredictable conditions robots will face in the real world.
Conclusion
Generative AI could make robots much more capable, but it also creates new challenges around safety and predictability.
Safeworld is trying to address that problem by allowing robotics companies to test AI-controlled machines in realistic virtual environments before deploying them around people.
Its approach could become increasingly important as robots move from controlled demonstrations into factories, construction sites and eventually homes.
The company is still at an early stage, but its founders believe that robot safety needs to be solved before large-scale deployment, not after something goes wrong.