Humanoid robots arrive in factories. Who is responsible when they fail? – Unite.AI

A robot can successfully complete a performance test and yet become a safety hazard if it faces an unexpected obstacle, a change in its behavior due to a software update, or a real worker. The question is not whether humanoid robots can perform a task anymore, but how they can do it. Whether they can do it safely or not and who is responsible if something goes wrong.

Agility Robotics and FORT Robotics announced a partnership in October to improve the safety architecture of Agility’s Digit 5 ​​humanoid robot. They planned their approach by incorporating onboard safety systems and external interfaces, communications and manual control. The development marks a significant shift: robots must be safe in human workplaces, and that means more than just the robot itself.

However, technology is not enough to answer the question of liability. When a robot makes a mistake based on a faulty component misidentified by an AI system, the manufacturer, software developer, system integrator, and employer using the system could be held liable.

When AI decisions become physical risks

Traditional industrial robots are typically found in closed work cells, where they are isolated from workers by barriers and safety devices. Humanoid robots represent another challenge: they would have to navigate through environments built for humans, perform different tasks and adapt to different environments.

This flexibility creates new risks. If false detection occurs, it could lead to unnecessary movement. A software update could alter a robot’s behavior. Changes to plant layout may create hazards not present in previous tests.

Dana R. Wolfe, Vice President and Head of Industry, GFT Americas, told Unite AI that preventing unsafe interventions requires multiple layers of protection, such as deterministic software guardrails, factory-specific confidence thresholds, sensor fusion, and stepwise validation before deployment.

There is no way to completely avoid false positive results, Wolfe said. Rather, manufacturers need to ensure that errors do not result in unsafe physical movements.

Trust thresholds and human authority should be part of the system’s security architecture, as should hardware interlocks, Wolfe said.

Safety standards must keep pace

Current industrial robotics standards offer a valuable foundation, but are not necessarily the solution to all the problems presented by mobile humanoids.

The International Organization for Standardization (ISO) has published safety requirements for industrial robots, and ISO 10218-2 covers the integration of robots into complete systems.

These standards, however, do not address all applications or risks of mobile robots working with humans.

Independent testing, scenario-based validation and phased workplace pilots are especially important as industry practices evolve, Wolfe said.

Security assessments should also be repeated if changes to the AI ​​model, software, movement restrictions, tasks, or factory environment may render previous security assessments invalid.

After a major change, a robot that was safe under one set of conditions may no longer be safe under the new set of conditions.

Human oversight must mean more than just approval

It’s impractical to monitor every decision made by an AI system, but when people aren’t involved in critical decisions, another challenge arises: Accountability becomes harder to prove.

Adrian Figueroa Ramos, co-founder and CEO of Flexio and Routina, said the focus of oversight should be on decisions that present the greatest uncertainty or risk.

“In today’s AI world, the volume of results is more than a human can process. Reviewing everything makes the human the bottleneck, but human judgment is still needed for high-risk or low-trust decisions,” he told Unite AI.

This is especially true for physical AI. Human oversight should include appointed individuals with the power to override a system and ensure decisions are documented and audited and not simply click an approval button.

Who takes responsibility when a robot breaks down?

Responsibility should be shared based on each party’s control over the system.

For example, manufacturers are responsible for creating and documenting safety measures. Changes that may affect perception or movement should be tested by software developers.

The entire system must be validated in the environment intended by the integrator. Employers have a responsibility to train workers, ensure that safeguards are in place and that procedures are established to stop unsafe operations.

Cyber ​​security is also important. Operational technology may be vulnerable to problems with sensors, software and external systems connected to the robots. A security failure that impacts physical behavior is a workplace safety issue as well as an IT issue.

Humanoid robots are expected to become more flexible and productive in factories in the future. However, adoption will not be based solely on performance measures. Clear responsibilities, independent validation and safeguards that are maintained throughout the working life of a robot are needed.

The question is: can a robot make mistakes? It is, if the people who use it have created a system that will hold them accountable for such errors.