Training robots in the AI-powered industrial metaverse
Imagine the bustling floors of tomorrow’s manufacturing plant: Robots, well-versed in multiple disciplines through adaptive AI education, work seamlessly and safely alongside human counterparts. These robots can transition effortlessly between tasks—from assembling intricate electronic components to handling complex machinery assembly. Each robot’s unique education enables it to predict maintenance needs, optimize energy consumption, and innovate processes on the fly, dictated by real-time data analyses and learned experiences in their digital worlds.
Training for robots like this will happen in a “virtual school,” a meticulously simulated environment within the industrial metaverse. Here, robots learn complex skills on accelerated timeframes, acquiring in hours what might take humans months or even years.
Beyond traditional programming
Training for industrial robots was once like a traditional school: rigid, predictable, and limited to practicing the same tasks over and over. But now we’re at the threshold of the next era. Robots can learn in “virtual classrooms”—immersive environments in the industrial metaverse that use simulation, digital twins, and AI to mimic real-world conditions in detail. This digital world can provide an almost limitless training ground that mirrors real factories, warehouses, and production lines, allowing robots to practice tasks, encounter challenges, and develop problem-solving skills.
What once took days or even weeks of real-world programming, with engineers painstakingly adjusting commands to get the robot to perform one simple task, can now be learned in hours in virtual spaces. This approach, known as simulation to reality (Sim2Real), blends virtual training with real-world application, bridging the gap between simulated learning and actual performance.
Although the industrial metaverse is still in its early stages, its potential to reshape robotic training is clear, and these new ways of upskilling robots can enable unprecedented flexibility.
Italian automation provider EPF found that AI shifted the company’s entire approach to developing robots. “We changed our development strategy from designing entire solutions from scratch to developing modular, flexible components that could be combined to create complete solutions, allowing for greater coherence and adaptability across different sectors,” says EPF’s chairman and CEO Franco Filippi.
Learning by doing
AI models gain power when trained on vast amounts of data, such as large sets of labeled examples, learning categories, or classes by trial and error. In robotics, however, this approach would require hundreds of hours of robot time and human oversight to train a single task. Even the simplest of instructions, like “grab a bottle,” for example, could result in many varied outcomes depending on the bottle’s shape, color, and environment. Training then becomes a monotonous loop that yields little significant progress for the time invested.
Building AI models that can generalize and then successfully complete a task regardless of the environment is key for advancing robotics. Researchers from New York University, Meta, and Hello Robot have introduced robot utility models that achieve a 90% success rate in performing basic tasks across unfamiliar environments without additional training. Large language models are used in combination with computer vision to provide continuous feedback to the robot on whether it has successfully completed the task. This feedback loop accelerates the learning process by combining multiple AI techniques—and avoids repetitive training cycles.
Robotics companies are now implementing advanced perception systems capable of training and generalizing across tasks and domains. For example, EPF worked with Siemens to integrate visual AI and object recognition into its robotics to create solutions that can adapt to varying product geometries and environmental conditions without mechanical reconfiguration.
Learning by imagining
Scarcity of training data is a constraint for AI, especially in robotics. However, innovations that use digital twins and synthetic data to train robots have significantly advanced on previously costly approaches.
For example, Siemens’ SIMATIC Robot Pick AI expands on this vision of adaptability, transforming standard industrial robots—once limited to rigid, repetitive tasks—into complex machines. Trained on synthetic data—virtual simulations of shapes, materials, and environments—the AI prepares robots to handle unpredictable tasks, like picking unknown items from chaotic bins, with over 98% accuracy. When mistakes happen, the system learns, improving through real-world feedback. Crucially, this isn’t just a one-robot fix. Software updates scale across entire fleets, upgrading robots to work more flexibly and meet the rising demand for adaptive production.
Another example is the robotics firm ANYbotics, which generates 3D models of industrial environments that function as digital twins of real environments. Operational data, such as temperature, pressure, and flow rates, are integrated to create virtual replicas of physical facilities where robots can train. An energy plant, for example, can use its site plans to generate simulations of inspection tasks it needs robots to perform in its facilities. This speeds the robots’ training and deployment, allowing them to perform successfully with minimal on-site setup.
Simulation also allows for the near-costless multiplication of robots for training. “In simulation, we can create thousands of virtual robots to practice tasks and optimize their behavior. This allows us to accelerate training time and share knowledge between robots,” says Péter Fankhauser, CEO and co-founder of ANYbotics.
Because robots need to understand their environment regardless of orientation or lighting, ANYbotics and partner Digica created a method of generating thousands of synthetic images for robot training. By removing the painstaking work of collecting huge numbers of real images from the shop floor, the time needed to teach robots what they need to know is drastically reduced.
Similarly, Siemens leverages synthetic data to generate simulated environments to train and validate AI models digitally before deployment into physical products. “By using synthetic data, we create variations in object orientation, lighting, and other factors to ensure the AI adapts well across different conditions,” says Vincenzo De Paola, project lead at Siemens. “We simulate everything from how the pieces are oriented to lighting conditions and shadows. This allows the model to train under diverse scenarios, improving its ability to adapt and respond accurately in the real world.”
Digital twins and synthetic data have proven powerful antidotes to data scarcity and costly robot training. Robots that train in artificial environments can be prepared quickly and inexpensively for wide varieties of visual possibilities and scenarios they may encounter in the real world. “We validate our models in this simulated environment before deploying them physically,” says De Paola. “This approach allows us to identify any potential issues early and refine the model with minimal cost and time.”
This technology’s impact can extend beyond initial robot training. If the robot’s real-world performance data is used to update its digital twin and analyze potential optimizations, it can create a dynamic cycle of improvement to systematically enhance the robot’s learning, capabilities, and performance over time.
The well-educated robot at work
With AI and simulation powering a new era in robot training, organizations will reap the benefits. Digital twins allow companies to deploy advanced robotics with dramatically reduced setup times, and the enhanced adaptability of AI-powered vision systems makes it easier for companies to alter product lines in response to changing market demands.
The new ways of schooling robots are transforming investment in the field by also reducing risk. “It’s a game-changer,” says De Paola. “Our clients can now offer AI-powered robotics solutions as services, backed by data and validated models. This gives them confidence when presenting their solutions to customers, knowing that the AI has been tested extensively in simulated environments before going live.”
Filippi envisions this flexibility enabling today’s robots to make tomorrow’s products. “The need in one or two years’ time will be for processing new products that are not known today. With digital twins and this new data environment, it is possible to design today a machine for products that are not known yet,” says Filippi.
Fankhauser takes this idea a step further. “I expect our robots to become so intelligent that they can independently generate their own missions based on the knowledge accumulated from digital twins,” he says. “Today, a human still guides the robot initially, but in the future, they’ll have the autonomy to identify tasks themselves.”
This content was produced by Insights, the custom content arm of MIT Technology Review. It was not written by MIT Technology Review’s editorial staff.