Is it difficult for robots to learn to walk? NVIDIA uses virtual obstacles to train quickly


Tencent technology news on October 13, for humans, walking with legs is almost an instinct, and they can easily cross obstacles, climb slopes or stairs. But for robots, especially four legged robots such as Boston power’s spot, it is very difficult to learn how to walk correctly. However, NVIDIA, an American chip giant, is teaming up with researchers at the Federal Institute of technology in Zurich, Switzerland to train a quadruped robot called anymals using a virtual obstacle course.
The scenes of more than 4000 dog like robots marching forward will give people a vague sense of threat even in the simulation. But this may point the way for machine learning new skills. The virtual robot Corps was jointly developed by researchers from the Federal Institute of technology in Zurich, Switzerland and NVIDIA. They use roaming robots to train an algorithm, which is then used to control the legs of robots in the real world.
In the simulation, these anymals machines face many challenges, such as slopes, steps and steep hillsides in the virtual landscape. Each time robots successfully overcome the challenge, they will “advance” to the more difficult level, and then promote the control algorithm to become more complex. During training, the robot can easily master the skills of going up and down stairs, but it takes longer to overcome more complex obstacles. Coping with slopes has proved particularly difficult, although some virtual robots have learned how to slide down slopes.
When the final algorithm was transferred to the real version of anymal, it could navigate between stairs and blocks, but encountered problems at higher speeds. Anymal is a four legged robot, the size of a dog, with sensors on its head and a detachable manipulator. The researchers attribute this to the fact that the way sensors perceive the real world is not accurate enough compared with simulation.
Similar training can help robots learn a variety of useful skills, from sorting parcels, sewing clothes to harvesting crops. The project also reflects the importance of simulating and customizing computer chips for the future development of Applied Artificial Intelligence (AI).
“At a higher level, fast simulation is really great,” said Pieter abbel, a professor at the University of California, Berkeley and co-founder of covariant, a company that uses AI and simulation technology to train robot arms to select and sort items for logistics companies. He said that researchers at the Federal Institute of technology in Zurich, Switzerland, and NVIDIA had “achieved a good acceleration.”
AI shows great potential in training robots to complete real-world tasks, which are not easy to write into software or require some form of adaptation. For example, the ability to grab clumsy, slippery, or unfamiliar objects is less likely to be written to code.
4000 simulated robots were trained in reinforcement learning, an AI method inspired by the research on how animals learn through positive feedback and negative feedback. When robots move their legs, an algorithm will judge the impact on their walking ability and adjust the control algorithm accordingly.
These simulations run on NVIDIA’s dedicated AI chip rather than the general-purpose chip used in computers and servers. Therefore, the researchers say they can train robots in less than one percent of the time normally required.
Using specialized chips also poses challenges because NVIDIA’s chips are good at rendering graphics and running key calculations of neural networks, but they are not suitable for simulating physical properties such as climbing and sliding. As a result, researchers have to come up with some smart software alternatives. Rev lebardian, vice president of simulation technology at NVIDIA, said: “it took us a long time to get things done.”
Simulation, AI and special chips may promote the improvement of robot intelligence. NVIDIA has developed software tools to make it easier to simulate and control industrial robots using its chips. The company also built a robot research laboratory in Seattle, and also sold chips and software for self driving cars.
Unity technologies, which develops 3D video game software, is also involved in developing software suitable for robotics experts. Danny Lange, the company’s senior vice president in charge of AI business, said that unity technologies noticed that many researchers were using its software for simulation, so they made it more realistic and compatible with other robot software. Unity technologies is currently working with algoryx, Sweden, which is testing whether reinforcement learning and simulation can train forestry robots to pick up logs.
Reinforcement learning has existed for decades, but due to the progress of other technologies, many AI milestones worthy of attention have been produced recently. In 2015, reinforcement learning was used to train a computer to play go. Recently, it has been put into practical application, including chip design automation that requires experience and judgment. The problem is that this learning method requires a lot of time and data support.
For example, open AI spent more than 14 days training a manipulator to operate the Rubik’s cube in a rough way through intensive learning when multiple CPUs were running at the same time. Every time robots receive retraining, they have to wait for two weeks, which may discourage enterprises from using robots. Early efforts to train robots with reinforcement learning dispersed this process to several real-world robots. The improvement of physical simulation made it possible to accelerate learning in virtual environment.
MIT student Andrew Spielberg said the new work was “very exciting for end users”. He used similar simulation methods to provide new physical designs for robots. He pointed out that a research team of Google has done relevant work to speed up the learning speed of the robot by splitting the robot and running it on the tensor processing unit chip customized by the company.

Tully foot manages the widely used open source robot operating system at the open robotics foundation. He said that simulation is becoming more and more important for business users. “Verifying software in real scenarios can save a lot of time and money before deploying to hardware”. It can run faster than real-time and will never damage the robot. If an error occurs, it can be reset automatically immediately.
But Ford added that moving robot learning to the real world would be more challenging. “There is much more uncertainty in the real world. Dirt, light, weather, uneven hardware and wear all need to be tracked,” he said
Le baredian, vice president of NVIDIA, said that the simulation used to train walking robots may eventually affect the design of relevant algorithms. He said: “the virtual world is valuable to almost everything, but one of the most important is to build an amusement park or training ground for the AI we want to create.” (reviewed by Tencent technology / Jinlu)