When developing new technologies, computer scientists and roboticists often draw inspiration from animals and other living organisms. This allows them to artificially replicate complex behaviors and locomotion patterns to enhance their systems' performance, efficiency and capabilities.
Researchers at Zhejiang Sci-Tech University and University of Essex have recently developed a reinforcement learning technique that can be used to control the movements of a beaver-inspired, single-legged robot. Their method, introduced in a paper published in Robotics and Autonomous Systems, allows the robot to autonomously learn how to perform swimming motions that resemble those observed in beavers.
"In this study, we introduce a biological-inspired reinforcement learning control method to model the motion of underwater robots," Gang Chen, one of the researchers who carried out the study, told TechXplore. "This method is mainly based on one of our previous works studying the motion of beavers, published in Springer Link's Journal of Intelligent & Robotic Systems."
Underwater robots such as the one created by Chen and his colleagues are nonlinear systems, and their movements involve complex hydrodynamics. Accurately modeling their motion can thus be a very complex and challenging task that involves significant computing efforts.
In contrast with other models to guide the motion of underwater robots introduced in the past, the approach devised by Chen and his colleagues does not require the integration of complex motion models based on hydrodynamics. This is mainly because it relies on simplified joint angle representations that dynamically replicate the swimming motion of beavers. These joint representations make the model easier to train while also reducing the robot's ineffective motions during training.
"By combining reinforcement learning with the mechanisms underpinning the swimming behavior of beavers, our method implements the robot's swimming control as quickly and operably as possible," Chen explained. "Its most notable and unique advantage is that it can avoid building complex motion control models and quickly realize the swimming control of a beaver-like, single-legged robot."
Chen and his colleagues evaluated their beaver-inspired reinforcement learning-based method in a series of experiments, using a single-legged robotic platform. Their results were very promising, with their approach resulting in effective beaver-like swimming motions that improved the robot's locomotion.
In the future, the method introduced by this team of researchers could be used to improve the performance and movements of other one-legged robots designed to operate in water. In addition, their work could inspire the development of similar approaches to control the movements of other underwater robots.
"In our future work, we plan to improve the structure and performance of the beaver-like swimming robot," Chen added. "We would also like to investigate ways to improve the intelligence behind robotic swimming motions using reinforcement learning, not only focusing on the robot's swimming velocity, but also on swimming stability, trajectory planning, and obstacle avoidance, all within a real underwater environment."
A technique that allows legged robots to continuously learn from their environment
More information: Gang Chen et al, Reinforcement learning control for the swimming motions of a beaver-like, single-legged robot based on biological inspiration, Robotics and Autonomous Systems (2022). DOI: 10.1016/j.robot.2022.104116
Gang Chen et al, Design of Beaver-like Hind Limb and Analysis of Two Swimming Gaits for Underwater Narrow Space Exploration, Journal of Intelligent & Robotic Systems (2022). DOI: 10.1007/s10846-022-01610-7
© 2022 Science X Network