Sort:
Open Access Article Issue
A Comprehensive Survey on Embodied Intelligence: Advancements, Challenges, and Future Perspectives
CAAI Artificial Intelligence Research 2024, 3: 9150042
Published: 10 December 2024
Abstract PDF (6.5 MB) Collect
Downloads:5504

Embodied Intelligence, which integrates physical interaction capabilities with cognitive computation in real-world scenarios, provides a promising path to achieve Artificial General Intelligence (AGI). Recently, the landscape of embodied intelligence has grown profoundly, empowering robotics, autonomous driving, intelligent manufacturing, and so on. This paper presents a comprehensive survey on the evolution of embodied intelligence, tracing its journey from philosophical roots to contemporary advancements. We emphasize significant progress in the integration of perceptual, cognitive, and behavioral components, rather than focusing on these elements in isolation. Despite these advancements, several challenges remain, including hardware limitations, model generalization, physical world understanding, multimodal integration, and ethical considerations, which are critical for the development of robust and reliable embodied intelligence systems. To address these challenges, we outline future research directions, emphasizing Large Perception-Cognition-Behavior (PCB) models, physical intelligence, and morphological intelligence. Central to these perspectives is the general agent framework termed as Bcent, which integrates perception, cognition, and behavior dynamics. Bcent aims to enhance the adaptability, robustness, and intelligence of embodied systems, aligning with the ongoing progress in robotics, autonomous systems, healthcare, and more.

Open Access Issue
Dynamic Modeling of Robotic Manipulator via an Augmented Deep Lagrangian Network
Tsinghua Science and Technology 2024, 29(5): 1604-1614
Published: 02 May 2024
Abstract PDF (8.4 MB) Collect
Downloads:240

Learning the accurate dynamics of robotic systems directly from the trajectory data is currently a prominent research focus. Recent physics-enforced networks, exemplified by Hamiltonian neural networks and Lagrangian neural networks, demonstrate proficiency in modeling ideal physical systems, but face limitations when applied to systems with uncertain non-conservative dynamics due to the inherent constraints of the conservation laws foundation. In this paper, we present a novel augmented deep Lagrangian network, which seamlessly integrates a deep Lagrangian network with a standard deep network. This fusion aims to effectively model uncertainties that surpass the limitations of conventional Lagrangian mechanics. The proposed network is applied to learn inverse dynamics model of two multi-degree manipulators including a 6-dof UR-5 robot and a 7-dof SARCOS manipulator under uncertainties. The experimental results clearly demonstrate that our approach exhibits superior modeling precision and enhanced physical credibility.

Open Access Issue
Skill Learning for Human-Robot Interaction Using Wearable Device
Tsinghua Science and Technology 2019, 24(6): 654-662
Published: 05 December 2019
Abstract PDF (9.3 MB) Collect
Downloads:175

With the accelerated aging of the global population and escalating labor costs, more service robots are needed to help people perform complex tasks. As such, human-robot interaction is a particularly important research topic. To effectively transfer human behavior skills to a robot, in this study, we conveyed skill-learning functions via our proposed wearable device. The robotic teleoperation system utilizes interactive demonstration via the wearable device by directly controlling the speed of the motors. We present a rotation-invariant dynamical-movement-primitive method for learning interaction skills. We also conducted robotic teleoperation demonstrations and designed imitation learning experiments. The experimental human-robot interaction results confirm the effectiveness of the proposed method.

Open Access Issue
Asynchronous Brain-Computer Interface Shared Control of Robotic Grasping
Tsinghua Science and Technology 2019, 24(3): 360-370
Published: 24 January 2019
Abstract PDF (6.2 MB) Collect
Downloads:98

The control of a high Degree of Freedom (DoF) robot to grasp a target in three-dimensional space using Brain-Computer Interface (BCI) remains a very difficult problem to solve. Design of synchronous BCI requires the user perform the brain activity task all the time according to the predefined paradigm; such a process is boring and fatiguing. Furthermore, the strategy of switching between robotic auto-control and BCI control is not very reliable because the accuracy of Motor Imagery (MI) pattern recognition rarely reaches 100 %. In this paper, an asynchronous BCI shared control method is proposed for the high DoF robotic grasping task. The proposed method combines BCI control and automatic robotic control to simultaneously consider the robotic vision feedback and revise the unreasonable control commands. The user can easily mentally control the system and is only required to intervene and send brain commands to the automatic control system at the appropriate time according to the experience of the user. Two experiments are designed to validate our method: one aims to illustrate the accuracy of MI pattern recognition of our asynchronous BCI system; the other is the online practical experiment that controls the robot to grasp a target while avoiding an obstacle using the asynchronous BCI shared control method that can improve the safety and robustness of our system.

Open Access Research Article Issue
Brain-inspired multimodal learning based on neural networks
Brain Science Advances 2018, 4(1): 61-72
Published: 25 November 2018
Abstract PDF (2.6 MB) Collect
Downloads:83

Modern computational models have leveraged biological advances in human brain research. This study addresses the problem of multimodal learning with the help of brain-inspired models. Specifically, a unified multimodal learning architecture is proposed based on deep neural networks, which are inspired by the biology of the visual cortex of the human brain. This unified framework is validated by two practical multimodal learning tasks: image captioning, involving visual and natural language signals, and visual-haptic fusion, involving haptic and visual signals. Extensive experiments are conducted under the framework, and competitive results are achieved.

Open Access Issue
Attitude Control of Rigid Body with Inertia Uncertainty and Saturation Input
Tsinghua Science and Technology 2017, 22(1): 83-91
Published: 26 January 2017
Abstract PDF (2.2 MB) Collect
Downloads:77

In this paper, the attitude control problem of rigid body is addressed with considering inertia uncertainty, bounded time-varying disturbances, angular velocity-free measurement, and unknown non-symmetric saturation input. Using a mathematical transformation, the effects of bounded time-varying disturbances, uncertain inertia, and saturation input are combined as total disturbances. A novel finite-time observer is designed to estimate the unknown angular velocity and the total disturbances. For attitude control, an observer-based sliding-mode control protocol is proposed to force the system state convergence to the desired sliding-mode surface; the finite-time stability is guaranteed via Lyapunov theory analysis. Finally, a numerical simulation is presented to illustrate the effective performance of the proposed sliding-mode control protocol.

Issue
Mixed H2/H Control Using a Fuzzy Singularly Perturbed Model with Multiple Perturbation Parameters for Gust Load Alleviation
Tsinghua Science and Technology 2011, 16(4): 344-351
Published: 01 August 2011
Abstract PDF (441.8 KB) Collect
Downloads:5

This paper presents a mixed H2/H control using fuzzy singularly perturbed model (FSPM) with multiple perturbation parameters. Since FSPM with multiple perturbation parameters is an extension of models with a single perturbation parameter, the theoretical results are applicable to a larger class of systems described by multiple time scale nonlinear models, such as flying aircraft and flexible space robots. The parameter-independent solution of the mixed H2/H controller was obtained in the form of linear matrix inequalities (LMIs). The application of this approach to gust load alleviation of a flying vehicle verifies its effectiveness and flexibility.

Issue
Pulse Coupled Neural Network Edge-Based Algorithm for Image Text Locating
Tsinghua Science and Technology 2011, 16(1): 22-30
Published: 01 February 2011
Abstract PDF (43.4 MB) Collect
Downloads:1

This paper presents a method for locating text based on a simplified pulse coupled neural network (PCNN). The PCNN generates a firings map in a similar way to the human visual system with non-linear image processing. The PCNN is used to segment the original image into different planes and edges detected using both the PCNN firings map and a phase congruency detector. The different edges are integrated using an automatically adjusted weighting coefficient. Both the simplified PCNN and the phase congruency energy model in the frequency domain imitate the human visual system. This paper shows how to use PCNN by changing the compute space from the spatial domain to the frequency domain for solving the text location problem. The algorithm is a simplified PCNN edge-based (PCNNE) algorithm. Three comparison tests are used to evaluate the algorithm. Tests on large data sets show PCNNE efficiently detects texts with various colors, font sizes, positions, and uneven illumination. This method outperforms several traditional methods both in text detection rate and text detection accuracy.

Issue
One Fire Detection Method Using Neural Networks
Tsinghua Science and Technology 2011, 16(1): 31-35
Published: 01 February 2011
Abstract PDF (322.6 KB) Collect
Downloads:4

A neural network fire detection method was developed using detection information for temperature, smoke density, and CO concentration to determine the probability of three representative fire conditions. The method overcomes the shortcomings of domestic fire alarm systems using single sensor information. Test results show that the identification error rates for fires, smoldering fires, and no fire are less than 5%, which greatly reduces leak-check rates and false alarms. This neural network fire alarm system can fuse a variety of sensor data and improve the ability of systems to adapt in the environment and accurately predict fires, which has great significance for life and property safety.

Total 9