Artificial Intelligence & Robotics (Current Research)

Extending Autonomy in Seemingly Sensory-Denied Environments

Approximately 60% of the global population resides within 100 kilometers of coastal or estuarine environments. As population density and economic activities in these regions increase, so do the stressors on coastal ecosystems. Consequently, there has been growing attention on the application of intelligent autonomous systems (IAS), including autonomous underwater vehicles (AUVs) and autonomous surface vehicles (ASVs), for the continuous monitoring of aquatic environments. These platforms have facilitated a range of research activities, including the study of physical phenomena such as temperature fluctuations, salinity levels, and harmful algae blooms, as well as the inspection and maintenance of offshore infrastructure, and environmental monitoring and restoration efforts. 

The primary objectives of this proposal are to address the theoretical and technical challenges associated with creating a comprehensive framework for autonomous sampling, navigation, and human-robot collaboration in dynamic marine environments.

To achieve these objectives, we propose a research agenda that will be focused on the following research topics:

• Research Topic 1: Autonomous Robotic Information Gathering Systems

Overview: Robotic Information Gathering (RIG) is a process of optimizing an information-theoretic metric from efficient exploration of a region of interest by robots with motion constraints while considering inferences from a probabilistic model within a limited mission time. Effective data collection in collaborative information-gathering systems relies heavily on maintaining uninterrupted connectivity. Yet, real-world communication disruptions often pose challenges to information-gathering processes. 

Contributions: In our recent contribution [1], we introduce a novel method - a limited communication decentralized information-gathering system for multiple robots to explore environmental phenomena characterized as unknown spatial fields. Our method leverages quadtree structures to ensure comprehensive workspace coverage and efficient exploration. Unlike traditional systems that depend on global and synchronous communication, our method enables robots to share local experiences within a limited transmission range and coordinate their tasks through pairwise and asynchronous communication.

• Research Topic 2: Learning-based Approaches for Autonomous Environmental Monitoring

Overview: Over the past few decades, advances in machine learning and data processing have significantly contributed to the exploration and sampling of aquatic environments using autonomous robots. However, predicting and estimating phenomena of interest in these environments remain challenging due to their complex spatio-temporal dynamics. In response, this part of my research introduces novel frameworks that address these challenges by leveraging learning techniques and adaptive navigation for improved field estimation and mapping [2, 3].

Contributions: Recently, I have been working on learning-based adaptive navigation for scalar field mapping and feature tracking. Scalar field features such as extrema, contours, and saddle points are essential for applications in environmental monitoring, search and rescue, and resource exploration. Traditional navigation methods often rely on predefined trajectories, leading to inefficient and resource-intensive mapping. In [4], we introduce a new adaptive navigation framework that leverages learning techniques to enhance exploration efficiency and effectiveness in scalar fields, even under noisy data and obstacles.

• Research Topic 3: State Estimation in Environments with Extreme Sensing and Communication Constraints

Overview: Localization in aquatic environments presents significant challenges due to the spatial and temporal variability of sensing, modeling, and prediction, as well as the unavailability of GPS signals. Position prediction errors, often caused by process and sensor noise, can severely impact the success of autonomous underwater operations. In this scenario, how can energy-efficient localization solutions be designed for resource-constrained underwater robots? How can water quality data be employed to create and maintain accurate underwater maps for localization purposes?

Contributions: Motivated by the need for prompt solutions to detect and respond to harmful water events, such as algal blooms or fish kills, our recent work [5] aimed to improve the state estimation of underwater robots by incorporating detailed 3D maps of underwater regions, known as Satellite-Derived Underwater Environments (SDUEs). Our methodology leverages remote sensing imagery from the Sentinel-2 (S2) and Landsat 8-9 (L8-9) satellites with in-situ water measurements from Biscayne Bay, Florida.

• Research Topic 4: Human-Robot Collaboration

Overview: The design of intuitive, robust, and efficient Human-Robot interfaces (HRI) can enable a new generation of applications in critical areas such as precision agriculture, automated construction, rehabilitation, and environmental monitoring [6]. These interfaces, which aim to foster seamless and natural collaboration between humans and robots, rely on key properties like consistency, linearity, and continuity. 

Contributions: Building upon my recent work [7], my next steps will focus on exploring mappings between human and robot spaces and developing the mathematical formulation of these challenges. This aligns with recent advancements in HMD-based immersive teleoperation interfaces, human perception-optimized planning, and strategies that leverage optimal control for teleoperating robots.

Related publications: 

• [1] A. Redwan, P. Padrao, J. Fuentes, T. Alam, G. Govindarajan, L. Bobadilla. LCD-RIG: Limited Communication Decentralized Robotic Information Gathering Systems. IEEE Robotics and Automation Letters, 2024

• [2] P. Padrao, A. Dominguez, L. Bobadilla, and R. N. Smith. Towards learning ocean models for long-term navigation in dynamic environments. IEEE Oceanic Engineering Society - OCEANS 2022.

• [3] P. Padrao, J. Fuentes, L. Bobadilla, and R. N. Smith. Estimating spatio-temporal fields through reinforcement learning. Frontiers in Robotics and AI, 9, 2022.

• [4] J. Fuentes, P. Padrao, A. Redwan, L. Bobadilla, Scalar Field Feature Seeking with Learning-based Adaptive Robot Navigation. ICRA 2025, IEEE International Conference on Robotics and Automation (submitted)

• [5] C. Rojas, P. Padrao, J. Fuentes, G. Reis, A. Albayrak, B. Osmanoglu, L. Bobadilla. Combining Multi-Satellite Remote and in-situ Sensing for Unmanned Underwater Vehicle State Estimation. Elsevier Journal of Ocean Engineering, 2024

• [6] Kaarlela, T.; Pitkäaho, T.; Pieskä, S.; Padrão, P.; Bobadilla, L.; Tikanmäki, M.; Haavisto, T.; Blanco Bataller, V.; Laivuori, N.; Luimula, M. Towards Metaverse: Utilizing Extended Reality and Digital Twins to Control Robotic Systems. Actuators 2023, 12, 219.

• [7] P. Padrao, J. Fuentes, T. Kaarlela, A. Bayuelo, L. Bobadilla., Towards Optimal Human-Robot Interface Design Applied to Underwater Robotics Teleoperation. 2024 IEEE CASE - International Conference on Automation Science and Engineering

Fault Detection & Diagnosis

Sensor Fault Detection and Diagnosis Approaches applied to an Autonomous Solar-powered Aircraft

Developed by Elektra Solar, a spin-off of the DLR Institute for Robotics and Mechatronics (DLR-RMC), the Elektra 2 Solar is a solar-powered autonomous aircraft designed for long-distance and high-altitude missions. Elektra 2 Solar uses a simple limit-checking approach for certain measurements, such as angular velocities and attitude angles. However, this oversimplified detection method can hide faulty system behaviors, potentially leading to missed fault alarms and posing serious risks to the aircraft's overall system.

The motivation of this work is to compare various IMU sensor fault detection and diagnosis (FDD) approaches that can be applied to the Elektra 2 aircraft.

The first proposed FDD approach is based on decoupled lateral and longitudinal linear models of the aircraft, combined with Kalman filters for residual generation. An adaptive threshold technique (ATLMS) is used for fault detection. 

The second approach utilizes the aircraft's kinematic model in conjunction with an extended Kalman filter for residual generation, with a decision table based on alarm flag activation sequences for fault diagnosis. 

The third FDD approach is model-free, relying on principal component analysis (PCA), where contributions to squared prediction errors are used for fault diagnosis. Finally, a fourth FDD strategy is developed, leveraging the key advantages of the previous approaches.

Simulations were performed across various flight scenarios, applying additive faults to measurements of roll and pitch rates as well as longitudinal acceleration. Real flight data from nominal operations were used for validation purposes.

Related publications:

• Padrao, P., Hsu, L., Vilzmann, M. and Kondak, K., 2019, December. A Comparative Study of Sensor Fault Detection Approaches applied to an Autonomous Solar-powered Aircraft. In 2019 19th International Conference on Advanced Robotics (ICAR) (pp. 761-766). IEEE. 

• Padrao, P., Hsu, L., Vilzmann, M. and Kondak, K., 2019, October. Model-based sensor fault detection in an autonomous solar-powered aircraft. In FT2019. Proceedings of the 10th Aerospace Technology Congress, October 8-9, 2019, Stockholm, Sweden (No. 162, pp. 247-254). Linköping University Electronic Press. 

Optimal Threshold Synthesis for State-of-Health Classification and Evaluation of a Tribological System

The primary objective of this project was to develop a method to enhance state-of-health classification performance based on experimental data from a tribological system. The study was conducted at the Chair of Dynamics and Control, University of Duisburg-Essen. We introduced two performance metrics to evaluate system behavior: the Switching State Rate and Regeneration Time. Based on the analysis of experimental data, four distinct operational states were defined: 

• State 1 (stable and error-free operation)

• State 2 (stable with minor surface condition changes) 

• State 3 (acceptable surface condition changes)

• and State 4 (significant surface condition changes).

To facilitate optimization across different datasets, we employed an in-house Optimization Graphical Interface, which streamlined problem setup and enabled numerical optimization through MATLAB algorithms. The optimal threshold values were determined by minimizing a global objective function that evaluated all datasets simultaneously.

Related publications:

• Rothe, S., Padrao, P., Leite, A., and Söffker, D., 2015. Improvement and comparison of wear-oriented state-of-health classification methods using optimization techniques. Structural Health Monitoring 2015. 

• Padrao, P., Rothe, S., Leite, A. and Soeffker, D. (2015). Optimal Threshold Synthesis for State-of-Health Classification and Evaluation of a Tribological System. In: 17th International Symposium on Dynamic Problems of Mechanics. Brazilian Society of Mechanical Sciences and Engineering.

STEM Education

Digital Twins Utilizing XR-Technology as Robotic Training Tools

Digital technology has evolved towards a new way of processing information: web searches, social platforms, internet forums, and video games have substituted reading books and writing essays. Trainers and educators currently face the challenge of providing natural training and learning environments for digital-natives. In addition to physical spaces, effective training and education require virtual spaces. Digital twins enable trainees to interact with real hardware in virtual training environments. Interactive real-world elements are essential in the training of robot operators. A natural environment for the trainee supports an interesting learning experience while including enough professional substances. We investigate examples of how virtual environments utilizing digital twins and extended reality can be applied to enable natural and effective robotic training scenarios. 

Related publications:

• Kaarlela, T.; Padrao, P.; Pitkäaho, T.; Pieskä, S.; Bobadilla, L., 2023. Digital Twins Utilizing XR-Technology as Robotic Training Tools. Machines 2023, 11, 13.

MSP Talks photos & videos