Enhancing Safety and Efficiency in Autonomous Vehicles: Insights from Fei Miao’s MARL Research
Fei Miao, a professor at the University of Connecticut, discussed “Learning and Control for Safety, Efficiency, and Resiliency of Embodied AI” in a seminar on November 8. She highlighted her team’s work on Multi-agent Reinforcement Learning (MARL) for Connected and Automated Vehicles (CAVs). This research focuses on how multiple autonomous vehicles communicate in real-time to improve driving decisions.
Embodied AI learns from its environment and interacts with other agents. Multi-agent systems, like those for CAVs, are more complex than single-agent systems. Miao emphasized the need for robots to collaborate on tasks in various settings, including manufacturing and transportation.
A key challenge for deploying multi-agent systems is ensuring safety. Failures in a simulation are manageable, but failures in real-world scenarios can endanger lives. Efficiency is another challenge, as the system’s performance relies on the combined actions of all agents. Miao’s team aims to create policies that help vehicles act in their own best interests while enhancing overall system efficiency through communication.
Miao’s team focuses on uncertainty quantification (UQ) in computer vision for multi-agent systems. Current methods often overlook uncertainties in computer vision models. To address this, her team developed a novel UQ approach called Double-M quantification. This combines learning-based models with statistics to improve object detection accuracy.
Miao explained that traditional models often only predict the main position of objects. Their method adds predictions for the covariance matrix. It uses KL-divergence loss to control the volume of the covariance matrix.
Miao also introduced a bootstrap calibration method to improve model predictions. This involves retraining on various data subsets to better calibrate prediction errors. This calibration enhances accuracy, as shown in tests with open-source datasets.
How does uncertainty quantification improve object detection in autonomous driving systems?
Interview with Prof. Fei Miao: Pioneering Multi-Agent Reinforcement Learning in Autonomous Vehicles
November 9, 2023, by News Directory 3 Staff
In our latest feature, we delve into the innovative research of Prof. Fei Miao at the University of Connecticut. On November 8, she conducted a seminar titled “Learning and Control for Safety, Efficiency, and Resiliency of Embodied AI,” highlighting her team’s groundbreaking work in Multi-agent Reinforcement Learning (MARL) specifically catered to Connected and Automated Vehicles (CAVs). We had the opportunity to sit down with her to discuss her research, its implications, and the challenges that lie ahead.
News Directory 3: Thank you for joining us, Prof. Miao. Your seminar shed light on the interconnectivity of autonomous vehicles through MARL. Can you briefly explain what MARL is and its significance in CAVs?
Prof. Fei Miao: Thank you for having me! Multi-agent Reinforcement Learning is a framework where multiple agents learn to make decisions through interactions with each other and their environment. In the context of Connected and Automated Vehicles, MARL facilitates real-time communication among vehicles, allowing them to coordinate actions, reduce accidents, and enhance traffic efficiency. The complexity arises because each vehicle, as an agent, must anticipate the actions of others, making it essential for them to collaborate effectively.
News Directory 3: You mentioned the importance of safety and efficiency in deploying multi-agent systems for CAVs. What specific challenges do you foresee?
Prof. Fei Miao: Safety is paramount. While we can manage failures in simulations, a real-world failure can have dire consequences. Ensuring that vehicles make safe decisions on the road is challenging. Efficiency is another major hurdle; the performance of the entire system heavily depends on how well each vehicle acts both independently and within the collective framework. Our goal is to establish policies that allow vehicles to pursue their individual objectives while also maximizing overall system efficiency through effective communication.
News Directory 3: Let’s talk about uncertainty quantification (UQ) in computer vision which you highlighted in your seminar. Why is this a critical focus for your team?
Prof. Fei Miao: Many current computer vision models inadequately address uncertainties, which can lead to catastrophic misunderstandings in dynamic environments like roadways. By quantifying uncertainty, we can improve object detection accuracy significantly. Our novel approach, Double-M quantification, integrates learning-based models with traditional statistical methods to provide better insights into the positioning and movement of objects. This includes generating predictions for the covariance matrix, which helps us gauge the reliability of our detections effectively.
News Directory 3: That sounds fascinating! Can you elaborate on how your method differs from traditional object detection models?
Prof. Fei Miao: Traditional models usually focus solely on predicting the most probable location of objects but often fail to account for potential variations around that prediction. Our method complements this by incorporating predictions of uncertainty, which helps in understanding the possible range and confidence of these detections. By utilizing KL-divergence loss for controlling the volume of uncertainty, we enhance the robustness of our predictions under different scenarios, which is vital for the safety of CAVs.
News Directory 3: Looking ahead, what are the next steps for your research team?
Prof. Fei Miao: Our immediate focus will be to refine our UQ methods and integrate them into the MARL framework. We aim to conduct more extensive field tests to assess our models in real-world settings, addressing the complexities and dynamics of urban environments. Sharing insights with industry partners will also be crucial as we advance towards practical implementation in CAV technology.
News Directory 3: Thank you, Prof. Miao, for your enlightening insights. Your work is vital in shaping the future of autonomous vehicles and ensuring safer, more efficient roads.
Prof. Fei Miao: Thank you for the opportunity to discuss our work! It’s an exciting field, and I look forward to seeing how we can bridge theory and practice for the betterment of transport safety and efficiency.
Stay tuned for more updates as we continue to follow the advancements in embodied AI and CAV research from Prof. Miao and her team.
Additionally, Miao discussed the development of a framework for predicting 3D semantic occupancy from 2D images. This method enhances the prediction of scene details and semantics with limited data. They also introduced a hierarchical conformal prediction method to tackle class imbalances in occupancy datasets, which is crucial for recognizing pedestrians.
For trajectory prediction, Miao’s team created the Conformal Uncertainty Quantification under Distribution Shift (CUQDS) framework. This method uses Gaussian regression combined with congruent predictions for more reliable trajectory forecasts. Their results outperformed other models in tests.
In her discussion of MARL, Miao explained how her team integrates game theory principles with reinforcement learning. This integration helps individual agents make decisions that align with the overall system goals. Miao emphasized the importance of decentralized policies for agent safety.
To enhance safety, Miao proposed the Safe-RMM framework. This two-level system learns cooperative behaviors and generates planning actions for vehicles. It uses model predictive control with Control Barrier Functions to ensure safe actions.
In conclusion, Miao’s research in data-driven optimization and cyber-physical system security contributes to bridging theoretical models and practical applications, especially in self-driving technology. Tianmin Shu, an assistant professor at Hopkins, noted that while Miao’s research may not seem flashy, it is crucial for the successful deployment of multi-agent systems like self-driving cars.