Research Projects

Security Assurance in Multi-Agent Reinforcement Learning

Deep reinforcement learning (DRL) policies are known to be vulnerable to adversarial perturbations to their observations, similar to adversarial examples for deep learning classifiers. However, an attacker is not usually able to directly modify another agent’s observations. This might lead one to wonder: is it possible to attack an RL agent simply by choosing an adversarial policy acting in a multi-agent environment so as to create natural observations that are adversarial? In this research, we investigate the potential attack strategies that an adversary can use to disturb autonomous systems that are built for multi-agent environments.

Autonomous Coordination and Communication in Platoons

The design of control algorithms for platoons of vehicles is challenging, particularly considering that coordination between vehicles is obtained through diverse communication channels. Many modern vehicles are already equipped with Adaptive Cruise Control (ACC) to regulate certain driving functions. ACC can be extended to leverage inter-vehicle communications, creating a tightly-coupled vehicle stream in the form of a platoon. This extension is called Cooperative Adaptive Cruise Control (CACC) and typically assumes full communication among the distinct vehicles. In this research, we develop different deep reinforcement learning algorithms to coordinate autonomous operation of a platoon under different communication levels. The ultimate goal of this research is to build more robust and reliable CACC controllers even under communication impairments that could happen due to jamming attacks.

Autonomous Navigation in Unknown Environments

The ability to perform autonomous exploration is essential for unmanned aerial vehicles (UAV) operating in  unknown environments where it is difficult to describe the environment beforehand. Algorithms for autonomous exploration often focus on optimizing time and full coverage in a greedy fashion. These algorithms can collect irrelevant data and wastes time navigating areas with no important information. In this research project, we aim to improve the efficiency of exploration by maximizing the probability of detecting valuable information.  We explore different optimization theories to resolve this hard problem including robustness theory of Probabilistic Metric Temporal Logic (P-MTL), ergodicity theory, and deep reinforcement learning. We target in this project several environmental and conservation navigation problems such as detecting areas occupied by illegal Artisanal Small-scale Gold Mining (ASGM) activities in Amazonian rainforest. Our preliminary results from the robustness of P-MTL show that our approach outperforms a greedy exploration approach from the literature by 38% in terms of ASGM coverage.

Information Driven, Adaptive Distributed Planning

The overall goal of this project was to develop new technologies to dynamically control and coordinate multiple Unmanned Aerial Vehicles (UAVs) so they can accomplish their missions while the enemy is attempting to deny them access and prevent them from communicating.  This report details our approach to addressing this complex problem by augmenting dynamic, distributed constraint reasoning with machine learning techniques and adaptive response strategies.  By combining these technologies, we built a system that can 1) develop robust, adaptable mission plans, 2) exploit knowledge learned through prior interactions with an adversary, and 3) autonomously and dynamically alter its behavior during mission execution to improve the likelihood of a successful outcome.  This system has been thoroughly tested in the ATE2 and ATE3 simulators that were provided by AFRL/RI against four increasingly difficult milestones.

A forensics framework for service clouds

To mitigate the issues of privacy and security in service clouds and to ensure that service clouds meet users’ needs, I developed in my dissertation research a forensic investigation framework to a) consolidate a list of local and distributed proactive forensic artifacts across the cloud tenants, b) quickly capture malicious events and identify their timeframes, attack categories, targets, and responsible parties as close to near real-time analysis as possible and c) consider the awareness right of tenants about the security of their data by alerting them about their data-related incidents in form of forensic investigative reports. I studied three primary threats using the proposed framework on a service cloud: Conflict of Interest (COI) threat, Distributed Denial of Service (DDoS) attack, and the Service Impersonation attack. I developed a set of distributed algorithms to mitigate and detect each threat using the logged data. The developed algorithms were evaluated by conducting live attacks on a wide case study deployed on the MS Azure cloud system. The outcome of the research was a service cloud forensic framework (SCFF), a methodical collection of evidence for threat analysis, appropriate detection algorithms for the above threats, a distributed architecture of databases across the service cloud, and a functional prototype of the proposed framework. The contributions are immediately useful to the adoption of service cloud technology, as a result of reasoned knowledge about forensic investigations and practical techniques to address them. According to a 2018 Cloud Security Report from Cybersecurity Insiders shows that 91% of respondents are concerned about cloud security, an increase of 11% over last year’s report. Hence, my dissertation findings can still be utilized and extended to enhance cloud security.