Technology to enable AI systems rapidly adapt to novelty in their environment - on the fly

Adapting to novelty is a crucial skill for intelligent agents, allowing them to modify their behaviour in response to new challenges. However, this remains a significant hurdle for deep reinforcement learning (DRL). To address this, Researchers at The Australian National University (ANU) have developed a novel learning approach named NAPPING (Novelty Adaptation Principles Learning). NAPPING empowers intelligent machines utilizing DRL to swiftly adjust to diverse new open-world scenarios. 

The NAPPING algorithm adeptly manages "unknown unknowns" in an efficient and flexible manner, eliminating the need for an explicit understanding of the environment's structure. With NAPPING, DRL agents can efficiently generalize to analogous novel situations without unnecessary policy adjustments.

TT2023-008 Potential benefits

• Sample efficiency: Requires lesser training data compared to deep learning methods
• Managing Novelty: NAPPING adeptly manages"unknown unknowns," referring to scenarios where the target task is not predetermined, enhancing its flexibility in contrast to transfer learning, which is limited to addressing only "known unknowns
• Environment Understanding: It does not require an explicit understanding of the environment's structure, such as a model or handcrafted predicates, and is not based on planning or formal languages like PDDL.

Potential applications

• Automotive and Aerospace Applications: Ideal for autopilot and collision negation systems
• Remote Exploration with Robots: Tailored for unfamiliar environments
• Industrial Robotics Advancements: Versatility for various robotics applications, especially in industrial Production.

Opportunity

ANU is seeking engagement with industry partners looking to develop the capabilities of their AI systems.

IP status

The IP is owned by the ANU and is a subject of a patent application.

Key research team

• Cheng Xue, Research Fellow, ANU College of Engineering, Computing and Cybernetics
• Peng Zhang, Research Fellow, ANU College of Engineering, Computing and Cybernetics
• Jochen Renz, Associate Professor, ANU College of Engineering, Computing and Cybernetics

Data

Rapid performance recovery of NAPPING to novelty introduced on baseline agents compared to standard DRL approaches.