EdgeRunner and Army Research Laboratory Collaborate to Bring High-Level Intelligence to the Edge for Heterogeneous Autonomous Military Platforms

Autonomous systems and robotics represent the next frontier in warfighting. They are taking on increasingly critical roles, supporting everything from offensive and defensive military actions to vital rescue and recovery operations, reducing the likelihood of human casualties.

Today, most robots rely on an active network connection, where humans make decisions and route them back to the machine. However, operating in Denied, Degraded, Intermittent, or Limited (DDIL) environments means robots must be able to make nonlethal autonomous decisions locally—without needing a network connection.

Over the past six months, EdgeRunner AI has collaborated with the U.S. Army Combat Capabilities Development Command (DEVCOM) Army Research Laboratory (ARL) under a CRADA (Cooperative Research and Development Agreement) to build completely on-device autonomous capabilities. Together, we pursued two main goals:

• Bring LLMs to the Edge: Replace the robot’s cloud-hosted Large Language Model (LLM) with an on-device, edge-hosted LLM while maintaining all existing capabilities.
• Advance Robotic Reasoning: Implement advanced reasoning and instruction-following that go beyond the robot’s current baseline.

Why Local Edge Models Matter

Shifting from cloud-hosted to local edge models provides several critical operational advantages:

• Reliability: Continuous operation in harsh, network-denied (DDIL) environments.
• Security: Enhanced operational and data security by reducing the robot's electromagnetic signature.
• Latency: Instantaneous decision-making without waiting for cloud data transmission.
• Cost Efficiency: Elimination of per-token cloud LLM API charges.

Field-Testing at Aberdeen Proving Ground

To test this technology in the field, the EdgeRunner AI team recently visited ARL’s Robotics Research Collaboration Campus (R2C2) near Aberdeen Proving Ground.

Goal 1: Seamless Edge LLM Integration

We replaced the cloud-hosted LLM endpoints with our latest EdgeRunner Light and Medium models, matching them to the specific compute and memory profiles of the physical hardware:

• EdgeRunner Medium: Deployed on ARL’s Clearpath Robotics Warthog UGV.
• EdgeRunner Light: Deployed on ARL’s Ghost Robotics Vision 60 (V60) quadruped "dog".

To make integration as frictionless as possible, we packaged our robot-ready models and inference engines into Docker containers. Transitioning the robots from cloud reliance to local EdgeRunner models required changing just a single detail in the robot's configuration: the model's IP address.

Goal 2: Advanced Reasoning & Simulator Alignment

Prior to our 3-day trip, we used ARL's behavior simulator to rapidly iterate and code new approaches to key capabilities, such as instructing a robot to navigate multiple waypoints with Visual Language Model (VLM)-based object search to find items of interest along the route.

Putting Reasoning into Action (Upcoming Q3 Objective):

Imagine a hypothetical scenario where a robot is dispatched to a person's last known location. If the target is missing, the robot autonomously searches the nearby area, locates their vehicle, and inspects it. Finding the vehicle empty, it alerts the operator and keeps searching until it finds the person. If the person is missing an object they were supposed to have (like a briefcase), the robot can coordinate with other robots so one tracks the person while others hunt for the asset. These capabilities require complex search, teaming, and reasoning skills.

Key Takeaways and What's Next

The field visit was a resounding success. We validated local LLM inference and new reasoning capabilities on both the physical Warthog and V60 platforms.

The trip also provided an invaluable learning experience regarding how simulation translates to reality. On Day 1, we encountered a feature that worked flawlessly in the simulator but failed on the physical hardware. After collaborating with the ARL team, we reworked the feature that evening and successfully verified its operation on the physical platform the next day.

It was enthralling to see these machines interact with their environment and reason in real time—watching the massive Warthog pause at a waypoint, analyze a photo with the VLM against its objectives, and then kick up dust as it zoomed off to its next destination.

According to Phil Osteen, Robotics Researcher, Autonomous Systems Branch, Science of Intelligent Systems Division, SISD, DEVCOM Army Research Laboratory: “The EdgeRunner team was able to deploy new algorithms and models on our autonomous robotic platforms on Day 1 of their first on-site visit, which is rare.”

Looking Ahead:

On our final day, we mapped out our next immediate milestones. Reinforcement Learning (RL) is one of the key focus areas of our partnership with ARL. RL requires a simulation environment that is sufficiently representative of the real world, which we were able to validate during this site visit. With a vetted end-to-end simulation, we will now conduct model training to create tailored models with maximum accuracy for these particular robots and hardware types. Moreover, we can now progress to substantially more advanced mission sets and increasingly high-judgment scenarios.

Robotics represents the next frontier for mission-critical AI at the edge, and we are thrilled to partner with ARL’s expert team to push these boundaries forward.

Brian King is Distinguished Scientist at EdgeRunner AI. He was previously a Senior Applied Scientist at Amazon where he worked on speech recognition for Alexa and multi-modal LLM’s. He holds 12 patents in applications in machine learning, speech recognition, and digital signal processing.