DOE ARM Facility Advances AI to Reshape Data Infrastructure

en.Wedoany.com Reported - The U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) user facility is advancing artificial intelligence technology within its data infrastructure, aiming to help researchers access over 30 years of atmospheric observation data more quickly. The facility has collected more than 8 PB of atmospheric data.

Giri Prakash, ARM's Chief Data and Computing Officer, stated that AI-ready infrastructure has become a necessity, and the center is adopting a phased approach to enhance infrastructure to meet growing computational demands. These developments will be added to ARM's existing infrastructure to satisfy the demanding requirements of AI applications. Prakash manages the ARM Data Center located at Oak Ridge National Laboratory (ORNL) in Tennessee.

Approximately four years ago, ARM began installing Graphics Processing Units (GPUs) for the Cumulus high-performance computing cluster. These GPUs have been used for projects such as data quality analysis, radar processing, and data product generation. The center is currently undertaking more significant upgrades, replacing file servers with an AI-ready storage platform directly connected to the GPU environment, enabling AI models to access data at high speed without waiting for file transfers. ARM plans to procure 25 to 30 new GPUs, including processing units specifically designed to accelerate AI workloads, to meet computing needs over the next two to five years. ARM's cybersecurity and network engineering teams are also strengthening controls to manage access to computing, data, and AI resources and tools.

On the software side, ARM is developing an environment that enables Large Language Models (LLMs) and agent-based systems to communicate with data assets, metadata, and quality records. Giri Prakash explained that LLMs are the "brain for understanding and interpretation," while agents are systems that use that brain to access data and complete tasks. Together, they connect the general reasoning capabilities of LLMs with institutional knowledge, tools, and actions. Unlike traditional AI assistants that only answer questions, agent-based systems can reason through multi-step tasks, access external tools, and coordinate workflows with minimal human intervention.

ARM has launched the beta version of ARM Data Advisor (ADA, pronounced "ā-duh"). According to Wade Darnell, an ORNL software developer and lead developer of ADA, this AI agent answers questions, recommends datasets, displays data plots, explains data quality, and places data orders through a natural language conversational interface. Basic data ordering will be available in the initial version, with advanced ordering and data extraction added in future versions. ADA will provide personalized recommendations for returning users and offer files in multiple formats. ADA is expected to launch in July 2026, and traditional search tools will be retained until developers are confident that it meets user needs.

ARM developers have also built a framework called the Agentic Tooling and LLM Augmentation Stack (ATLAS). ATLAS provides a shared platform enabling AI-driven tools to work together, including providing model inference through OpenAI-compatible endpoints, converting information into a searchable vectorized format, orchestrating workflows guided by domain-specific agents, and offering secure access to data and services. The framework supports metadata generation, data quality analysis, and enhanced website search through the digital assistant Ask ARM. ATLAS also connects to multiple model-serving environments requiring GPUs, including internal platforms and the U.S. Science Cloud, part of the DOE's Genesis mission, which consolidates national laboratory supercomputing resources into a secure cloud for AI-driven scientific discovery.

Over the next three years, ARM expects to add 2 to 3 PB of storage for vectorized content, including searchable metadata embeddings, guidance pages, and instrument manuals. Simultaneously, ARM plans to convert and chunk scientific variables and observational data into AI-readable contextual representations, enabling agent systems to efficiently index, retrieve, and analyze them. By leveraging GPU Direct Storage and a tightly integrated high-performance storage architecture, ARM reduces data movement bottlenecks and accelerates AI model response times. As the GPU cluster expands, facility operations teams are coordinating efforts in energy, cooling, and networking.

Prakash stated that user feedback will continue to guide the rollout of these tools. The ARM Data Center team is developing a governance document to set guardrails for the responsible use of AI, define ethical standards aligned with DOE principles, and establish evaluation criteria and best practices. By aligning AI-optimized data flows with the DOE's Genesis mission, the ARM Data Center team is committed to shortening the time from observation to discovery. Prakash emphasized: "ARM is not simply attaching AI to existing systems; we are rebuilding our data ecosystem around AI."