CONCEPT T4 / HPC I/O AND STORAGE HIERARCHIES

HPC I/O under scientific AI.

HPC I/O research studies how applications read and write data on supercomputers: through memory, burst buffers, node-local flash, parallel file systems, and archives. The deep memory and storage hierarchy is fast only if data is in the right tier at the right time.

Where IOWarp comes from

IOWarp builds on Hermes, an earlier system for multi-tiered distributed I/O buffering, and on a line of Gnosis Research Center systems for labeled I/O, time-ordered logging, and data-task scheduling. The founding aim was stated directly: bending the I/O fabric for AI-infused scientific workflows.

Measured results in the line of work

MegaMmap (SC 2024) manages DRAM and storage as tiered distributed shared memory and lowered peak DRAM use while keeping performance competitive. WisIO (ICS 2025) automates I/O bottleneck detection in HPC workflows. The papers state the workloads and figures.

Why it matters for agents

Agents add a new kind of client to the storage stack: many small, latency-sensitive reads guided by reasoning. The CLIO transfer engine carries the tiering ideas into that setting.

How IOWarp approaches it

Where to go deeper.

01Storage and context

How CLIO Core’s transfer engine places data across tiers.

Continue →02Where results live

Papers, release notes, and project sites that carry the measurements with their conditions.

Continue →03Research archive

Publications behind the systems lineage.

Continue →04Compose storage tiers

Configure local tiers and inspect how the Core filesystem uses its storage chain.

Open ↗

Applications that use it

A4
Agentic workflows across HPC and cloud
A DOE Genesis Mission project at PNNL on performance-aware agentic workflows for the Genesis Mission Platform. The IOWarp team works on workflow motifs and performance models.
Project began August 2026
A5
CLIO on national research systems
Where CLIO Core has been built and run: ALCF Aurora, DeltaAI, and the Ares cluster.
Built and run on national systems
A6
GPU direct I/O and compression
CUDA kernels that read and write IOWarp from the GPU, GPU payloads offloaded to host memory and NVMe, and compression on the device. A research thread built on NVIDIA technologies.
Research thread across the stack

Common questions

Common questions.

Does IOWarp replace the parallel file system?

No. It sits alongside existing storage and manages data flow across tiers during a workflow.

Which I/O interfaces does CLIO Core intercept?

In source builds, the transfer engine has adapters for POSIX, STDIO, HDF5 (VOL and VFD), and FUSE, an optional MPI-IO adapter, and GPU paths.

References

Next step

See this topic
at work.

Agentic workflows across HPC and cloud All concepts

Where IOWarp comes from

Measured results in the line of work

Why it matters for agents

Where to go deeper.

Agentic workflows across HPC and cloud

CLIO on national research systems

GPU direct I/O and compression

Common questions.

Research and demos on this topic.

References

See this topicat work.

See this topic
at work.