FAQ
Frequently Asked Questions
Q: TCP server port conflict — "TCP server on port 5555 failed to bind"
A: Another process is using port 5555. Either kill the conflicting process or change the port in your configuration.
# Find the process using port 5555
lsof -i :5555
# Kill the process (replace <PID> with actual process ID)
kill -9 <PID>
# Or change the port in wrp_conf.yaml
network:
zmq_port: 5556Q: Shared memory not available
A: IPC requires shared memory. Set shm_size: 8g in Docker or configure /dev/shm on bare metal.
# Docker: Add to docker-compose.yml
services:
iowarp-runtime:
shm_size: 8g
# Bare metal: Mount /dev/shm with sufficient size
sudo mount -o remount,size=8G /dev/shm
# Verify shared memory size
df -h /dev/shmQ: How do I check if IOWarp runtime is running?
A: Use clio status or check the ZeroMQ port.
# Check runtime status
clio status
# Check if ZeroMQ port is listening
netstat -tuln | grep 5555
# or
ss -tuln | grep 5555
# Test connection (Python SDK coming soon)
# Currently available as C++ API via IOWarp CoreQ: What scientific formats are supported?
A: 50+ formats: HDF5, NetCDF, Zarr, CSV, JSON, Parquet, FITS, ROOT, PDB, and more via CLIO Ingest plugins.
Scientific
- • HDF5
- • NetCDF
- • Zarr
- • FITS
- • ROOT
Tabular
- • CSV
- • JSON
- • Parquet
- • Excel
- • Arrow
Domain-Specific
- • PDB (Bio)
- • DICOM (Medical)
- • NeXus (Neutron)
- • CDF (Space)
Q: Can I use IOWarp with GPU workloads?
A: Yes. CLIO Transfer supports GPU Direct Storage for NVMe-to-GPU transfers, and Hermes provides CUDA memory backends for GPU HBM.
# Enable CUDA support in Spack
spack install iowarp +cuda cuda_arch=70
# Configure GPU HBM storage tier
storage:
- path: "gpu::cte_gpu_tier1"
bdev_type: "cuda"
capacity_limit: "32GB"
score: 0.0
device: 0 # GPU device ID
# Use GPU memory backend in Hermes
auto alloc = mem->GetAllocator(alloc_id); Q: How do I add a custom format plugin?
A: Implement the CLIO Ingest plugin interface and register your plugin.
# Implement CLIO Ingest plugin interface
#include
class MyFormatPlugin : public CLIOIngestPlugin {
public:
std::string GetFormatName() override { return "myformat"; }
bool CanRead(const std::string& path) override;
Context Ingest(const std::string& path) override;
void Export(const Context& ctx, const std::string& path) override;
};
// Register plugin
CLIO_REGISTER_PLUGIN(MyFormatPlugin);
# Load plugin in wrp_conf.yaml
clio_ingest:
plugins:
- libmyformat_plugin.so Q: What are the minimum system requirements?
A: 8GB RAM, Docker or Linux. For HPC: Spack, shared filesystem.
Minimum Requirements
- • 8GB RAM
- • Docker or Linux
- • 10GB disk space
- • x86_64 or ARM64
HPC Requirements
- • Spack package manager
- • Shared filesystem (Lustre/GPFS)
- • High-speed interconnect (optional)
- • Job scheduler (Slurm/PBS)
Q: IOWarp is slow. How do I improve performance?
A: Check worker threads, buffer sizes, storage tier configuration, and network settings.
# Increase worker threads
export CHIMAERA_NUM_WORKERS=16
# Increase buffer sizes for large I/O
performance:
buffer_size: "4MB"
queue_depth: 128
# Ensure hot data is on fast tiers
storage:
- path: "ram::tier1"
score: 0.0 # Highest priority
# Use high-speed interconnect for multi-node
network:
interconnect: "ib0"