Core Architecture

Relevant source files

• .env.example
• README.md
• core/Cargo.toml
• core/README.md
• core/src/raw/ops.rs
• core/src/raw/rps.rs
• core/src/services/mod.rs
• core/src/types/capability.rs
• core/src/types/mod.rs
• core/src/types/operator/builder.rs
• core/src/types/operator/operator.rs
• core/src/types/operator/operator_futures.rs
• core/src/types/options.rs
• core/src/types/scheme.rs
• core/tests/behavior/async_write.rs
• core/tests/behavior/main.rs

This document provides a detailed technical overview of OpenDAL's core architecture, focusing on the internal design patterns and abstractions that enable unified data access across diverse storage services. It covers the main components including the Operator interface, layer system, service abstraction, and data flow patterns.

For information about specific storage service implementations, see Storage Services. For details about language bindings and their integration, see Language Bindings.

Architecture Overview

OpenDAL's core architecture follows a layered design that provides a unified interface over diverse storage backends through a set of well-defined abstractions.

Core Component Relationships

Sources: core/src/types/operator/operator.rs31-151 core/src/types/operator/builder.rs25-71 core/src/types/capability.rs20-66

Operator Interface Architecture

The Operator serves as the primary entry point for all OpenDAL operations, providing both synchronous and asynchronous APIs with a builder pattern for configuration.

Operator Core Structure

The Operator struct wraps an Accessor and provides three API patterns:

1. Direct Operations: Simple methods like read(), write(), stat()
2. Builder Pattern: Methods like read_with() that return futures with chainable options
3. Advanced I/O: Methods like reader() and writer() for streaming operations

Sources: core/src/types/operator/operator.rs147-151 core/src/types/operator/operator_futures.rs34-51

Operation Options and Futures

The future-based API uses OperatorFuture<I, O, F> where:

• I represents input options (e.g., ReadOptions, WriteOptions)
• O represents output type (e.g., Buffer, Metadata)
• F represents the underlying future type

Sources: core/src/types/operator/operator_futures.rs42-51 core/src/types/options.rs27-495

Access Trait and Service Implementation

The Access trait provides the core abstraction that all storage services must implement, enabling uniform access across different backends.

Access Trait Implementation Pattern

Sources: core/src/raw/access.rs core/src/services/mod.rs18-207

Service Builder Pattern

All storage services follow a consistent Builder → Backend → Core pattern:

Sources: core/src/types/operator/builder.rs68-71 core/src/services/mod.rs18-21

Layer System Architecture

OpenDAL's layer system provides cross-cutting concerns like retry logic, logging, and metrics through a decorator pattern applied to the Access trait.

Layer Application Flow

Each layer wraps the underlying Access implementation and can:

• Modify request parameters before delegation
• Transform responses after delegation
• Add cross-cutting behavior (logging, metrics, retry)
• Short-circuit requests for caching or validation

Sources: core/README.md101-125 core/src/layers/

Layer Implementation Pattern

Sources: core/src/layers/ core/src/types/operator/builder.rs371-396

Operation Flow and Data Types

OpenDAL operations follow a consistent request-response pattern using operation arguments (OpXxx) and responses (RpXxx).

Operation Argument and Response Types

Sources: core/src/raw/ops.rs18-29 core/src/raw/rps.rs23-212

I/O Operation Data Flow

The operation flow follows these steps:

1. User calls Operator method with path and optional parameters
2. Operator creates appropriate OpXxx argument struct
3. Access.operation() method is called with path and arguments
4. Service implementation returns RpXxx response with associated I/O objects
5. Response is converted to user-facing types (Buffer, Metadata, etc.)

Sources: core/src/types/operator/operator.rs467-555 core/src/raw/ops.rs309-327

Capability-Driven Feature Detection

The Capability struct allows runtime feature detection, enabling applications to adapt their behavior based on what the underlying storage service supports.

Sources: core/src/types/capability.rs65-220 core/src/types/operator/operator.rs184-186