Documentation System

Relevant source files

Purpose and Scope

The Documentation System manages the generation, building, and deployment of comprehensive documentation for the OpenDAL project. This system orchestrates documentation from multiple language bindings, integration projects, and the core Rust library into a unified website using Docusaurus as the static site generator.

This page covers the technical infrastructure for documentation generation and deployment. For information about the CI/CD pipeline that triggers documentation builds, see CI/CD Pipeline. For details about the actual documentation content structure, see Development Guide.

Architecture Overview

The documentation system follows a multi-stage build pipeline that collects documentation from various sources and combines them into a single deployable website.

Sources: .github/workflows/docs.yml46-720 website/docusaurus.config.js88-152

Documentation Build Pipeline

The documentation build process is orchestrated through GitHub Actions with multiple parallel jobs for different documentation sources.

Sources: .github/workflows/docs.yml47-516

Build Job Configuration

Each documentation build job follows a consistent pattern:

Rust Documentation: Uses nightly toolchain with RUSTDOCFLAGS="--cfg docsrs" for feature gates and documentation-specific features
Java Documentation: Uses Maven javadoc:javadoc goal with Temurin JDK 17
Node.js Documentation: Uses pnpm with TypeDoc for API documentation generation
Python Documentation: Uses MkDocs with uv for dependency management
C Documentation: Uses Doxygen for header file documentation
Integration Projects: Use cargo doc with nightly toolchain for Rust-based integrations

Sources: .github/workflows/docs.yml69-74 .github/workflows/docs.yml92-94 .github/workflows/docs.yml129-131

Website Configuration

The website is built using Docusaurus 2 with extensive customization for multi-language documentation.

Sources: website/docusaurus.config.js40-286

Key Configuration Features

The Docusaurus configuration includes several environment-specific customizations:

Base URL Configuration: Supports different base URLs for staging, nightlies, and production deployments
Version Management: Automatically extracts version information from Git tags using semver parsing
Multi-Document Support: Separate document collections for main documentation and community content
Custom Redirects: Maintains backward compatibility with legacy URLs

Sources: website/docusaurus.config.js30-38 website/docusaurus.config.js49-68

Custom Image SSR Plugin

The documentation system includes a custom plugin for processing external images to ensure reliable offline access and faster loading.

The plugin processes both HTML files and JavaScript bundles to ensure all external image references are localized:

HTML Processing: Uses cheerio to parse and modify image src attributes
JavaScript Processing: Uses regex patterns to find and replace image URLs in bundled code
Caching: Maintains a map of processed images to avoid duplicate downloads
Error Handling: Falls back to original URLs if download fails

Sources: website/plugins/image-ssr-plugin.js27-233

Deployment Strategies

The documentation system supports multiple deployment targets based on the trigger event:

Deployment Configuration Details

Production Deployment: Uses peaceiris/actions-gh-pages to deploy to the gh-pages branch with force orphan commits
Staging Deployment: Creates separate branches for RC versions using the pattern site/*-staging
Nightlies Deployment: Uses rsync to deploy to Apache Infrastructure nightlies server with different base URLs
Environment Variables: Supports OPENDAL_WEBSITE_BASE_URL, OPENDAL_WEBSITE_NOT_LATEST, and OPENDAL_WEBSITE_STAGING for customization

Sources: .github/workflows/docs.yml637-719 website/docusaurus.config.js275-283

Documentation Artifact Management

The build system manages documentation artifacts through GitHub Actions artifacts:

Artifact Name	Source	Build Tool	Output Path
`rust-docs`	`core/`	`cargo doc`	`./core/target/doc`
`java-docs`	`bindings/java`	`mvn javadoc`	`./bindings/java/target/site/apidocs`
`nodejs-docs`	`bindings/nodejs`	`pnpm docs`	`./bindings/nodejs/docs`
`python-docs`	`bindings/python`	`mkdocs build`	`./bindings/python/site`
`ruby-docs`	`bindings/ruby`	`rake doc`	`./bindings/ruby/doc`
`C-docs`	`bindings/c`	`make doc`	`./bindings/c/docs/doxygen/html`
`lua-docs`	`bindings/lua`	`ldoc`	`./bindings/lua/doc/`
`haskell-docs`	`bindings/haskell`	`cabal haddock`	`./bindings/haskell/doc/`
`cpp-docs`	`bindings/cpp`	`cmake + ninja`	`./bindings/cpp/build/docs_doxygen/html`
`ocaml-docs`	`bindings/ocaml`	`dune build @doc`	`./bindings/ocaml/_build/default/_doc/_html`

Each artifact is downloaded to a specific path under ./website/static/docs/ during the website build process, making them accessible through the final website's URL structure.

Sources: .github/workflows/docs.yml536-594

Apache Software Foundation Integration

The documentation system integrates with Apache Software Foundation infrastructure through the .asf.yaml configuration:

GitHub Pages Publishing: Configured to publish from the gh-pages branch
Staging Support: Automatic staging deployment using the autostage: site/* pattern
Notification Integration: Documentation changes trigger notifications to Apache mailing lists

Sources: .asf.yaml55-61 .asf.yaml49-53