# Java Binding

Relevant source files

The following files were used as context for generating this wiki page: - [bindings/java/src/async_operator.rs](bindings/java/src/async_operator.rs) - [bindings/java/src/convert.rs](bindings/java/src/convert.rs) - [bindings/java/src/lib.rs](bindings/java/src/lib.rs) - [bindings/java/src/main/java/org/apache/opendal/AsyncOperator.java](bindings/java/src/main/java/org/apache/opendal/AsyncOperator.java) - [bindings/java/src/main/java/org/apache/opendal/Capability.java](bindings/java/src/main/java/org/apache/opendal/Capability.java) - [bindings/java/src/main/java/org/apache/opendal/ListOptions.java](bindings/java/src/main/java/org/apache/opendal/ListOptions.java) - [bindings/java/src/main/java/org/apache/opendal/Metadata.java](bindings/java/src/main/java/org/apache/opendal/Metadata.java) - [bindings/java/src/main/java/org/apache/opendal/NativeLibrary.java](bindings/java/src/main/java/org/apache/opendal/NativeLibrary.java) - [bindings/java/src/main/java/org/apache/opendal/OpenDAL.java](bindings/java/src/main/java/org/apache/opendal/OpenDAL.java) - [bindings/java/src/main/java/org/apache/opendal/Operator.java](bindings/java/src/main/java/org/apache/opendal/Operator.java) - [bindings/java/src/main/java/org/apache/opendal/StatOptions.java](bindings/java/src/main/java/org/apache/opendal/StatOptions.java) - [bindings/java/src/main/java/org/apache/opendal/WriteOptions.java](bindings/java/src/main/java/org/apache/opendal/WriteOptions.java) - [bindings/java/src/operator.rs](bindings/java/src/operator.rs) - [bindings/java/src/test/java/org/apache/opendal/test/UtilityTest.java](bindings/java/src/test/java/org/apache/opendal/test/UtilityTest.java) - [bindings/java/src/test/java/org/apache/opendal/test/behavior/AsyncListTest.java](bindings/java/src/test/java/org/apache/opendal/test/behavior/AsyncListTest.java) - [bindings/java/src/test/java/org/apache/opendal/test/behavior/AsyncStatOptionsTest.java](bindings/java/src/test/java/org/apache/opendal/test/behavior/AsyncStatOptionsTest.java) - [bindings/java/src/test/java/org/apache/opendal/test/behavior/AsyncWriteOptionsTest.java](bindings/java/src/test/java/org/apache/opendal/test/behavior/AsyncWriteOptionsTest.java) - [bindings/java/src/test/java/org/apache/opendal/test/behavior/BlockingListTest.java](bindings/java/src/test/java/org/apache/opendal/test/behavior/BlockingListTest.java) - [bindings/java/src/test/java/org/apache/opendal/test/behavior/BlockingStatOptionsTest.java](bindings/java/src/test/java/org/apache/opendal/test/behavior/BlockingStatOptionsTest.java) - [bindings/java/src/test/java/org/apache/opendal/test/behavior/BlockingWriteOptionTest.java](bindings/java/src/test/java/org/apache/opendal/test/behavior/BlockingWriteOptionTest.java)

The Java binding provides access to OpenDAL from Java applications through JNI (Java Native Interface), offering both synchronous and asynchronous operators with proper native object lifecycle management. ## Architecture Overview The Java binding bridges Java applications to OpenDAL's Rust core through JNI functions that manage async operations, native object lifecycles, and executor management. ### JNI Function Mapping ```mermaid flowchart TD subgraph "Java Classes" Operator["org.apache.opendal.Operator"] AsyncOperator["org.apache.opendal.AsyncOperator"] NativeObject["org.apache.opendal.NativeObject"] AsyncRegistry["AsyncOperator.AsyncRegistry"] end subgraph "JNI Functions" OperatorRead["Java_org_apache_opendal_Operator_read"] OperatorWrite["Java_org_apache_opendal_Operator_write"] OperatorStat["Java_org_apache_opendal_Operator_stat"] AsyncOperatorRead["Java_org_apache_opendal_AsyncOperator_read"] AsyncOperatorWrite["Java_org_apache_opendal_AsyncOperator_write"] DisposeInternal["Java_org_apache_opendal_Operator_disposeInternal"] end subgraph "Rust Implementation" InternRead["intern_read()"] InternWrite["intern_write()"] InternStat["intern_stat()"] BlockingOperator["*mut blocking::Operator"] AsyncOperatorRust["*mut opendal::Operator"] BoxIntoRaw["Box::into_raw()"] BoxFromRaw["Box::from_raw()"] end Operator --> OperatorRead Operator --> OperatorWrite Operator --> OperatorStat AsyncOperator --> AsyncOperatorRead AsyncOperator --> AsyncOperatorWrite NativeObject --> DisposeInternal OperatorRead --> InternRead OperatorWrite --> InternWrite OperatorStat --> InternStat AsyncOperatorRead --> InternRead AsyncOperatorWrite --> InternWrite InternRead --> BlockingOperator InternWrite --> BlockingOperator InternStat --> BlockingOperator AsyncOperatorRead --> AsyncOperatorRust AsyncOperatorWrite --> AsyncOperatorRust BlockingOperator --> BoxIntoRaw AsyncOperatorRust --> BoxIntoRaw DisposeInternal --> BoxFromRaw ``` Sources: [bindings/java/src/operator.rs:42-48](), [bindings/java/src/operator.rs:67-101](), [bindings/java/src/async_operator.rs:86-92](), [bindings/java/src/async_operator.rs:185-226]() Looking at the existing content and the current codebase, I need to update several sections to better reflect the technical implementation details around async operators, native object lifecycle, and JNI integration. Let me make targeted updates: ## JNI Implementation The JNI bridge implements separate function sets for blocking and async operations, with sophisticated async request management through the `AsyncRegistry` pattern. ### Blocking Operations Flow ```mermaid flowchart LR subgraph "Java" OperatorRead["Operator.read()"] end subgraph "JNI" JNIRead["Java_org_apache_opendal_Operator_read"] InternRead["intern_read()"] end subgraph "Rust" BlockingOp["blocking::Operator.read_options()"] end OperatorRead --> JNIRead JNIRead --> InternRead InternRead --> BlockingOp ``` ### Async Operations Flow ```mermaid flowchart TD subgraph "Java" AsyncOpRead["AsyncOperator.read()"] AsyncRegistry["AsyncRegistry.requestId()"] CompletableFuture["CompletableFuture"] end subgraph "JNI" JNIAsyncRead["Java_org_apache_opendal_AsyncOperator_read"] InternAsyncRead["intern_read()"] RequestId["request_id()"] CompleteFuture["complete_future()"] end subgraph "Rust" Executor["Executor.spawn()"] AsyncOp["Operator.read_with()"] TokioTask["Tokio Task"] end AsyncOpRead --> JNIAsyncRead JNIAsyncRead --> RequestId RequestId --> AsyncRegistry AsyncRegistry --> CompletableFuture JNIAsyncRead --> InternAsyncRead InternAsyncRead --> Executor Executor --> TokioTask TokioTask --> AsyncOp AsyncOp --> CompleteFuture CompleteFuture --> CompletableFuture ``` Sources: [bindings/java/src/operator.rs:67-101](), [bindings/java/src/async_operator.rs:185-226](), [bindings/java/src/main/java/org/apache/opendal/AsyncOperator.java:64-102]() ## Native Object Lifecycle Management The Java binding uses a `NativeObject` base class to manage Rust object lifecycles, ensuring proper cleanup when Java objects are garbage collected. ### Native Handle Management ```mermaid flowchart TD subgraph "Java Objects" NativeObject["NativeObject\n(base class)"] Operator["Operator\n(extends NativeObject)"] AsyncOperator["AsyncOperator\n(extends NativeObject)"] AsyncExecutor["AsyncExecutor\n(extends NativeObject)"] end subgraph "Native Handles" BlockingHandle["*mut blocking::Operator"] AsyncHandle["*mut opendal::Operator"] ExecutorHandle["*mut Executor"] end subgraph "Cleanup Functions" DisposeBlocking["Java_org_apache_opendal_Operator_disposeInternal"] DisposeAsync["Java_org_apache_opendal_AsyncOperator_disposeInternal"] DisposeExecutor["Java_org_apache_opendal_AsyncExecutor_disposeInternal"] end Operator --> BlockingHandle AsyncOperator --> AsyncHandle AsyncExecutor --> ExecutorHandle BlockingHandle --> DisposeBlocking AsyncHandle --> DisposeAsync ExecutorHandle --> DisposeExecutor ``` Sources: [bindings/java/src/operator.rs:42-48](), [bindings/java/src/async_operator.rs:86-92](), [bindings/java/src/executor.rs:106-112]() ## Async Registry Pattern The `AsyncOperator` uses a singleton `AsyncRegistry` to manage outstanding `CompletableFuture` instances without requiring global JNI references. ### AsyncRegistry Implementation | Component | Purpose | Location | |-----------|---------|----------| | `AsyncRegistry.INSTANCE` | Singleton registry for futures | [AsyncOperator.java:48-103]() | | `requestId()` | Generate unique ID and store future | [AsyncOperator.java:64-74]() | | `get(long requestId)` | Retrieve future by ID (from native) | [AsyncOperator.java:85-87]() | | `take(long requestId)` | Remove and return future | [AsyncOperator.java:95-102]() | The registry avoids the complexity of managing global JNI references across multiple native threads by using request IDs as lightweight handles. Sources: [bindings/java/src/main/java/org/apache/opendal/AsyncOperator.java:48-103](), [bindings/java/src/async_operator.rs:709-729]() # Java Binding The Java binding provides access to OpenDAL from Java applications through JNI (Java Native Interface), offering both synchronous and asynchronous operators with proper native object lifecycle management. ## Architecture Overview The Java binding bridges Java applications to OpenDAL's Rust core through a JNI layer that manages async operations, native object lifecycles, and executor management. ```mermaid flowchart TD subgraph "Java Layer" JavaApp["Java Application"] Operator["Operator"] AsyncOperator["AsyncOperator"] AsyncRegistry["AsyncRegistry"] CompletableFuture["CompletableFuture"] NativeObject["NativeObject"] end subgraph "JNI Bridge" JNIFunctions["JNI Functions\n(Java_org_apache_opendal_*)"] InternFunctions["intern_* functions"] Executor["Executor"] RequestIds["Request ID Management"] end subgraph "Rust Core" BlockingOperator["blocking::Operator"] AsyncOperatorRust["opendal::Operator"] TokioRuntime["tokio::runtime::Runtime"] CoreAPI["OpenDAL Core"] end JavaApp --> Operator JavaApp --> AsyncOperator AsyncOperator --> AsyncRegistry AsyncRegistry --> CompletableFuture Operator --> NativeObject AsyncOperator --> NativeObject Operator --> JNIFunctions AsyncOperator --> JNIFunctions JNIFunctions --> InternFunctions InternFunctions --> Executor AsyncRegistry --> RequestIds InternFunctions --> BlockingOperator InternFunctions --> AsyncOperatorRust Executor --> TokioRuntime BlockingOperator --> CoreAPI AsyncOperatorRust --> CoreAPI ``` Sources: [bindings/java/src/lib.rs](), [bindings/java/src/operator.rs](), [bindings/java/src/async_operator.rs](), [bindings/java/src/executor.rs]() ## Core Java Classes The Java binding provides several key classes for interacting with storage services: ### Main Operator Classes ```mermaid classDiagram class NativeObject { <> #long nativeHandle #void disposeInternal(long handle)* +void close() } class Operator { +OperatorInfo info +static Operator of(String scheme, Map~String,String~ map) +Operator duplicate() +void write(String path, byte[] content, WriteOptions options) +byte[] read(String path, ReadOptions options) +void delete(String path) +Metadata stat(String path, StatOptions options) +void createDir(String path) +void copy(String sourcePath, String targetPath) +void rename(String sourcePath, String targetPath) +void removeAll(String path) +Entry[] list(String path, ListOptions options) +OperatorInputStream createInputStream(String path) +OperatorOutputStream createOutputStream(String path) -static long duplicate(long op) -static void write(long op, String path, byte[] content, WriteOptions options) -static byte[] read(long op, String path, ReadOptions options) } class AsyncOperator { +OperatorInfo info -long executorHandle +static AsyncOperator of(String scheme, Map~String,String~ map, AsyncExecutor executor) +AsyncOperator duplicate() +AsyncOperator layer(Layer layer) +Operator blocking() +CompletableFuture~Void~ write(String path, byte[] content, WriteOptions options) +CompletableFuture~byte[]~ read(String path, ReadOptions options) +CompletableFuture~Metadata~ stat(String path, StatOptions options) +CompletableFuture~PresignedRequest~ presignRead(String path, Duration duration) +CompletableFuture~List~Entry~~ list(String path, ListOptions options) -static long write(long nativeHandle, long executorHandle, String path, byte[] content, WriteOptions options) -static long read(long nativeHandle, long executorHandle, String path, ReadOptions options) } NativeObject <|-- Operator NativeObject <|-- AsyncOperator ``` ### Data Model Classes ```mermaid classDiagram class Metadata { +EntryMode mode +long contentLength +String contentDisposition +String contentMd5 +String contentType +String cacheControl +String etag +Instant lastModified +String version +boolean isFile() +boolean isDir() } class Entry { +String path +Metadata metadata +String getPath() +Metadata getMetadata() } class EntryMode { <> FILE DIR UNKNOWN +static EntryMode of(int mode) } class Capability { +boolean stat +boolean read +boolean write +boolean delete +boolean list +boolean copy +boolean rename +boolean presign +boolean createDir +long writeMultiMaxSize +long writeMultiMinSize +boolean statWithIfMatch +boolean writeWithContentType +boolean listWithLimit } class OperatorInfo { +String scheme +String root +String name +Capability fullCapability +Capability nativeCapability } Metadata --> EntryMode Entry --> Metadata OperatorInfo --> Capability ``` ### Options Classes ```mermaid classDiagram class WriteOptions { +String contentType +String contentDisposition +String cacheControl +String contentEncoding +String ifMatch +String ifNoneMatch +Map~String,String~ userMetadata +boolean append +boolean ifNotExists +int concurrent +long chunk +static WriteOptionsBuilder builder() } class ReadOptions { +long offset +long length +static ReadOptionsBuilder builder() } class ListOptions { +boolean recursive +long limit +String startAfter +boolean versions +boolean deleted +static ListOptionsBuilder builder() } class StatOptions { +String ifMatch +String ifNoneMatch +Instant ifModifiedSince +Instant ifUnmodifiedSince +String version +String overrideContentType +String overrideCacheControl +String overrideContentDisposition +static StatOptionsBuilder builder() } ``` Sources: [bindings/java/src/main/java/org/apache/opendal/Operator.java:30-172](), [bindings/java/src/main/java/org/apache/opendal/AsyncOperator.java:36-339](), [bindings/java/src/main/java/org/apache/opendal/Metadata.java:28-148](), [bindings/java/src/main/java/org/apache/opendal/Capability.java:25-305]() ## Asynchronous Operations The `AsyncOperator` provides `CompletableFuture`-based APIs backed by a Tokio runtime. Each async operation returns immediately with a future that completes when the underlying Rust operation finishes. ### AsyncRegistry Request Management The `AsyncOperator` uses a singleton `AsyncRegistry` to manage outstanding futures without requiring global JNI references: ```mermaid flowchart TD subgraph "Java AsyncOperator" AsyncOpRead["asyncOp.read(path)"] AsyncRegistry["AsyncRegistry.INSTANCE"] RequestIdGen["requestId()"] GetFuture["get(requestId)"] TakeFuture["take(requestId)"] CompletableFuture1["CompletableFuture"] ConcurrentHashMap["ConcurrentHashMap>"] end subgraph "JNI Layer" JNIAsyncRead["Java_org_apache_opendal_AsyncOperator_read"] InternRead["intern_read()"] RequestIdNative["request_id(env)"] CompleteFuture["complete_future(id, result)"] GetFutureNative["get_future(env, id)"] end subgraph "Rust Execution" ExecutorSpawn["executor.spawn(async move { ... })"] AsyncTask["Tokio Async Task"] OpReadWith["op.read_with(&path).await"] CallComplete["complete_future(id, result)"] end AsyncOpRead --> JNIAsyncRead JNIAsyncRead --> RequestIdNative RequestIdNative --> RequestIdGen RequestIdGen --> ConcurrentHashMap RequestIdGen --> CompletableFuture1 JNIAsyncRead --> InternRead InternRead --> ExecutorSpawn ExecutorSpawn --> AsyncTask AsyncTask --> OpReadWith OpReadWith --> CallComplete CallComplete --> CompleteFuture CompleteFuture --> GetFutureNative GetFutureNative --> GetFuture GetFuture --> ConcurrentHashMap TakeFuture --> ConcurrentHashMap ``` Sources: [bindings/java/src/main/java/org/apache/opendal/AsyncOperator.java:48-103](), [bindings/java/src/async_operator.rs:709-729](), [bindings/java/src/async_operator.rs:199-226]() ### JNI Implementation Patterns The JNI layer follows consistent patterns for error handling and native resource management: ```mermaid flowchart TD subgraph "JNI Function Pattern" JNIPublic["#[no_mangle] pub unsafe extern \"system\" fn Java_org_apache_opendal_Operator_read"] CallIntern["intern_read(&mut env, &mut *op, path, options)"] ErrorHandling[".unwrap_or_else(|e| { e.throw(&mut env); default_return })"] end subgraph "Intern Function Pattern" InternImpl["fn intern_read(env: &mut JNIEnv, op: &mut blocking::Operator, ...)"] ConvertInput["jstring_to_string(env, &path)?"] CallRust["op.read_options(&path, options)?"] ConvertOutput["bytes_to_jbytearray(env, content)?"] ReturnResult["Ok(result.into_raw())"] end subgraph "Error Conversion" RustError["opendal::Error"] BindingError["crate::error::Error"] ThrowJava["error.throw(&mut env)"] JavaException["OpenDALException"] end JNIPublic --> CallIntern CallIntern --> ErrorHandling CallIntern --> InternImpl InternImpl --> ConvertInput ConvertInput --> CallRust CallRust --> ConvertOutput ConvertOutput --> ReturnResult ErrorHandling --> RustError RustError --> BindingError BindingError --> ThrowJava ThrowJava --> JavaException ``` Sources: [bindings/java/src/operator.rs:67-101](), [bindings/java/src/operator.rs:80-101](), [bindings/java/src/convert.rs:189-192](), [bindings/java/src/error.rs]() ## Installation and Build System ### Maven Dependency Add the OpenDAL Java binding to your Maven project: ```xml org.apache.opendal opendal ${opendal.version} ``` ### Platform-Specific Native Libraries The Java binding automatically loads native libraries using the `NativeLibrary` utility: ```mermaid flowchart TD subgraph "Library Loading Strategy" LoadLibrary["NativeLibrary.loadLibrary()"] SystemLib["System.loadLibrary(\"opendal_java\")"] BundledLib["doLoadBundledLibrary()"] ClasspathResource["/native/{classifier}/{libraryName}"] TempFile["File.createTempFile()"] SystemLoad["System.load(tmpFile.getAbsolutePath())"] end subgraph "Platform Detection" Environment["Environment.getClassifier()"] OsName["System.getProperty(\"os.name\")"] OsArch["System.getProperty(\"os.arch\")"] Classifier["linux-x86_64, osx-aarch_64, windows-x86_64, etc."] end LoadLibrary --> SystemLib SystemLib --> |"UnsatisfiedLinkError"| BundledLib BundledLib --> Environment Environment --> OsName Environment --> OsArch OsName --> Classifier OsArch --> Classifier BundledLib --> ClasspathResource ClasspathResource --> TempFile TempFile --> SystemLoad ``` ### Build Process The Java binding uses a Python-driven build process to compile native code: ```mermaid flowchart LR subgraph "Maven Build Phases" CompileJava["compile (Java)"] ExecPython["exec:exec (Python build.py)"] TestJava["test (Java)"] PackageJar["package (JAR)"] end subgraph "Python Build Script" BuildPy["tools/build.py"] CargoArgs["--release --lib --crate-type cdylib"] CargoBuild["cargo build"] NativeLib["target/release/libopendal_java.{so,dylib,dll}"] end subgraph "JAR Variants" MainJar["opendal-{version}.jar"] NativeJar["opendal-{version}-{classifier}.jar"] ClassesDir["target/classes/"] NativeDir["target/classes/native/{classifier}/"] end CompileJava --> ExecPython ExecPython --> BuildPy BuildPy --> CargoBuild CargoBuild --> NativeLib ExecPython --> TestJava TestJava --> PackageJar PackageJar --> MainJar PackageJar --> NativeJar ClassesDir --> MainJar NativeDir --> NativeJar NativeLib --> NativeDir ``` Sources: [bindings/java/src/main/java/org/apache/opendal/NativeLibrary.java:69-124](), [bindings/java/pom.xml:197-273](), [bindings/java/tools/build.py]() ## Build System The Java binding uses Maven for building and includes a custom build process to compile the native Rust code. ```mermaid flowchart TD Maven["Maven Build"] Python["Python Build Script"] Rust["Cargo Build"] JarNoNative["JAR without native lib"] JarWithNative["JAR with native lib"] Maven --> |"exec-maven-plugin"| Python Python --> Rust Maven --> |"maven-jar-plugin\n(default-jar)"| JarNoNative Maven --> |"maven-jar-plugin\n(native-jar)"| JarWithNative Rust --> |"Compiled native library"| JarWithNative ``` The build process involves: 1. Maven invokes a Python script (`tools/build.py`) via the `exec-maven-plugin` 2. The Python script builds the Rust code with appropriate platform settings 3. Maven generates two JAR files: - A platform-independent JAR without native libraries - A platform-specific JAR containing the native library Sources: [bindings/java/pom.xml:197-273]() ## Using the Java Binding ### Creating an Operator There are two ways to create an operator: 1. Using a `ServiceConfig` object: ```java ServiceConfig config = S3Config.builder() .bucket("my-bucket") .region("us-east-1") .build(); Operator op = Operator.of(config); ``` 2. Using a scheme and property map: ```java Map props = new HashMap<>(); props.put("bucket", "my-bucket"); props.put("region", "us-east-1"); Operator op = Operator.of("s3", props); ``` ### Basic Operations ```java // Writing data op.write("path/to/file", "Hello, world!".getBytes()); // Reading data byte[] content = op.read("path/to/file"); // Checking metadata Metadata meta = op.stat("path/to/file"); if (meta.isFile()) { System.out.println("Content length: " + meta.contentLength); } // Listing files List entries = op.list("path/to/dir"); for (Entry entry : entries) { System.out.println(entry.path); } // Deleting files op.delete("path/to/file"); ``` ### Stream Operations For handling large files efficiently: ```java // Reading a large file try (OperatorInputStream is = op.createInputStream("path/to/large/file")) { // Use the input stream byte[] buffer = new byte[8192]; int bytesRead; while ((bytesRead = is.read(buffer)) != -1) { // Process data } } // Writing a large file try (OperatorOutputStream os = op.createOutputStream("path/to/large/file")) { // Use the output stream os.write("Hello, world!".getBytes()); } ``` ## Implementation Details ### Native Resource Management The `NativeObject` base class provides automatic cleanup of native resources through the `disposeInternal` pattern: ```mermaid flowchart TD subgraph "Java Object Lifecycle" JavaCreate["Object Creation"] JavaUse["Object Usage"] JavaGC["Garbage Collection"] JavaFinalize["finalize()"] end subgraph "Native Resource Lifecycle" NativeAlloc["Box::into_raw()"] NativeUse["Native Operations"] NativeCleanup["disposeInternal()"] NativeFree["Box::from_raw()"] end JavaCreate --> NativeAlloc JavaUse --> NativeUse JavaGC --> JavaFinalize JavaFinalize --> NativeCleanup NativeCleanup --> NativeFree ``` ### Native Object Lifecycle Management The `NativeObject` base class provides automatic cleanup through the `disposeInternal` pattern: ```mermaid flowchart TD subgraph "Java Object Creation" Constructor["new Operator(nativeHandle, info)"] NativeHandle["long nativeHandle"] BoxIntoRaw["Box::into_raw(Box::new(blocking_op)) as jlong"] end subgraph "Java Object Usage" JavaMethod["operator.read(path)"] JNICall["Java_org_apache_opendal_Operator_read(env, class, op, path, options)"] UnsafeDeref["unsafe { &mut *op }"] RustOperation["blocking_op.read_options(&path, options)"] end subgraph "Java Object Cleanup" JavaClose["operator.close() or GC finalize()"] DisposeCall["disposeInternal(nativeHandle)"] BoxFromRaw["drop(Box::from_raw(op))"] RustDrop["Drop trait implementation"] end Constructor --> NativeHandle BoxIntoRaw --> NativeHandle JavaMethod --> JNICall JNICall --> UnsafeDeref UnsafeDeref --> RustOperation JavaClose --> DisposeCall DisposeCall --> BoxFromRaw BoxFromRaw --> RustDrop ``` ### JNI Data Conversion Utilities | Java Type | Rust Type | Function | Usage Pattern | |-----------|-----------|----------|---------------| | `String` | `String` | `jstring_to_string()` | Path and option conversion | | `byte[]` | `Vec` | `env.convert_byte_array()` | Content data transfer | | `Map` | `HashMap` | `jmap_to_hashmap()` | Configuration and metadata | | `Instant` | `DateTime` | `read_instant_field_to_date_time()` | Timestamp conversion | | `Metadata` | `opendal::Metadata` | `make_metadata()` | File metadata objects | | `Entry` | `opendal::Entry` | `make_entry()` | Directory listing results | | `PresignedRequest` | `opendal::raw::PresignedRequest` | `make_presigned_request()` | URL presigning | Sources: [bindings/java/src/operator.rs:42-48](), [bindings/java/src/convert.rs:34-49](), [bindings/java/src/convert.rs:189-192](), [bindings/java/src/lib.rs:143-209](), [bindings/java/src/lib.rs:45-70]() ### Error Handling and Exception Translation The JNI layer converts Rust `Result` types to Java exceptions using the `Error::throw()` method pattern: ```rust // Pattern used throughout JNI functions fn jni_function() -> jlong { intern_function().unwrap_or_else(|e| { e.throw(&mut env); 0 // Return null handle on error }) } ``` Sources: [bindings/java/src/convert.rs:189-192](), [bindings/java/src/lib.rs:143-209](), [bindings/java/src/async_operator.rs:56-60]() ### Async Executor Integration with JVM The Java binding creates custom Tokio executors that integrate with the JVM for async operations: ```mermaid flowchart TD subgraph "Executor Creation" MakeExecutor["make_tokio_executor(num_threads)"] TokioBuilder["tokio::runtime::Builder::new_multi_thread()"] WorkerThreads["worker_threads(num_threads)"] OnThreadStart["on_thread_start(move || { ... })"] ThreadParkTimeout["thread_park_timeout(Duration::from_millis(1))"] end subgraph "JVM Thread Integration" AttachDaemon["vm.attach_current_thread_as_daemon()"] SetThreadName["set_current_thread_name(&mut env)"] ThreadLocalEnv["ENV.with(|cell| { *cell.borrow_mut() = Some(env.get_raw()) })"] GetCurrentEnv["get_current_env() -> JNIEnv"] end subgraph "Async Operation Flow" ExecutorSpawn["executor.spawn(async move { ... })"] AsyncWork["op.read_with(&path).await"] GetEnv["get_current_env()"] CompleteFuture["complete_future(id, result)"] JavaCallback["CompletableFuture.complete(result)"] end MakeExecutor --> TokioBuilder TokioBuilder --> WorkerThreads TokioBuilder --> OnThreadStart TokioBuilder --> ThreadParkTimeout OnThreadStart --> AttachDaemon AttachDaemon --> SetThreadName SetThreadName --> ThreadLocalEnv ExecutorSpawn --> AsyncWork AsyncWork --> GetEnv ThreadLocalEnv --> GetCurrentEnv GetCurrentEnv --> GetEnv GetEnv --> CompleteFuture CompleteFuture --> JavaCallback ``` ### Error Handling and Exception Translation Rust errors are converted to Java exceptions using a consistent pattern: ```mermaid flowchart LR subgraph "Rust Error Types" OpenDALError["opendal::Error"] JNIError["jni::errors::Error"] ConvertError["convert::Error"] IOError["std::io::Error"] end subgraph "Binding Error Layer" BindingError["crate::error::Error"] FromOpenDAL["impl From"] FromJNI["impl From"] ThrowMethod["error.throw(&mut env)"] end subgraph "Java Exceptions" OpenDALException["OpenDALException"] RuntimeException["RuntimeException"] IOException["IOException"] end OpenDALError --> FromOpenDAL JNIError --> FromJNI ConvertError --> BindingError IOError --> BindingError FromOpenDAL --> BindingError FromJNI --> BindingError BindingError --> ThrowMethod ThrowMethod --> OpenDALException ThrowMethod --> RuntimeException ThrowMethod --> IOException ``` Sources: [bindings/java/src/executor.rs:114-144](), [bindings/java/src/executor.rs:53-71](), [bindings/java/src/error.rs](), [bindings/java/src/async_operator.rs:683-707]() ## Configuration Options The Java binding supports all the storage services and configuration options that OpenDAL core provides. Some of the commonly used services include: | Service | Scheme | Key Configuration Properties | |---------|--------|------------------------------| | Amazon S3 | "s3" | bucket, region, endpoint | | Azure Blob | "azblob" | container, account_name | | Google Cloud Storage | "gcs" | bucket, root | | File System | "fs" | root | | Redis | "redis" | url | | Memory | "memory" | - | ## Error Handling The Java binding throws `OpenDALException` for any errors that occur during operations: ```java try { op.read("non-existent-file"); } catch (OpenDALException e) { System.err.println("Error: " + e.getMessage()); // Optional: get the specific error kind String errorKind = e.getKind(); } ``` ## Implementation Details ### Native Resource Management The Java binding uses a `NativeObject` base class to manage the lifecycle of native resources. Both `Operator` and `AsyncOperator` extend this class, which ensures that native resources are properly released when the Java object is garbage collected. ### JNI Data Conversion The JNI layer handles conversion between Java and Rust data types: 1. Java strings are converted to Rust strings 2. Java byte arrays are converted to Rust vectors 3. Rust metadata is converted to Java `Metadata` objects 4. Rust entries are converted to Java `Entry` objects Sources: [bindings/java/src/lib.rs:45-205]() ## Cross-Platform Support The Java binding supports multiple platforms through platform-specific native libraries: 1. The Maven build detects the platform using `os-maven-plugin` 2. Platform-specific JAR files include the appropriate native library 3. Custom classifiers are used to distinguish platform-specific JARs Sources: [bindings/java/pom.xml:188-194](), [bindings/java/pom.xml:250-271]() ## Performance Considerations The Java binding is designed to provide good performance by: 1. Minimizing JNI crossings for operations on large data 2. Providing stream-based APIs for efficient I/O 3. Offering both synchronous and asynchronous operation modes 4. Supporting direct access to native capabilities through `operator.info` ## Limitations 1. The Java binding requires native libraries, which must be built for each target platform 2. Some advanced features of OpenDAL may not be fully exposed in the Java API 3. Error messages from the Rust core may not always provide the most detailed Java stack traces ## Conclusion The OpenDAL Java binding provides a robust and efficient way for Java applications to access various storage services through a unified API. It combines the type safety and familiarity of Java with the performance and flexibility of OpenDAL's Rust core.