File Operation Patterns - Auto-Detection & Explicit Control

Smart Defaults with Expert Control - The SDK provides intelligent file size detection while offering explicit control when needed.

This guide explains the SDK's file operation pattern, helping you choose the right method for your use case.

Pattern Overview

The AnkaSecure SDK implements a three-tier API pattern for file operations:

graph TD
    A[SDK File Operation] --> B{Method Pattern}
    B -->|No Suffix| C[Auto-Detection]
    B -->|Compact Suffix| D[Force Compact]
    B -->|Stream Suffix| E[Force Streaming]

    C --> F{File Size Check<br/>vs discovered limit}
    F -->|< maxPlaintextBytes| G[Compact JWE/JWS<br/>Single-line format]
    F -->|>= maxPlaintextBytes| H[Streaming JWE/JWS<br/>Detached format]

    D --> G
    E --> H

    style C fill:#4CAF50
    style D fill:#2196F3
    style E fill:#FF9800
    style G fill:#E8F5E9
    style H fill:#FFF3E0

Decision Matrix

When to Use Each Pattern

Auto-Detection Pattern (Recommended)

How It Works

sequenceDiagram
    participant App as Application
    participant SDK as SDK
    participant Helper as StreamingThresholdHelper
    participant API as AnkaSecure API

    App->>SDK: encryptFile(kid, input, output)
    SDK->>Helper: shouldUseStreaming(input)
    Helper->>Helper: Check file size

    Helper->>API: GET /api/v3/crypto/limits (cached)
    API-->>Helper: { maxPlaintextBytes }

    alt File >= maxPlaintextBytes
        Helper-->>SDK: true (use streaming)
        SDK->>API: POST /api/v3/crypto/stream/encrypt
        API-->>SDK: Detached JWE
    else File < maxPlaintextBytes
        Helper-->>SDK: false (use compact)
        SDK->>API: POST /api/v3/crypto/encrypt
        API-->>SDK: Compact JWE (or 413 → SDK retries via streaming)
    end

    SDK-->>App: EncryptResult

Threshold Details

The auto-detection threshold is discovered from the server at runtime, not hardcoded. The SDK reads the effective compact-mode limit from GET /api/v3/crypto/limits and caches it (short TTL):

// StreamingThresholdHelper resolves the threshold dynamically:
//   GET /api/v3/crypto/limits  ->  { "maxPlaintextBytes": 4000000 }
// shouldUseStreaming(input) == (inputSize >= maxPlaintextBytes)
// Fail-safe to a conservative code default if discovery is unavailable.

Why discover it (instead of a hardcoded constant)?

• The server limit (crypto.max-plaintext-bytes, default 4,000,000 bytes) is operator-configurable and hot-reloadable — a hardcoded client constant would drift from the deployment.
• It removes the former 5 MiB (5_242_880) client vs 5 MB (5_000_000) server mismatch.
• Compact mode loads the entire file into memory (inefficient for large files); streaming uses chunked transfer with constant memory.

413 → streaming automatic fallback. Discovery is cached, so a mid-flight operator change (or a lowered limit) could leave a stale value. To stay correct, a compact producer call that receives 413 payload-too-large transparently retries the equivalent streaming operation — the 413 is never surfaced to your code.

Example: Auto-Detection in Action

import co.ankatech.ankasecure.sdk.AuthenticatedSdk;
import java.nio.file.Path;

// Authenticate once, reuse for all operations
AuthenticatedSdk sdk = factory.authenticateApplication(clientId, clientSecret);

// Small file (2 MB) - automatically uses compact mode
Path configFile = Path.of("config.json");  // 2 MB
EncryptResult result1 = sdk.encryptFile("my-key", configFile, Path.of("config.jwe"));
// → Compact JWE: single-line Base64 string

// Large file (50 MB) - automatically uses streaming mode
Path databaseDump = Path.of("backup.sql");  // 50 MB
EncryptResult result2 = sdk.encryptFile("my-key", databaseDump, Path.of("backup.jwe"));
// → Streaming JWE: detached format with multipart structure

// You don't need to think about file sizes!

Debug logs show the decision:

DEBUG co.ankatech.cli.dev - Auto-streaming: 2097152 bytes, using compact mode
DEBUG co.ankatech.cli.dev - Auto-streaming: 52428800 bytes, using streaming mode

Explicit Control Patterns

Force Compact Mode

Use when you need a single-line string for APIs or storage.

// Force compact JWE (will fail if file >= the configured maxPlaintextBytes)
Path smallFile = Path.of("document.pdf");  // 2 MB
EncryptResult result = sdk.encryptFileCompact("my-key", smallFile, Path.of("document.jwe"));

// Read as single-line string
String jweToken = Files.readString(Path.of("document.jwe"));

// Send via REST API
JsonObject payload = new JsonObject();
payload.addProperty("encryptedData", jweToken);
httpClient.post("/your-app/store", payload);

Error handling:

try {
    sdk.encryptFileCompact("my-key", largeFile, output);
} catch (AnkaSecureSdkException e) {
    if (e.getHttpStatus() == 413) {
        // Payload exceeded the server's compact limit -- use streaming instead
        sdk.encryptFileStream("my-key", largeFile, output);
    }
}

The oversize condition is the server's 413, read from getHttpStatus(). SdkErrorCode is a coarse category and carries no size-specific member, so a 413 arrives as HTTP. Read the current member list from SdkErrorCode itself rather than from this page: an enumeration copied into prose is a second copy of a vocabulary, and the first version of this paragraph already listed five of its six members. Forcing the compact form is the only case where you handle this yourself: the auto-detecting encryptFile already retries via streaming on a 413.

Force Streaming Mode

Use for large files or memory-constrained environments.

// Force streaming (works for any file size)
Path largeFile = Path.of("video.mp4");  // 2 GB
EncryptResult result = sdk.encryptFileStream("archive-key", largeFile, Path.of("video.jwe"));

// Memory usage: bounded and independent of file size (see the note below)
// Process time: Linear with file size

When to force streaming:

• Files > 100 MB (even below the discovered threshold, streaming is more efficient at this scale)
• Embedded systems with limited RAM
• Processing user-uploaded files of unknown size
• Need binary output format (not Base64 text)

Operations Supporting Auto-Detection

Encryption Operations

Note: Decryption auto-detects by input format, not file size:

• If input is compact JWE → uses compact mode
• If input is streaming JWE → uses streaming mode

Signature Operations

Note: Verification requires knowing the input format explicitly (no auto-detection).

Combined Operations

Limitation: Combined operations only support compact mode (inputs below the discovered maxPlaintextBytes).

Format Comparison

Compact JWE Format

Structure: Single-line Base64 string (5 parts separated by dots)

eyJhbGc...header.eyJlbmM...iv.ciphertext.tag

Characteristics:

• Size limit: below maxPlaintextBytes (server-enforced, operator-configurable, default 4 MB; discover via GET /api/v3/crypto/limits)
• Memory: Entire file loaded into RAM
• Output: Text (Base64-encoded)
• Use case: APIs, JSON storage, small files

Example output:

{
  "jweToken": "eyJhbGciOiJSU0EtT0FFUC0yNTYiLCJlbmMiOiJBMjU2R0NNIn0.rT-1...",
  "format": "COMPACT",
  "algorithmUsed": "RSA-OAEP-256 + AES-256-GCM"
}

Streaming JWE Format

Structure: Detached format with metadata file

output.jwe       ← Encrypted ciphertext (binary)
output.jwe.meta  ← JWE metadata (JSON)

Characteristics:

• Size limit: No limit (tested up to 2 GB)
• Memory: bounded and independent of file size

• Output: Binary + metadata
• Use case: Large files, low memory, archives

Example metadata (output.jwe.meta):

{
  "jwe": "eyJhbGciOiJSU0EtT0FFUC0yNTYiLCJlbmMiOiJBMjU2R0NNIn0.header.iv..tag",
  "format": "DETACHED",
  "originalSize": 524288000,
  "algorithmUsed": "RSA-OAEP-256 + AES-256-GCM"
}

Performance Implications

Memory Usage

graph LR
    A[File Size] --> B{Operation Mode}

    B -->|Compact| C[RAM = File Size]
    B -->|Streaming| D[RAM bounded]

    C --> E[100 MB file<br/>= 100 MB RAM]
    D --> F[2 GB file<br/>= same bounded RAM]

    style E fill:#FFEBEE
    style F fill:#E8F5E9

Processing Speed

The compact/streaming boundary is not a file-size rule of thumb — it is the server's maxPlaintextBytes, default 4,000,000 bytes, operator-configurable and discoverable at GET /api/v3/crypto/limits. A 5 MB input is above that default, so compact mode rejects it, and the SDK's auto-detection selects streaming for exactly that reason. Do not hardcode a threshold: read the limit from the endpoint.

Key insight: compact mode avoids streaming's multipart overhead for small inputs, while streaming scales to arbitrary size with bounded memory. Latency depends on algorithm, payload size, network and deployment hardware, so measure against your own environment rather than budgeting from a published figure.

Best Practices

1. Use Auto-Detection by Default

// ✓ GOOD: Let SDK choose optimal format
sdk.encryptFile("my-key", unknownSizeFile, output);

// ✗ BAD: Manual size checking against a hardcoded constant
if (Files.size(file) < 5_000_000) {   // drifts from the server's configurable limit
    sdk.encryptFileCompact("my-key", file, output);
} else {
    sdk.encryptFileStream("my-key", file, output);
}
// → SDK already does this, using the server-discovered maxPlaintextBytes!

2. Force Compact for API Integration

// ✓ GOOD: Explicit compact for REST API
EncryptResult result = sdk.encryptFileCompact("api-key", configFile, tempFile);
String jweToken = Files.readString(tempFile);
apiClient.sendEncryptedConfig(jweToken);

3. Force Streaming for Large Files

// ✓ GOOD: Streaming for known large files
sdk.encryptFileStream("backup-key", databaseDump, output);

4. Handle Both Formats in Decrypt

// ✓ GOOD: Auto-detection handles both formats
EncryptResult encResult = sdk.encryptFile("my-key", input, encrypted);

// Later: decryption auto-detects format
DecryptResultMetadata decResult = sdk.decryptFile(encrypted, decrypted);
// Works whether encrypted is compact or streaming!

Common Scenarios

Scenario 1: Unknown File Sizes (User Uploads)

// User uploads file via web form (size unknown)
@PostMapping("/upload")
public ResponseEntity<String> handleUpload(@RequestParam("file") MultipartFile upload) {
    Path tempFile = Files.createTempFile("upload", ".bin");
    upload.transferTo(tempFile.toFile());

    // ✓ Auto-detection handles any size
    EncryptResult result = sdk.encryptFile("upload-key", tempFile, Path.of("encrypted.jwe"));

    return ResponseEntity.ok("Encrypted with: " + result.getAlgorithmUsed());
}

Scenario 2: Database Storage (Compact Required)

// Store encrypted tokens in PostgreSQL JSONB column
Path sensitiveData = Path.of("credentials.json");

// ✓ Force compact for database storage
EncryptResult result = sdk.encryptFileCompact("db-key", sensitiveData, tempOutput);
String jweToken = Files.readString(tempOutput);

// Store in database
jdbcTemplate.update(
    "UPDATE secrets SET encrypted_data = ?::jsonb WHERE id = ?",
    "{\"jwe\": \"" + jweToken + "\"}",
    secretId
);

Scenario 3: Batch Processing (Memory Efficiency)

// Process 1000 files in parallel (limited RAM)
List<Path> files = findAllFiles("/data/archive/");

files.parallelStream().forEach(file -> {
    // ✓ Streaming mode keeps memory constant
    sdk.encryptFileStream("batch-key", file, file.resolveSibling(file.getFileName() + ".jwe"));
});

// Total memory: 1000 threads * a bounded per-stream footprint (not 1000 * file_size)

Scenario 4: Signature Verification (Format-Specific)

// Received signed document from external partner
Path document = Path.of("contract.pdf");
Path signature = Path.of("contract.jws");

// ✓ Check signature format first
String sigContent = Files.readString(signature);
if (sigContent.startsWith("eyJ")) {
    // Compact JWS (single-line)
    VerifySignatureResult result = sdk.verifySignature(signature);
} else {
    // Streaming JWS (detached)
    VerifySignatureResult result = sdk.verifySignatureStream(document, signature);
}

Troubleshooting

Error: File Too Large for Compact Mode

Symptom:

AnkaSecureSdkException: File size (6291456 bytes) exceeds the configured compact mode limit (maxPlaintextBytes)
HTTP status: 413   ErrorCode: HTTP

Solution:

// Option 1: Use auto-detection (will switch to streaming automatically;
//           also retries via streaming if the server responds 413)
sdk.encryptFile("my-key", largeFile, output);

// Option 2: Explicitly use streaming
sdk.encryptFileStream("my-key", largeFile, output);

Error: Format Mismatch in Decryption

Symptom:

AnkaSecureSdkException: Cannot decrypt compact JWE with streaming endpoint
ErrorCode: INVALID_FORMAT

Solution:

// Use auto-detection (handles both formats)
sdk.decryptFile(encrypted, decrypted);

// OR: Check format first
if (isCompactFormat(encrypted)) {
    sdk.decryptFileCompact(encrypted, decrypted);
} else {
    sdk.decryptFileStream(encrypted, decrypted);
}

Memory Issues with Large Files

Symptom:

OutOfMemoryError: Java heap space

Cause: Using compact mode for large files

Solution:

// ✗ BAD: Compact mode loads entire file
sdk.encryptFileCompact("key", gigabyteFile, output);

// ✓ GOOD: Streaming mode uses constant memory
sdk.encryptFileStream("key", gigabyteFile, output);

Summary

Quick Reference

Decision Flowchart

flowchart TD
    Start([Need to encrypt/sign file]) --> Question1{Know the file size?}

    Question1 -->|No| UseAuto[Use Auto-Detection<br/>encryptFile]
    Question1 -->|Yes| Question2{File < maxPlaintextBytes?}

    Question2 -->|Yes| Question3{Need single-line string?}
    Question2 -->|No| UseStream[Force Streaming<br/>encryptFileStream]

    Question3 -->|Yes| UseCompact[Force Compact<br/>encryptFileCompact]
    Question3 -->|No| UseAuto2[Use Auto-Detection<br/>encryptFile]

    UseAuto --> Success([Operation Complete])
    UseCompact --> Success
    UseStream --> Success
    UseAuto2 --> Success

    style UseAuto fill:#4CAF50,color:#fff
    style UseCompact fill:#2196F3,color:#fff
    style UseStream fill:#FF9800,color:#fff
    style Success fill:#8BC34A,color:#fff

Complete Flow Examples by Pattern

The SDK includes 21 production-ready integration flows. Here are the most relevant for each pattern:

🔄 Auto-Detection Pattern

📦 Force Compact Pattern (below the configured limit)

🌊 Force Streaming Pattern (Large Files)

🔐 Combined & Advanced Patterns

See Integration Flows Catalogue for all 21 flows organized by category.

Next Steps

• Integration Flows: See 21 complete examples of SDK operations
• Usage Guide: API reference for all SDK methods
• Security Best Practices: Credential management and secure integration

© 2025 AnkaTech. All rights reserved.