File Operation Patterns - Auto-Detection & Explicit Control

Smart Defaults with Expert Control - The SDK provides intelligent file size detection while offering explicit control when needed.

This guide explains the SDK's file operation pattern, helping you choose the right method for your use case.

Pattern Overview

The AnkaSecure SDK implements a three-tier API pattern for file operations:

graph TD
    A[SDK File Operation] --> B{Method Pattern}
    B -->|No Suffix| C[Auto-Detection]
    B -->|Compact Suffix| D[Force Compact]
    B -->|Stream Suffix| E[Force Streaming]

    C --> F{File Size Check}
    F -->|< 5 MB| G[Compact JWE/JWS<br/>Single-line format]
    F -->|>= 5 MB| H[Streaming JWE/JWS<br/>Detached format]

    D --> G
    E --> H

    style C fill:#4CAF50
    style D fill:#2196F3
    style E fill:#FF9800
    style G fill:#E8F5E9
    style H fill:#FFF3E0

Decision Matrix

When to Use Each Pattern

Pattern	When to Use	Example Use Case
Auto-Detection `encryptFile()`	Default choice - handles any file size	General-purpose encryption without size concerns
Force Compact `encryptFileCompact()`	Need single-line JWE/JWS for APIs or JSON	Storing encrypted tokens in databases, REST API transmission
Force Streaming `encryptFileStream()`	Large files, low memory, or binary format required	Encrypting 500MB+ files, embedded systems with limited RAM

Auto-Detection Pattern (Recommended)

How It Works

sequenceDiagram
    participant App as Application
    participant SDK as SDK
    participant Helper as StreamingThresholdHelper
    participant API as AnkaSecure API

    App->>SDK: encryptFile(kid, input, output)
    SDK->>Helper: shouldUseStreaming(input)
    Helper->>Helper: Check file size

    alt File >= 5 MB
        Helper-->>SDK: true (use streaming)
        SDK->>API: POST /api/crypto/v1/encrypt/stream
        API-->>SDK: Detached JWE
    else File < 5 MB
        Helper-->>SDK: false (use compact)
        SDK->>API: POST /api/crypto/v1/encrypt
        API-->>SDK: Compact JWE
    end

    SDK-->>App: EncryptResult

Threshold Details

The auto-detection uses a 5 MiB threshold (5,242,880 bytes):

// From StreamingThresholdHelper.java
public static final long STREAMING_THRESHOLD = 5_242_880; // 5 MiB

Why 5 MB? - Server compact endpoint rejects payloads >= 5 MB (HTTP 413) - Compact mode loads entire file into memory (inefficient for large files) - Streaming mode uses chunked transfer encoding (constant memory)

Example: Auto-Detection in Action

import co.ankatech.ankasecure.sdk.AuthenticatedSdk;
import java.nio.file.Path;

// Authenticate once, reuse for all operations
AuthenticatedSdk sdk = factory.authenticateApplication(clientId, clientSecret);

// Small file (2 MB) - automatically uses compact mode
Path configFile = Path.of("config.json");  // 2 MB
EncryptResult result1 = sdk.encryptFile("my-key", configFile, Path.of("config.jwe"));
// → Compact JWE: single-line Base64 string

// Large file (50 MB) - automatically uses streaming mode
Path databaseDump = Path.of("backup.sql");  // 50 MB
EncryptResult result2 = sdk.encryptFile("my-key", databaseDump, Path.of("backup.jwe"));
// → Streaming JWE: detached format with multipart structure

// You don't need to think about file sizes!

Debug logs show the decision:

DEBUG co.ankatech.cli.dev - Auto-streaming: 2097152 bytes, using compact mode
DEBUG co.ankatech.cli.dev - Auto-streaming: 52428800 bytes, using streaming mode

Explicit Control Patterns

Force Compact Mode

Use when you need a single-line string for APIs or storage.

// Force compact JWE (will fail if file >= 5 MB)
Path smallFile = Path.of("document.pdf");  // 2 MB
EncryptResult result = sdk.encryptFileCompact("my-key", smallFile, Path.of("document.jwe"));

// Read as single-line string
String jweToken = Files.readString(Path.of("document.jwe"));

// Send via REST API
JsonObject payload = new JsonObject();
payload.addProperty("encryptedData", jweToken);
httpClient.post("/api/store", payload);

Error handling:

try {
    sdk.encryptFileCompact("my-key", largeFile, output);
} catch (AnkaSecureSdkException e) {
    if (e.getErrorCode() == SdkErrorCode.FILE_TOO_LARGE) {
        // File >= 5 MB, use streaming instead
        sdk.encryptFileStream("my-key", largeFile, output);
    }
}

Force Streaming Mode

Use for large files or memory-constrained environments.

// Force streaming (works for any file size)
Path largeFile = Path.of("video.mp4");  // 2 GB
EncryptResult result = sdk.encryptFileStream("archive-key", largeFile, Path.of("video.jwe"));

// Memory usage: ~64KB constant (streaming buffer)
// Process time: Linear with file size

When to force streaming: - Files > 100 MB (even though threshold is 5 MB, streaming is more efficient) - Embedded systems with limited RAM - Processing user-uploaded files of unknown size - Need binary output format (not Base64 text)

Operations Supporting Auto-Detection

Encryption Operations

Method	Auto-Detection	Compact	Streaming
Encrypt	`encryptFile()` ✓	`encryptFileCompact()`	`encryptFileStream()`
Decrypt	`decryptFile()` ✓	`decryptFileCompact()`	`decryptFileStream()`
Re-encrypt	`reencryptFile()` ✓	`reencryptFileCompact()`	`reencryptFileStream()`

Note: Decryption auto-detects by input format, not file size: - If input is compact JWE → uses compact mode - If input is streaming JWE → uses streaming mode

Signature Operations

Method	Auto-Detection	Compact	Streaming
Sign	`signFile()` ✓	`signFileCompact()`	`signFileStream()`
Verify	N/A	`verifySignatureCompact()`	`verifySignatureStream()`
Re-sign	N/A	`resignFileCompact()`	`resignFileStream()`

Note: Verification requires knowing the input format explicitly (no auto-detection).

Combined Operations

Method	Auto-Detection	Compact	Streaming
Sign-then-Encrypt	N/A	`signThenEncryptFileCompact()`	Not available
Decrypt-then-Verify	N/A	`decryptThenVerifyFileCompact()`	Not available

Limitation: Combined operations only support compact mode (<5 MB files).

Format Comparison

Compact JWE Format

Structure: Single-line Base64 string (5 parts separated by dots)

eyJhbGc...header.eyJlbmM...iv.ciphertext.tag

Characteristics: - Size limit: < 5 MB (server enforced) - Memory: Entire file loaded into RAM - Output: Text (Base64-encoded) - Use case: APIs, JSON storage, small files

Example output:

{
  "jweToken": "eyJhbGciOiJSU0EtT0FFUC0yNTYiLCJlbmMiOiJBMjU2R0NNIn0.rT-1...",
  "format": "COMPACT",
  "algorithmUsed": "RSA-OAEP-256 + AES-256-GCM"
}

Streaming JWE Format

Structure: Detached format with metadata file

output.jwe       ← Encrypted ciphertext (binary)
output.jwe.meta  ← JWE metadata (JSON)

Characteristics: - Size limit: No limit (tested up to 2 GB) - Memory: Constant ~64KB buffer - Output: Binary + metadata - Use case: Large files, low memory, archives

Example metadata (output.jwe.meta):

{
  "jwe": "eyJhbGciOiJSU0EtT0FFUC0yNTYiLCJlbmMiOiJBMjU2R0NNIn0.header.iv..tag",
  "format": "DETACHED",
  "originalSize": 524288000,
  "algorithmUsed": "RSA-OAEP-256 + AES-256-GCM"
}

Performance Implications

Memory Usage

graph LR
    A[File Size] --> B{Operation Mode}

    B -->|Compact| C[RAM = File Size]
    B -->|Streaming| D[RAM = ~64KB]

    C --> E[100 MB file<br/>= 100 MB RAM]
    D --> F[2 GB file<br/>= 64 KB RAM]

    style E fill:#FFEBEE
    style F fill:#E8F5E9

Processing Speed

File Size	Compact Mode	Streaming Mode	Recommendation
< 1 MB	~50 ms	~80 ms	Use compact (faster)
5 MB	~200 ms	~250 ms	Auto-detection ideal
50 MB	N/A (fails)	~2 seconds	Streaming only
500 MB	N/A	~20 seconds	Streaming only

Key insight: Compact mode is slightly faster for small files, but streaming mode scales linearly with no memory penalty.

Best Practices

1. Use Auto-Detection by Default

// ✓ GOOD: Let SDK choose optimal format
sdk.encryptFile("my-key", unknownSizeFile, output);

// ✗ BAD: Manual size checking
if (Files.size(file) < 5_000_000) {
    sdk.encryptFileCompact("my-key", file, output);
} else {
    sdk.encryptFileStream("my-key", file, output);
}
// → SDK already does this!

2. Force Compact for API Integration

// ✓ GOOD: Explicit compact for REST API
EncryptResult result = sdk.encryptFileCompact("api-key", configFile, tempFile);
String jweToken = Files.readString(tempFile);
apiClient.sendEncryptedConfig(jweToken);

3. Force Streaming for Large Files

// ✓ GOOD: Streaming for known large files
sdk.encryptFileStream("backup-key", databaseDump, output);

4. Handle Both Formats in Decrypt

// ✓ GOOD: Auto-detection handles both formats
EncryptResult encResult = sdk.encryptFile("my-key", input, encrypted);

// Later: decryption auto-detects format
DecryptResultMetadata decResult = sdk.decryptFile(encrypted, decrypted);
// Works whether encrypted is compact or streaming!

Common Scenarios

Scenario 1: Unknown File Sizes (User Uploads)

// User uploads file via web form (size unknown)
@PostMapping("/upload")
public ResponseEntity<String> handleUpload(@RequestParam("file") MultipartFile upload) {
    Path tempFile = Files.createTempFile("upload", ".bin");
    upload.transferTo(tempFile.toFile());

    // ✓ Auto-detection handles any size
    EncryptResult result = sdk.encryptFile("upload-key", tempFile, Path.of("encrypted.jwe"));

    return ResponseEntity.ok("Encrypted with: " + result.getAlgorithmUsed());
}

Scenario 2: Database Storage (Compact Required)

// Store encrypted tokens in PostgreSQL JSONB column
Path sensitiveData = Path.of("credentials.json");

// ✓ Force compact for database storage
EncryptResult result = sdk.encryptFileCompact("db-key", sensitiveData, tempOutput);
String jweToken = Files.readString(tempOutput);

// Store in database
jdbcTemplate.update(
    "UPDATE secrets SET encrypted_data = ?::jsonb WHERE id = ?",
    "{\"jwe\": \"" + jweToken + "\"}",
    secretId
);

Scenario 3: Batch Processing (Memory Efficiency)

// Process 1000 files in parallel (limited RAM)
List<Path> files = findAllFiles("/data/archive/");

files.parallelStream().forEach(file -> {
    // ✓ Streaming mode keeps memory constant
    sdk.encryptFileStream("batch-key", file, file.resolveSibling(file.getFileName() + ".jwe"));
});

// Total memory: 1000 threads * 64KB = 64 MB (not 1000 * file_size)

Scenario 4: Signature Verification (Format-Specific)

// Received signed document from external partner
Path document = Path.of("contract.pdf");
Path signature = Path.of("contract.jws");

// ✓ Check signature format first
String sigContent = Files.readString(signature);
if (sigContent.startsWith("eyJ")) {
    // Compact JWS (single-line)
    VerifySignatureResult result = sdk.verifySignatureCompact(signature);
} else {
    // Streaming JWS (detached)
    VerifySignatureResult result = sdk.verifySignatureStream(document, signature);
}

Troubleshooting

Error: File Too Large for Compact Mode

Symptom:

AnkaSecureSdkException: File size (6291456 bytes) exceeds compact mode limit (5242880 bytes)
ErrorCode: FILE_TOO_LARGE

Solution:

// Option 1: Use auto-detection (will switch to streaming automatically)
sdk.encryptFile("my-key", largeFile, output);

// Option 2: Explicitly use streaming
sdk.encryptFileStream("my-key", largeFile, output);

Error: Format Mismatch in Decryption

Symptom:

AnkaSecureSdkException: Cannot decrypt compact JWE with streaming endpoint
ErrorCode: INVALID_FORMAT

Solution:

// Use auto-detection (handles both formats)
sdk.decryptFile(encrypted, decrypted);

// OR: Check format first
if (isCompactFormat(encrypted)) {
    sdk.decryptFileCompact(encrypted, decrypted);
} else {
    sdk.decryptFileStream(encrypted, decrypted);
}

Memory Issues with Large Files

Symptom:

OutOfMemoryError: Java heap space

Cause: Using compact mode for large files

Solution:

// ✗ BAD: Compact mode loads entire file
sdk.encryptFileCompact("key", gigabyteFile, output);

// ✓ GOOD: Streaming mode uses constant memory
sdk.encryptFileStream("key", gigabyteFile, output);

Summary

Quick Reference

Your Need	Use This Method	Why
General encryption	`encryptFile()`	Auto-detection handles everything
API/JSON storage	`encryptFileCompact()`	Single-line string required
Large files (>100 MB)	`encryptFileStream()`	Memory efficiency
Unknown file sizes	`encryptFile()`	Auto-detection adapts
Low RAM environment	`encryptFileStream()`	Constant 64KB memory

Decision Flowchart

flowchart TD
    Start([Need to encrypt/sign file]) --> Question1{Know the file size?}

    Question1 -->|No| UseAuto[Use Auto-Detection<br/>encryptFile]
    Question1 -->|Yes| Question2{File < 5 MB?}

    Question2 -->|Yes| Question3{Need single-line string?}
    Question2 -->|No| UseStream[Force Streaming<br/>encryptFileStream]

    Question3 -->|Yes| UseCompact[Force Compact<br/>encryptFileCompact]
    Question3 -->|No| UseAuto2[Use Auto-Detection<br/>encryptFile]

    UseAuto --> Success([Operation Complete])
    UseCompact --> Success
    UseStream --> Success
    UseAuto2 --> Success

    style UseAuto fill:#4CAF50,color:#fff
    style UseCompact fill:#2196F3,color:#fff
    style UseStream fill:#FF9800,color:#fff
    style Success fill:#8BC34A,color:#fff

Next Steps

Integration Flows: See 34 complete examples of SDK operations
Usage Guide: API reference for all SDK methods
Security Best Practices: Credential management and secure integration