rspade_system/app/RSpade/Core/Files/THUMBNAILS.md

Thumbnail Caching System - Developer Documentation

Overview

The RSX thumbnail caching system provides a two-tier architecture designed to prevent cache pollution while maintaining flexibility. This document explains the implementation details for framework developers.

Architecture

Two-Tier System

Preset Thumbnails (storage/rsx-thumbnails/preset/)

• Developer-defined named sizes in config
• Quota enforced via scheduled task (rsx:thumbnails:clean --preset)
• Used for regular application features
• Protected from abuse/spam

Dynamic Thumbnails (storage/rsx-thumbnails/dynamic/)

• Ad-hoc sizes via URL parameters
• Quota enforced synchronously after each generation
• Used for edge cases and development
• LRU eviction prevents unbounded growth

Design Philosophy

Problem: Traditional on-demand thumbnail generation allows users to request arbitrary sizes (e.g., by manipulating URL parameters), potentially generating thousands of cached thumbnails and consuming disk space.

Solution:

1. Define all application thumbnail sizes as named presets
2. Reference presets by name (get_thumbnail_url_preset('profile'))
3. Dynamic thumbnails still available but discouraged and quota-limited

Benefits:

• Prevents cache pollution from URL manipulation
• Centralizes thumbnail size definitions
• Enables cache warming via rsx:thumbnails:generate
• Predictable cache size and cleanup

Storage Structure

Filename Format

Preset: {preset_name}_{storage_hash}_{extension}.webp

• Example: profile_abc123def456_jpg.webp

Dynamic: {type}_{width}x{height}_{storage_hash}_{extension}.webp

• Example: cover_200x200_abc123def456_jpg.webp

Why Include Extension?

Edge case: Identical file content uploaded with different extensions (e.g., document.zip and document.docx) should render different icon-based thumbnails. Including the extension in the filename ensures separate caches.

Caching Flow

Request Processing

Route: /_thumbnail/dynamic/:key/:type/:width/:height (dynamic) or /_thumbnail/preset/:key/:preset_name (preset)

Common Flow (via generate_and_serve_thumbnail()):

1. Validate authorization (file.thumbnail.authorize event)
2. Generate cache filename
3. Check if cached file exists
4. If exists:
   • Attempt to open (handles race condition)
   • If successful: touch mtime (if enabled), serve from disk
   • If failed: fall through to regeneration
5. If not exists:
   • Generate thumbnail (Imagick for images, icon-based for others)
   • Save to cache
   • Enforce quota (dynamic only)
   • Serve from memory (no re-read)

Race Condition Handling

Scenario: Between file_exists() check and fopen(), file might be deleted by quota enforcement in another request.

Solution: _serve_cached_thumbnail() returns null if file cannot be opened, triggering immediate regeneration. User never sees an error.

if (file_exists($cache_path)) {
    $response = static::_serve_cached_thumbnail($cache_path);
    if ($response !== null) {
        return $response; // Success
    }
    // File deleted - fall through to regeneration
}

// Generate and serve
$thumbnail_data = static::__generate_thumbnail(...);

LRU Tracking

mtime Touching:

• On cache hit, touch file's mtime if older than touch_interval (default 10 minutes)
• Prevents excessive filesystem writes while maintaining LRU accuracy
• Used by quota enforcement to delete oldest (least recently used) files first

Configuration:

'touch_on_read' => true,        // Enable/disable
'touch_interval' => 600,        // Seconds (10 minutes)

Quota Enforcement

Dynamic Thumbnails (Synchronous)

Called after each new dynamic thumbnail generation:

protected static function _enforce_dynamic_quota()
{
    $max_bytes = config('rsx.thumbnails.quotas.dynamic_max_bytes');
    $dir = storage_path('rsx-thumbnails/dynamic/');

    // Scan directory
    $total_size = 0;
    $files = [];
    foreach (glob($dir . '*.webp') as $file) {
        $size = filesize($file);
        $mtime = filemtime($file);
        $total_size += $size;
        $files[] = ['path' => $file, 'size' => $size, 'mtime' => $mtime];
    }

    // Over quota?
    if ($total_size > $max_bytes) {
        // Sort by mtime (oldest first)
        usort($files, fn($a, $b) => $a['mtime'] <=> $b['mtime']);

        // Delete oldest until under quota
        foreach ($files as $file) {
            unlink($file['path']);
            $total_size -= $file['size'];
            if ($total_size <= $max_bytes) break;
        }
    }
}

Performance: Glob + stat operations are fast for directories with <10,000 files. Dynamic quota prevents unbounded growth.

Preset Thumbnails (Scheduled)

Not enforced synchronously. Run via scheduled task:

php artisan rsx:thumbnails:clean --preset

Same LRU algorithm as dynamic, but triggered externally.

Helper Methods

Internal Helpers (Underscore-Prefixed)

These are public static for artisan command access but prefixed with _ to indicate they're internal:

_get_cache_filename_preset($preset_name, $hash, $extension)

• Generates: {preset_name}_{hash}_{ext}.webp

_get_cache_filename_dynamic($type, $width, $height, $hash, $extension)

• Generates: {type}_{w}x{h}_{hash}_{ext}.webp

_get_cache_path($cache_type, $filename)

• Returns: storage_path("rsx-thumbnails/{$cache_type}/{$filename}")

_save_thumbnail_to_cache($cache_path, $thumbnail_data)

• Writes WebP data to cache
• Creates directory if needed

_serve_cached_thumbnail($cache_path)

• Attempts to open file (race condition safe)
• Touches mtime if configured
• Returns Response or null

_enforce_dynamic_quota()

• Scans dynamic directory
• Deletes oldest files until under quota

Why Public?

Artisan commands (Thumbnails_Generate_Command, etc.) need access to these helpers. Making them public with _ prefix indicates "internal use only - don't call from application code."

DRY Implementation

Single Generation Function:

protected static function generate_and_serve_thumbnail(
    $attachment,
    $type,
    $width,
    $height,
    $cache_type,
    $cache_filename
) {
    // Common logic for both preset and dynamic
    // Check cache -> Generate if needed -> Save -> Enforce quota -> Serve
}

Route Methods Are Thin Wrappers:

public static function thumbnail_preset(Request $request, array $params = [])
{
    // Parse preset params
    // Generate cache filename
    // Call common function
    return static::generate_and_serve_thumbnail(...);
}

public static function thumbnail(Request $request, array $params = [])
{
    // Parse dynamic params
    // Generate cache filename
    // Call common function
    return static::generate_and_serve_thumbnail(...);
}

This eliminates duplication while keeping route-specific validation separate.

Artisan Commands

Thumbnails_Clean_Command

Enforces quotas for preset and/or dynamic thumbnails.

Implementation:

• Scans directory, builds array of [path, size, mtime]
• Sorts by mtime ascending (oldest first)
• Deletes files until under quota
• Reports statistics

Flags: --preset, --dynamic, --all (default)

Thumbnails_Generate_Command

Pre-generates preset thumbnails for attachments.

Implementation:

• Queries File_Attachment_Model (optionally filtered by key)
• For each attachment/preset combination:
  • Generate cache filename
  • Skip if already cached
  • Generate thumbnail
  • Save to cache
  • No quota enforcement (manual command)

Use Cases:

• Cache warming after deployment
• Background generation for new uploads
• Regenerating after changing preset sizes

Thumbnails_Stats_Command

Displays cache usage statistics.

Implementation:

• Scans both directories
• Calculates: file count, total size, quota percentage, oldest/newest mtime
• For presets: breaks down by preset name (extracted from filename)

WebP Output

All thumbnails are WebP format for optimal size/quality:

$image->setImageFormat('webp');
$image->setImageCompressionQuality(85);
$thumbnail_data = $image->getImageBlob();

Why WebP?

• Superior compression vs JPEG/PNG
• Supports transparency (for 'fit' thumbnails)
• Universal browser support (96%+ as of 2024)

Non-Image Files

Icon-based thumbnails for PDFs, documents, etc.:

if ($attachment->is_image()) {
    $thumbnail_data = static::__generate_thumbnail(...);
} else {
    $thumbnail_data = File_Attachment_Icons::render_icon_as_thumbnail(
        $attachment->file_extension,
        $width,
        $height
    );
}

Icons are rendered as WebP images for consistency.

Security

Authorization: All thumbnail endpoints check file.thumbnail.authorize event gate. Implement handlers in /rsx/handlers/ to control access.

Dimension Limits: Dynamic thumbnails enforce 10-256 pixel limits to prevent abuse. Preset thumbnails have no limits (developer-controlled).

Extension Validation: File extensions are normalized (lowercase, jpeg→jpg) before cache filename generation.

Configuration

All configuration in /system/config/rsx.php under thumbnails key:

'thumbnails' => [
    'presets' => [
        'profile' => ['type' => 'cover', 'width' => 200, 'height' => 200],
        // ...
    ],
    'quotas' => [
        'preset_max_bytes' => 100 * 1024 * 1024,   // 100MB
        'dynamic_max_bytes' => 50 * 1024 * 1024,   // 50MB
    ],
    'touch_on_read' => env('THUMBNAILS_TOUCH_ON_READ', true),
    'touch_interval' => env('THUMBNAILS_TOUCH_INTERVAL', 600),
],

File_Thumbnail_Model Removal

Previous Design: Database table tracking thumbnails with source/thumbnail storage relationships.

Why Removed:

• Filesystem is sufficient for caching
• mtime provides LRU info
• No need for DB overhead
• Simpler, faster, less to maintain

Migration: 2025_11_16_093331_drop_file_thumbnails_table.php drops the table.

Testing Checklist

• Preset thumbnail generation and caching
• Dynamic thumbnail generation and caching
• Cache hits serve from disk
• Cache misses generate and cache
• Race condition handling (file deleted between check and open)
• Dynamic quota enforcement deletes oldest first
• mtime touching respects config and interval
• Preset/dynamic survive rsx:clean
• Same hash + different extension creates separate thumbnails
• Invalid preset name throws exception
• All generated thumbnails are WebP
• rsx:thumbnails:clean enforces quotas
• rsx:thumbnails:generate pre-generates correctly
• rsx:thumbnails:stats shows accurate data
• Icon-based thumbnails for non-images work
• Browser cache headers correct (1 year)

Future Enhancements

Not in current implementation but planned:

• Animated thumbnails: Extract first frame from GIF/WebP/video
• Video thumbnails: Frame extraction via FFmpeg
• PDF thumbnails: First page via Imagick
• CDN integration: Upload thumbnails to CDN storage
• Per-preset quotas: Different limits for different presets
• Background generation: Queue-based thumbnail generation
• Analytics: DB-backed usage tracking (optional)