rspade_system/app/RSpade/man/tasks.txt

RSX TASK SYSTEM
================

NAME
    Tasks - Unified background task execution with scheduling and queueing

SYNOPSIS
    Immediate CLI Execution
    -----------------------
    #[Task('Generate report')]
    public static function generate_report(Task_Instance $task, array $params = [])
    {
        return ['status' => 'complete'];
    }

    Run via: php artisan rsx:task:run Service_Name generate_report --param=value

    Scheduled Tasks (Recurring)
    ---------------------------
    #[Task('Clean thumbnails daily')]
    #[Schedule('0 3 * * *')]  // 3am daily, cron syntax
    public static function clean_thumbnails(Task_Instance $task, array $params = [])
    {
        $task->log("Starting cleanup...");
        // ... cleanup logic
    }

    Queued Tasks (Async from Application)
    --------------------------------------
    #[Task('Transcode video', queue: 'video', timeout: 3600)]
    public static function transcode(Task_Instance $task, array $params = [])
    {
        $task->set_status('progress', 0);
        $temp_dir = $task->get_temp_directory();

        // ... transcoding logic

        $task->set_status('progress', 100);
        return ['output_path' => '/storage/tasks/123/output.mp4'];
    }

    // Dispatch from controller:
    $task_id = Task::dispatch('Video_Service', 'transcode', ['video_id' => 123]);

    // Poll for status:
    $status = Task::status($task_id);

DESCRIPTION
    RSX provides a unified task execution system with three execution modes:

    1. Immediate CLI - Run tasks directly via artisan command
    2. Scheduled - Recurring tasks run automatically on cron schedule
    3. Queued - Async tasks dispatched from application code with status tracking

    Philosophy: Visual Basic simplicity - define tasks with attributes, the
    framework handles workers, scheduling, queueing, and cleanup automatically.

    All tasks are Service methods with #[Task] attribute. Execution mode is
    determined by additional attributes (#[Schedule]) or invocation method
    (Task::dispatch()).

TASK EXECUTION MODES

    Mode 1: Immediate CLI (Interactive)
    ------------------------------------
    Run task immediately from command line. No database tracking, no timeout,
    output printed to console.

    Usage:
        php artisan rsx:task:run Service_Name task_name --param=value

    Characteristics:
    - Synchronous execution
    - No timeout enforcement
    - No database tracking
    - Output to STDOUT
    - $task parameter provided but minimal functionality

    Mode 2: Scheduled (Recurring)
    ------------------------------
    Tasks run automatically on a cron schedule. Tracked in database, debounced
    (no parallel execution), run as soon as possible after scheduled time.

    Define with #[Schedule] attribute using cron syntax:
        #[Schedule('0 3 * * *')]     // Daily at 3am
        #[Schedule('*/15 * * * *')]  // Every 15 minutes
        #[Schedule('0 */6 * * *')]   // Every 6 hours
        #[Schedule('0 2 * * 1')]     // Mondays at 2am

    Characteristics:
    - Automatic execution when scheduled time reached
    - Database tracking (next_run_at, started_at, completed_at)
    - Debounced - if already running, skip until next schedule
    - If execution missed (server down), runs as soon as possible
    - Default queue: 'scheduled' with max_workers: 1
    - Optional custom queue via queue parameter in #[Task]

    Mode 3: Queued (Async Application)
    -----------------------------------
    Tasks dispatched from application code, run asynchronously with full
    status tracking and progress updates.

    Dispatch from code:
        $task_id = Task::dispatch('Service_Name', 'task_name', $params);

    Poll for status:
        $status = Task::status($task_id);
        // Returns: status, progress, log, result

    Characteristics:
    - Asynchronous execution
    - Full database tracking
    - Progress and status updates
    - Timeout enforcement (default 30 minutes)
    - Temporary file support with auto-cleanup
    - Queue management with configurable concurrency

CREATING TASKS

    Task Services
    -------------
    Tasks are static methods in Service classes extending Rsx_Service_Abstract.

    Location: /rsx/services/

    Basic structure:
        <?php
        namespace Rsx\Services;

        use App\RSpade\Core\Service\Rsx_Service_Abstract;
        use App\RSpade\Core\Task\Task_Instance;

        class My_Service extends Rsx_Service_Abstract
        {
            #[Task('Description of task')]
            public static function my_task(Task_Instance $task, array $params = [])
            {
                // Task implementation
                return $result;
            }
        }

    Task Method Requirements
    ------------------------
    - Must be public static methods
    - Must have #[Task('description')] attribute
    - Must accept (Task_Instance $task, array $params = [])
    - Should return data (stored as JSON in database for queued/scheduled)

    Task Attribute
    --------------
    #[Task('Description', queue: 'name', timeout: seconds)]

    Parameters:
    - description (required) - Human-readable task description
    - queue (optional) - Queue name for async execution (default: 'default')
    - timeout (optional) - Timeout in seconds (default: 1800 = 30 min)

    Queue must be defined in config/rsx.php or error thrown.
    Timeout only enforced for queued/scheduled tasks, not immediate CLI.

    Schedule Attribute
    ------------------
    #[Schedule('cron_expression')]

    Cron syntax (5 fields):
        * * * * *
        | | | | |
        | | | | +-- Day of week (0-7, 0=Sunday, 7=Sunday)
        | | | +---- Month (1-12)
        | | +------ Day of month (1-31)
        | +-------- Hour (0-23)
        +---------- Minute (0-59)

    Examples:
        '0 3 * * *'      - Daily at 3am
        '*/15 * * * *'   - Every 15 minutes
        '0 */6 * * *'    - Every 6 hours
        '0 9 * * 1-5'    - Weekdays at 9am
        '30 2 1 * *'     - First day of month at 2:30am

    Tasks with #[Schedule] are automatically registered and run by task processor.

TASK INSTANCE API

    The Task_Instance object provides methods for interacting with the task
    execution context. Available in all execution modes.

    Logging
    -------
    $task->log($message)

    Appends text to task status log. Each call adds timestamped entry.
    Stored in database for queued/scheduled tasks.

    Example:
        $task->log("Starting video processing...");
        $task->log("Frame 100/500 processed");

    Status Updates (Key-Value Pairs)
    ---------------------------------
    $task->set_status($key, $value)

    Sets arbitrary status data. Stored as JSON in database.
    Use for progress tracking, time estimates, current step, etc.

    Example:
        $task->set_status('progress', 25);
        $task->set_status('time_remaining', 120);
        $task->set_status('current_step', 'encoding');
        $task->set_status('frames_processed', 100);

    Client can read via Task::status($task_id)->get('progress')

    Temporary Directory
    -------------------
    $temp_dir = $task->get_temp_directory()

    Returns: storage/tasks/{task_id}/

    Creates temporary directory for task file operations. Automatically
    cleaned up 1 hour after task completion (configurable).

    Example:
        $temp = $task->get_temp_directory();
        $output = "{$temp}/output.mp4";
        FFmpeg::transcode($input, $output);

    Set Custom Cleanup Time
    -----------------------
    $task->set_temp_expiration($seconds)

    Override default 1 hour cleanup. Call after get_temp_directory().

    Example:
        $temp = $task->get_temp_directory();
        $task->set_temp_expiration(7200);  // Keep for 2 hours

DISPATCHING TASKS

    Dispatch from Application Code
    -------------------------------
    $task_id = Task::dispatch($service, $task, $params);

    Parameters:
    - $service - Service class name (e.g., 'Video_Service')
    - $task - Task method name (e.g., 'transcode')
    - $params - Associative array of parameters

    Returns: Task ID (integer) for status polling

    Example:
        $task_id = Task::dispatch('Video_Service', 'transcode', [
            'video_id' => 123,
            'quality' => 'high'
        ]);

    Auto-spawns worker if needed based on queue concurrency settings.

    Check Task Status
    -----------------
    $status = Task::status($task_id);

    Returns Task_Status object with:
    - $status->status - 'pending', 'running', 'completed', 'failed', 'stuck'
    - $status->get($key, $default) - Get status value
    - $status->log - Full text log
    - $status->result - Task return value (when completed)
    - $status->started_at - Timestamp
    - $status->completed_at - Timestamp

    Example:
        $status = Task::status($task_id);
        if ($status->status === 'completed') {
            $output = $status->result['output_path'];
        } else {
            $progress = $status->get('progress', 0);
        }

QUEUE CONFIGURATION

    Define queues in config/rsx.php:

    'tasks' => [
        'global_max_workers' => 1,
        'default_timeout' => 1800,  // 30 minutes
        'cleanup_stuck_after' => 1800,
        'temp_directory_default_ttl' => 3600,  // 1 hour

        'queues' => [
            'default' => ['max_workers' => 1],
            'scheduled' => ['max_workers' => 1],
            'video' => ['max_workers' => 2],
            'export' => ['max_workers' => 1],
            'email' => ['max_workers' => 5],
        ],
    ],

    Queue Settings
    --------------
    global_max_workers - Total concurrent workers across ALL queues
    default_timeout - Default task timeout in seconds
    cleanup_stuck_after - Consider task stuck after this many seconds
    temp_directory_default_ttl - Auto-delete temp files after this long

    Per-Queue Settings
    ------------------
    max_workers - Maximum concurrent workers for this queue

    Worker spawning logic:
    1. When task dispatched, check running workers for that queue
    2. If running < max_workers AND global limit not reached, spawn worker
    3. Worker processes queue FIFO until empty, then exits

    Undefined Queue Error
    ---------------------
    Using queue name not defined in config throws error at dispatch time.

TASK PROCESSOR COMMAND

    Start Task Processor
    --------------------
    php artisan rsx:task:process [--queue=name]

    This command:
    1. Cleans up stuck tasks (check PIDs, mark as stuck)
    2. Processes scheduled tasks that are due
    3. Checks all queues for pending tasks
    4. Spawns workers up to max_workers per queue
    5. Becomes a worker itself if under quota
    6. Processes tasks until queue empty, then exits

    Options:
    --queue=name  Process only specific queue

    Running via Cron
    ----------------
    Add to system crontab to run every minute:

        * * * * * cd /var/www/html && php artisan rsx:task:process

    This single cron entry handles:
    - All scheduled tasks
    - All queued tasks
    - Worker management
    - Stuck task cleanup

TASK DATABASE SCHEMA

    Table: _tasks
    -------------
    - id - Task instance ID
    - server_id - Server identifier (default 1, for future distributed tasks)
    - service_name - Service class name
    - task_name - Task method name
    - queue_name - Queue name (NULL for scheduled tasks)
    - execution_mode - 'scheduled' or 'queued'
    - status - 'pending', 'running', 'completed', 'failed', 'stuck'
    - params - JSON parameters
    - result - JSON task return value
    - status_data - JSON status key-value pairs
    - status_log - Text log (append-only)
    - scheduled_at - When task should run (scheduled tasks)
    - next_run_at - Next execution time (recurring scheduled)
    - started_at - When task started processing
    - completed_at - When task finished
    - timeout_seconds - Task timeout
    - worker_pid - Process ID of worker
    - has_temp_directory - Boolean flag
    - temp_expires_at - When to cleanup temp files
    - created_at, updated_at

    Advisory Locking
    ----------------
    Database advisory locks coordinate queue access:
    - Lock key: task_queue_{queue_name}
    - Prevents race conditions in worker spawning
    - Atomic task selection and status updates

TASK LIFECYCLE

    Scheduled Task Lifecycle
    ------------------------
    1. Task registered with #[Schedule] attribute
    2. Entry created in _tasks with next_run_at
    3. Task processor checks scheduled tasks every minute
    4. When next_run_at <= now, task status set to 'pending'
    5. Worker picks up task, sets status to 'running', records worker_pid
    6. Task executes, updates logged and status tracked
    7. On completion, status set to 'completed', result stored
    8. next_run_at calculated for next execution based on cron

    Queued Task Lifecycle
    ---------------------
    1. Task::dispatch() called from application code
    2. Entry created in _tasks with status 'pending'
    3. Worker spawned if needed based on concurrency settings
    4. Worker picks up task atomically via advisory lock
    5. Status set to 'running', worker_pid recorded
    6. Task executes with progress updates via $task API
    7. On completion, status set to 'completed', result stored
    8. Temp directory marked for cleanup if created

    Stuck Task Detection
    --------------------
    Task considered stuck if:
    - Status is 'running'
    - started_at + timeout_seconds < now

    Task processor handles stuck tasks:
    1. Check if worker_pid still running
    2. If PID dead, mark task 'stuck'
    3. If PID alive but timeout exceeded, kill PID and mark 'stuck'
    4. Log stuck task for manual review

EXAMPLES

    Example 1: Scheduled Cleanup Task
    ----------------------------------
    <?php
    namespace Rsx\Services;

    use App\RSpade\Core\Service\Rsx_Service_Abstract;
    use App\RSpade\Core\Task\Task_Instance;

    class Cleanup_Service extends Rsx_Service_Abstract
    {
        #[Task('Clean old thumbnails daily at 3am')]
        #[Schedule('0 3 * * *')]
        public static function clean_thumbnails(Task_Instance $task, array $params = [])
        {
            $task->log("Starting thumbnail cleanup");

            $deleted = 0;
            // ... cleanup logic

            $task->log("Cleanup complete. Deleted {$deleted} files.");
            return ['deleted_count' => $deleted];
        }
    }

    Example 2: Video Transcoding (Queued)
    --------------------------------------
    <?php
    namespace Rsx\Services;

    use App\RSpade\Core\Service\Rsx_Service_Abstract;
    use App\RSpade\Core\Task\Task_Instance;

    class Video_Service extends Rsx_Service_Abstract
    {
        #[Task('Transcode uploaded video', queue: 'video', timeout: 7200)]
        public static function transcode(Task_Instance $task, array $params = [])
        {
            $video_id = $params['video_id'];
            $video = Video_Model::find($video_id);

            $task->log("Loading video {$video_id}");
            $task->set_status('progress', 0);

            $temp = $task->get_temp_directory();
            $task->set_temp_expiration(7200);  // Keep for 2 hours

            $input = $video->get_storage_path();
            $output = "{$temp}/output.mp4";

            FFmpeg::transcode($input, $output, function($progress) use ($task) {
                $task->set_status('progress', $progress);
                $task->log("Progress: {$progress}%");
            });

            $task->set_status('progress', 100);
            $task->log("Transcoding complete");

            $attachment = File_Attachment_Model::create_from_path($output);

            return [
                'attachment_id' => $attachment->id,
                'output_path' => $output,
            ];
        }
    }

    // Controller dispatch:
    #[Ajax_Endpoint]
    public static function upload_video(Request $request, array $params = [])
    {
        $video = Video_Model::create([...]);

        $task_id = Task::dispatch('Video_Service', 'transcode', [
            'video_id' => $video->id
        ]);

        return ['video_id' => $video->id, 'task_id' => $task_id];
    }

    // Status polling:
    #[Ajax_Endpoint]
    public static function check_transcode(Request $request, array $params = [])
    {
        $status = Task::status($params['task_id']);

        return [
            'status' => $status->status,
            'progress' => $status->get('progress', 0),
            'log' => $status->log,
            'result' => $status->result,
        ];
    }

    Example 3: Export Generation (Queued)
    --------------------------------------
    #[Task('Generate CSV export', queue: 'export', timeout: 1800)]
    public static function generate_export(Task_Instance $task, array $params = [])
    {
        $task->log("Starting export generation");

        $temp = $task->get_temp_directory();
        $csv_path = "{$temp}/export.csv";

        $total = Client_Model::count();
        $processed = 0;

        $fp = fopen($csv_path, 'w');
        fputcsv($fp, ['ID', 'Name', 'Email', 'Created']);

        Client_Model::chunk(100, function($clients) use ($fp, &$processed, $total, $task) {
            foreach ($clients as $client) {
                fputcsv($fp, [
                    $client->id,
                    $client->name,
                    $client->email,
                    $client->created_at,
                ]);
                $processed++;
            }

            $progress = round(($processed / $total) * 100);
            $task->set_status('progress', $progress);
            $task->log("Processed {$processed}/{$total} clients");
        });

        fclose($fp);

        $attachment = File_Attachment_Model::create_from_path($csv_path);

        return ['attachment_id' => $attachment->id];
    }

BUILT-IN SCHEDULED TASKS

    The framework includes built-in scheduled tasks in app/RSpade/Core/Tasks/:

    Cleanup_Service::cleanup_old_tasks
    -----------------------------------
    Removes completed/failed tasks older than configured retention period.
    Runs daily at 2am.

    Cleanup_Service::cleanup_temp_directories
    ------------------------------------------
    Deletes temporary task directories past their expiration time.
    Runs hourly.

    These tasks are automatically registered and require no user configuration.

TROUBLESHOOTING

    Task Not Running
    ----------------
    Problem: Scheduled task not executing

    Checks:
    - Is cron configured? (* * * * * php artisan rsx:task:process)
    - Check _tasks table for next_run_at timestamp
    - Run manually: php artisan rsx:task:process
    - Check Laravel logs for errors

    Worker Not Spawning
    --------------------
    Problem: Dispatched task stays 'pending'

    Checks:
    - Is global_max_workers reached?
    - Is queue max_workers reached?
    - Is task processor running? (cron or manual)
    - Check _tasks table for worker_pid
    - Try: php artisan rsx:task:process --queue=queuename

    Task Stuck
    ----------
    Problem: Task shows 'stuck' status

    Causes:
    - Worker process killed/crashed
    - Timeout exceeded
    - Server restarted

    Recovery:
    - Task processor automatically detects stuck tasks
    - Stuck tasks marked for manual review
    - Check status_log for clues
    - Re-dispatch if needed

    Undefined Queue Error
    ---------------------
    Problem: Error when dispatching task

    Solution:
    Define queue in config/rsx.php:
        'queues' => [
            'myqueue' => ['max_workers' => 1],
        ],

BEST PRACTICES

    1. Choose Right Execution Mode
       - Immediate CLI: Development, testing, one-off operations
       - Scheduled: Recurring maintenance, cleanup, reports
       - Queued: User-triggered async work, long-running processes

    2. Set Appropriate Timeouts
       - Default 30 minutes is generous
       - Video transcoding: 2+ hours
       - Exports: 30 minutes
       - Email sending: 5 minutes

    3. Use Descriptive Logging
       - Log major steps, not every iteration
       - Include useful context (IDs, counts, progress)
       - Helps debugging stuck/failed tasks

    4. Queue Naming
       - Use semantic names: 'video', 'export', 'email'
       - Group similar workloads
       - Set concurrency based on resource usage

    5. Temp Directory Cleanup
       - Only create if needed
       - Set expiration based on use case
       - Return attachment_id instead of path when possible

    6. Status Updates
       - Update progress regularly (every 5-10%)
       - Include time estimates when possible
       - Use consistent key names across tasks

    7. Error Handling
       - Throw exceptions for failures
       - Task status automatically set to 'failed'
       - Exception message stored in result
       - Don't catch and swallow errors

FUTURE FEATURES (NOT YET IMPLEMENTED)

    The task system is designed to support future enhancements:

    - Distributed tasks across multiple servers (server_id support)
    - Metrics and monitoring (task duration, failure rates, queue depth)

SEE ALSO
    service.txt - Service class documentation
    ajax.txt - Ajax endpoint system
    config_rsx.txt - RSX configuration reference