rspade_system/app/RSpade/tests/CLAUDE.md

RSpade Testing Framework - AI Agent Instructions

PURPOSE: Comprehensive guidelines for AI agents working with RSpade framework tests.

TESTING PHILOSOPHY

The "Press Button, Light Turns Green" Philosophy

Analogy: You have a button that should turn a light green when pressed. If the button doesn't turn the light green, you DO NOT:

• Paint the light green with a brush
• Manually flip the light switch
• Bypass the button with a workaround
• Mark the test as passing

Instead, you FIX THE BUTTON so pressing it turns the light green.

Application to Testing:

• Tests must be fully automated - one command to run, no manual intervention
• Tests must be deterministic - same input always produces same output
• Tests must be repeatable - can run multiple times with identical results
• Tests must require zero human assistance to complete

Manual intervention to help a test complete is FORBIDDEN.

If you find yourself:

• Running commands manually to fix test database state
• Switching to production database because test database has issues
• Modifying files between test runs to make tests pass
• Skipping test assertions because they're "too hard to fix"

STOP IMMEDIATELY and report the issue to the user. These are symptoms that the test infrastructure is broken and needs fixing, NOT that you should work around the problem.

CRITICAL TESTING MANDATES

ABSOLUTE REQUIREMENTS - NO EXCEPTIONS:

1. NEVER Switch to Production Database

Tests MUST pass on the test database (rspade_test).

Why: Switching databases to avoid errors completely defeats the purpose of testing. The test database IS the controlled environment where tests should work. If they don't work there, there's a real problem to fix.

Forbidden:

# ❌ WRONG - Bypassing test database
mysql -urspade -prspadepass rspade -e "SELECT * FROM users"  # Using production DB

# ❌ WRONG - Modifying test to use production
DB_DATABASE=rspade ./run_test.sh

Correct:

# ✅ CORRECT - Tests use test database exclusively
test_db_query "SELECT * FROM users"  # Helper function for test DB only

2. NEVER Mark a Test as Passing If It's Failing

Do not write code to circumvent test failures or skip over difficult errors.

Why: Failing tests indicate real problems that must be fixed. A bypassed test is worse than no test - it creates false confidence.

Forbidden:

# ❌ WRONG - Hiding failures
if ! some_test_command; then
    echo "PASS: Test (ignored failure)"  # Lying about pass
    exit 0
fi

# ❌ WRONG - Skipping hard assertions
# if [ "$count" -ne 5 ]; then
#     echo "FAIL"  # Commented out because "too hard to fix"
# fi

Correct:

# ✅ CORRECT - Fail loud and clear
if [ "$count" -ne 5 ]; then
    echo "FAIL: Expected 5 records, found $count"
    exit 1
fi

3. When a Test Fails: Fix the Error OR Ask the User

Priority order:

1. Analyze the error - Understand what's actually wrong
2. Attempt to fix the underlying issue - Fix the button, not paint the light
3. If unclear or complex, STOP and ask user for direction - Better to ask than guess
4. NEVER work around the failure - Workarounds defeat the purpose

Why: Unexpected test errors are valuable - they reveal real issues. Simply skipping over them completely defeats the point of doing tests. The entire purpose of testing is to uncover unforeseen issues.

Example - Wrong Approach:

# Test fails because migrations didn't run
# ❌ WRONG - Manually run migrations
php artisan migrate --force
# ❌ WRONG - Skip the test
exit 0
# ❌ WRONG - Switch to production database where migrations exist

Example - Correct Approach:

# Test fails because migrations didn't run
# ✅ CORRECT - Fix db_reset.sh to run migrations automatically
# Update the db_reset.sh script to include migration step
# Now test runs migrations automatically and passes

When to ask user:

• Error is cryptic or unclear
• Multiple potential solutions exist
• Fix would require architectural changes
• Uncertain if behavior is intentional or a bug

When NOT to ask user (fix it yourself):

• Simple typos or syntax errors
• Missing files that should obviously exist
• Clear logic errors in test code
• Standard debugging (add logging, check return values, etc.)

TEST STRUCTURE

Test Location

All framework tests: /system/app/RSpade/tests/

/system/app/RSpade/tests/
├── CLAUDE.md                       # This file - AI agent instructions
├── README.md                       # Human-readable test documentation
├── _lib/                           # Test utilities and helpers
│   ├── test_env.sh                 # Test mode management functions
│   ├── rsx_test_config.php         # Manifest config for test mode
│   ├── db_reset.sh                 # Database reset utility
│   ├── db_snapshot_create.sh       # Create database snapshot
│   └── db_snapshot_restore.sh      # Restore database snapshot
├── services/                       # Test service classes
│   ├── service_test_service.php    # Example test service
│   └── scheduled_test_service.php  # Scheduled task tests
├── basic/                          # Basic framework tests
│   ├── 01_framework_verification/
│   │   ├── run_test.sh             # Test script
│   │   └── README.md               # Test documentation
│   └── 02_database_connection/
│       ├── run_test.sh
│       └── README.md
└── tasks/                          # Task system tests
    └── 01_task_dispatch_and_execution/
        ├── run_test.sh
        └── README.md

Test Script Pattern

Every test follows this pattern:

#!/bin/bash
set -e  # Exit on any error

TEST_NAME="Descriptive Test Name"
TEST_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

# Source test environment helpers
source "$TEST_DIR/../../_lib/test_env.sh"

# Parse arguments (for test composition)
SKIP_RESET=false
for arg in "$@"; do
    case $arg in
        --skip-reset)
            SKIP_RESET=true
            shift
            ;;
    esac
done

# Ensure test mode exits on script exit (success or failure)
trap test_trap_exit EXIT

# SETUP
echo "[SETUP] Preparing test..." >&2

# Enter test mode (switches to rspade_test database)
test_mode_enter

# Reset database to known state (unless called as sub-test)
if [ "$SKIP_RESET" = false ]; then
    echo "[SETUP] Resetting test database..." >&2
    "$TEST_DIR/../../_lib/db_reset.sh"
fi

# TEST LOGIC
echo "[TEST] Testing feature..." >&2

# Test assertions here
# Use test_db_query, test_db_count helpers
# Fail loud with clear error messages

# All tests passed
echo "PASS: $TEST_NAME"
exit 0

# TEARDOWN happens automatically via trap

Key elements:

• set -e - Any command failure fails the entire test
• trap test_trap_exit EXIT - Ensures cleanup even on failure
• test_mode_enter - Switches to test database
• db_reset.sh - Ensures known database state
• --skip-reset - For test composition (calling tests from other tests)
• Clear [SETUP] and [TEST] prefixes in output
• echo ... >&2 - Test output goes to stderr, only final result to stdout

TEST ENVIRONMENT MANAGEMENT

Test Mode Functions

Source the helpers (required in every test script):

source /system/app/RSpade/tests/_lib/test_env.sh

Available functions:

test_mode_enter()

Switches environment to test mode.

What it does:

1. Checks if already in test mode (fails if DB_DATABASE=rspade_test)
2. Backs up current .env to /var/www/html/system/.env.testbackup
3. Modifies .env to set DB_DATABASE=rspade_test
4. Sets RSX_ADDITIONAL_CONFIG=/var/www/html/system/app/RSpade/tests/_lib/rsx_test_config.php
5. Clears Laravel config cache
6. Rebuilds manifest to include test directory

Safety: Refuses to create backup if already in test mode (prevents backing up test config as production).

test_mode_exit()

Restores production environment.

What it does:

1. Restores original .env from backup
2. Clears Laravel config cache
3. Rebuilds manifest without test directory

Note: Automatically called by test_trap_exit trap handler.

test_trap_exit()

Cleanup handler for test scripts.

Usage:

trap test_trap_exit EXIT

What it does:

• Calls test_mode_exit() to restore environment
• Does NOT delete backup file (contains production credentials)

test_db_query(sql)

Execute SQL query on test database, return single value.

Usage:

name=$(test_db_query "SELECT name FROM users WHERE id=1")
status=$(test_db_query "SELECT status FROM _tasks WHERE id=$task_id")

Returns: Single value from first column of first row (or empty string if no results).

test_db_count(table [WHERE clause])

Count rows in test database table.

Usage:

count=$(test_db_count "users")
count=$(test_db_count "users WHERE active=1")
count=$(test_db_count "_tasks WHERE status='pending'")

Returns: Integer count of matching rows.

Test Database Isolation

CRITICAL: Tests NEVER touch the production database (rspade). They use a completely separate database (rspade_test).

Environment Variables During Test Mode:

• DB_DATABASE=rspade_test - Points to test database
• RSX_ADDITIONAL_CONFIG=.../rsx_test_config.php - Loads test manifest config

Test Manifest Config (/system/app/RSpade/tests/_lib/rsx_test_config.php):

return [
    'manifest' => [
        'scan_directories' => [
            'app/RSpade/tests',  // Include framework tests in manifest
        ],
    ],
];

This allows test services, tasks, and other test code to be discovered by the manifest system only when in test mode, without polluting the production manifest.

Database Reset Process

Script: /system/app/RSpade/tests/_lib/db_reset.sh

What it does:

1. Safety checks - Verifies we're in test mode, fails loud if not
2. Drop/recreate - DROP DATABASE rspade_test; CREATE DATABASE rspade_test;
3. Restore snapshot - If snapshot exists, restore it (fast)
4. Run migrations - php artisan migrate --production (runs any new migrations not in snapshot)

Safety checks:

# Ensures DB_DATABASE=rspade_test (fails if not)
if ! grep -q "^DB_DATABASE=rspade_test" /var/www/html/.env; then
    echo "FATAL ERROR: db_reset.sh must be called from within test mode!"
    exit 1
fi

# Ensures test config is loaded (fails if not)
if ! grep -q "^RSX_ADDITIONAL_CONFIG=.*rsx_test_config.php" /var/www/html/.env; then
    echo "FATAL ERROR: Test config not loaded!"
    exit 1
fi

When to call:

• At the beginning of every test (unless using --skip-reset for test composition)
• Ensures known database state for deterministic tests

When NOT to call manually:

• Tests call it automatically - no manual intervention needed

TEST COMPOSITION

The --skip-reset Flag

Purpose: Allows tests to call other tests without resetting database state between them.

Use case: Building complex state by chaining simpler tests.

Example:

# Test A creates a user
./tests/users/create_user.sh

# Test B needs a user to exist, calls Test A with --skip-reset
./tests/users/create_user.sh --skip-reset
# Now test database has a user from Test A
# Test B can proceed with its logic without losing that state

Pattern:

# In your test script
SKIP_RESET=false
for arg in "$@"; do
    case $arg in
        --skip-reset)
            SKIP_RESET=true
            shift
            ;;
    esac
done

# Later in setup
if [ "$SKIP_RESET" = false ]; then
    "$TEST_DIR/../../_lib/db_reset.sh"
fi

When to use:

• Calling prerequisite tests to set up state
• Composing complex tests from simpler building blocks
• Chaining tests that depend on each other's state

When NOT to use:

• Running tests independently (always reset for determinism)
• Tests should be runnable in isolation without --skip-reset by default

WRITING NEW TESTS

Step 1: Create Test Directory

mkdir -p /system/app/RSpade/tests/category/NN_test_name
cd /system/app/RSpade/tests/category/NN_test_name

Naming:

• category/ - Group of related tests (basic, tasks, database, etc.)
• NN_ - Two-digit prefix for ordering (01_, 02_, etc.)
• test_name - Descriptive name with underscores

Step 2: Create run_test.sh

#!/bin/bash
set -e

TEST_NAME="Descriptive Test Name"
TEST_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

source "$TEST_DIR/../../_lib/test_env.sh"

# --skip-reset support
SKIP_RESET=false
for arg in "$@"; do
    case $arg in
        --skip-reset) SKIP_RESET=true; shift ;;
    esac
done

trap test_trap_exit EXIT

# SETUP
echo "[SETUP] Preparing test..." >&2
test_mode_enter

if [ "$SKIP_RESET" = false ]; then
    echo "[SETUP] Resetting test database..." >&2
    "$TEST_DIR/../../_lib/db_reset.sh"
fi

# TEST LOGIC
echo "[TEST] Testing feature..." >&2

# Your test code here
# Use test_db_query and test_db_count helpers

# Example assertion
result=$(test_db_query "SELECT COUNT(*) FROM users")
if [ "$result" != "0" ]; then
    echo "FAIL: $TEST_NAME - Expected 0 users, found $result"
    exit 1
fi

# All tests passed
echo "PASS: $TEST_NAME"
exit 0

Make it executable:

chmod +x run_test.sh

Step 3: Create README.md

Document what the test verifies, prerequisites, expected output, etc. See existing test READMEs for examples.

Step 4: Test Your Test

Critical: Run your test multiple times to verify it's deterministic and repeatable.

# Run once
./run_test.sh

# Run again immediately - should pass again
./run_test.sh

# Run a third time - still should pass
./run_test.sh

If it fails on subsequent runs:

• Test is not properly resetting state
• Test is leaving artifacts in database
• Test has race conditions or timing issues
• FIX THE TEST - don't work around it

Writing Good Test Assertions

Fail loud with clear error messages:

# ❌ BAD - Cryptic failure
if [ "$count" != "5" ]; then
    exit 1
fi

# ✅ GOOD - Clear failure message
if [ "$count" != "5" ]; then
    echo "FAIL: $TEST_NAME - Expected 5 records, found $count"
    exit 1
fi

Test one thing per assertion:

# ✅ GOOD - Separate assertions
if [ "$status" != "completed" ]; then
    echo "FAIL: $TEST_NAME - Expected status 'completed', got '$status'"
    exit 1
fi

if [ -z "$result" ]; then
    echo "FAIL: $TEST_NAME - Expected result, got empty string"
    exit 1
fi

Use helper functions:

# ✅ GOOD - Using test_db_count helper
count=$(test_db_count "_tasks WHERE status='pending'")
if [ "$count" -ne 1 ]; then
    echo "FAIL: $TEST_NAME - Expected 1 pending task, found $count"
    exit 1
fi

DEBUGGING FAILED TESTS

Step 1: Read the Error Message

Failed tests should output clear error messages. Read them carefully.

[TEST] ✓ Task dispatched with ID: 1
FAIL: Task Dispatch and Execution - Expected status 'completed', got 'pending'

Analysis: Task was dispatched but never completed. Issue is in task processing, not dispatch.

Step 2: Add Debugging Output

Add temporary debugging to understand what's happening:

# Add debugging
echo "[DEBUG] Task ID: $task_id" >&2
echo "[DEBUG] Status: $status" >&2
echo "[DEBUG] Result: $result" >&2

# Run test
./run_test.sh

Step 3: Query Database Directly

Use test helpers to inspect database state:

# In your test script
test_db_query "SELECT * FROM _tasks WHERE id=$task_id" >&2

Step 4: Verify Test Database State

Check that test database is properly reset:

source _lib/test_env.sh
test_mode_enter
test_db_query "SHOW TABLES"
test_mode_exit

Step 5: Fix Root Cause

DO NOT:

• Skip the failing assertion
• Work around the failure with try/catch or || true
• Switch to production database
• Modify the test to expect wrong behavior

DO:

• Fix the underlying code being tested
• Fix the test infrastructure (db_reset, test_env, etc.)
• Ask user for guidance if unclear

COMMON MISTAKES

Mistake 1: Manual Database Setup

Wrong:

# Running commands manually before test
mysql -urspade -prspadepass rspade_test -e "INSERT INTO users ..."
./run_test.sh

Correct:

# Test sets up its own state automatically
./run_test.sh
# Test calls db_reset.sh internally, sets up state, runs assertions

Mistake 2: Using Production Database

Wrong:

# Test fails on rspade_test, so switch to rspade
sed -i 's/rspade_test/rspade/' .env
./run_test.sh

Correct:

# Fix the underlying issue so test passes on rspade_test
# Maybe db_reset.sh isn't running migrations?
# Maybe test service isn't in manifest?
# FIX THE PROBLEM, don't bypass it

Mistake 3: Hiding Failures

Wrong:

# Test assertion fails, so comment it out
# if [ "$count" -ne 5 ]; then
#     echo "FAIL"
#     exit 1
# fi

Correct:

# Fix why count isn't 5
# Add debugging to understand what's happening
echo "[DEBUG] Count: $count" >&2
test_db_query "SELECT * FROM users" >&2
# Then fix the underlying issue

Mistake 4: Non-Deterministic Tests

Wrong:

# Test depends on timestamp or random values
expected=$(date +%s)
actual=$(test_db_query "SELECT created_at FROM users WHERE id=1")
if [ "$actual" != "$expected" ]; then
    echo "FAIL"
fi

Correct:

# Test for existence, not exact timestamp
created_at=$(test_db_query "SELECT created_at FROM users WHERE id=1")
if [ -z "$created_at" ]; then
    echo "FAIL: $TEST_NAME - created_at is empty"
    exit 1
fi
echo "[TEST] ✓ created_at is set" >&2

AI AGENT RESPONSIBILITIES

When Writing Tests

1. Follow the standard pattern - Use existing tests as templates
2. Make tests deterministic - Same input = same output, always
3. Make tests repeatable - Run 3+ times to verify
4. Fail loud with clear messages - Future debugging will thank you
5. Document what's being tested - README.md for each test

When Debugging Failed Tests

1. Read the error message carefully - It's telling you something
2. Analyze the root cause - Don't jump to solutions
3. Fix the underlying issue - Not the test, the actual problem
4. If unclear, ask the user - Better to ask than guess wrong
5. Never work around failures - Defeats the entire purpose

When NOT to Create Tests

Tests should verify framework functionality, not individual features of user applications.

Create tests for:

• Core framework systems (manifest, bundles, routing, auth)
• Framework utilities (task system, file uploads, caching)
• Critical infrastructure (database connections, migrations)

Do NOT create tests for:

• User application features (unless explicitly requested)
• One-off debugging (use artisan commands instead)
• Features that aren't production-ready yet

Red Flags - Stop and Ask User

If you encounter any of these, STOP and ask user for guidance:

1. Test fails only on subsequent runs - State isn't properly reset
2. Test requires manual setup steps - Test infrastructure is broken
3. Test passes but output shows errors - Something is being silently caught
4. Can't reproduce failure - Race condition or non-deterministic test
5. Unclear what test should verify - Specification ambiguity

SUMMARY

Remember these core principles:

1. Tests are fully automated - One command, no manual steps
2. Tests are deterministic - Same input = same output, always
3. Tests are repeatable - Can run 100 times with identical results
4. Tests require zero human assistance - Complete autonomously
5. Test failures are valuable - They reveal real problems
6. Never work around test failures - Fix the root cause or ask user
7. Manual intervention is forbidden - Fix the infrastructure instead

The "Press Button, Light Turns Green" Philosophy: If the button doesn't turn the light green, fix the button. Don't paint the light green and declare success.

When in doubt, ask the user. It's better to raise a concern than silently compromise test integrity.