rspade_system/app/RSpade/Core/JavaScript

JavaScript Parser - RPC Server Architecture

Problem Statement

RSpade's manifest build system needs to parse ~1,200+ JavaScript files to extract metadata (classes, methods, decorators, etc). The original implementation spawned a new Node.js process for each file:

foreach ($js_files as $file) {
    $process = new Process(['node', 'js-parser.js', $file]);
    $process->run();
    // ... parse output
}

Performance Impact:

• 1,200+ process spawns per clean build
• Each spawn: ~50-150ms overhead
• Total overhead: 60-180 seconds just for process management
• Plus Node.js interpreter startup time per file

Solution: Long-Running RPC Server

Replace N process spawns with ONE long-running Node.js server that handles all parse requests via RPC over a Unix domain socket.

Benefits:

• Single Node.js interpreter startup
• No process spawn overhead per file
• Batch processing support
• ~25-50x speedup for clean builds

Architecture

Components

1. PHP Client (Js_Parser.php)

• Manages RPC server lifecycle
• Provides static methods for server operations
• Integrates with existing parse/cache logic

2. Node.js RPC Server (js-parser-server.js)

• Long-running process listening on Unix socket
• Handles JSON-RPC style requests
• Processes multiple files per request (batch support)

3. Unix Domain Socket

• Path: storage/rsx-tmp/js-parser-server.sock
• Protocol: Line-delimited JSON
• No port conflicts, no network stack overhead

Communication Protocol

Request Format:

{"id": 1, "method": "parse", "files": ["path1.js", "path2.js"]}\n

Response Format:

{"id": 1, "results": {"path1.js": {...}, "path2.js": {...}}}\n

Methods:

• ping → {id: N, result: "pong"} - Health check
• parse → {id: N, results: {file: data, ...}} - Parse files
• shutdown → {id: N, result: "shutting down"} - Graceful stop

Line-delimited JSON allows simple stream parsing without complex framing.

Server Lifecycle

1. Lazy Initialization

Server only starts when first JS file needs parsing (during manifest build Phase 2):

public static function extract_metadata(string $file_path): array
{
    // Start RPC server on first parse (lazy initialization)
    if (static::$rpc_server_process === null) {
        static::start_rpc_server();
    }
    // ... continue parsing
}

2. Startup Procedure

When start_rpc_server() is called:

1. Check for stale server:

   if (file_exists($socket_path)) {
       static::stop_rpc_server(force: true);
   }
   

   Force-stop any existing server to ensure clean slate.

2. Spawn Node process:

   $process = new Process([
       'node',
       base_path('app/RSpade/Core/JavaScript/resource/js-parser-server.js'),
       '--socket=' . $socket_path
   ]);
   $process->start();
   

3. Wait for ready (ping/pong):

   for ($i = 0; $i < 200; $i++) {  // 10 seconds max
       usleep(50000);  // 50ms
       if (static::ping_rpc_server()) {
           // Server ready!
           break;
       }
   }
   

4. Fatal error on timeout:

   if (!$ready) {
       throw new \RuntimeException('Failed to start RPC server');
   }
   

5. Register shutdown handler:

   register_shutdown_function([self::class, 'stop_rpc_server']);
   

3. Normal Operation

During manifest build, all JS parsing goes through the RPC server:

// Future implementation:
$results = static::parse_via_rpc([
    'file1.js',
    'file2.js',
    // ... batch of 50 files
]);

Cache is checked first - only uncached files sent to RPC.

4. Graceful Shutdown

When manifest build completes:

public static function stop_rpc_server(bool $force = false): void
{
    // Send shutdown command
    $sock = stream_socket_client("unix://{$socket_path}");
    fwrite($sock, json_encode(['method' => 'shutdown']) . "\n");
    fclose($sock);

    if ($force) {
        // Wait and force kill if needed
        sleep(1);
        if ($process->isRunning()) {
            $process->stop(3, SIGTERM);
        }
    }
    // Otherwise just send command and return immediately
}

Force Stop Parameter

stop_rpc_server($force = false) has two modes:

Normal Mode ($force = false)

• Send shutdown command
• Return immediately
• Trust server will exit gracefully
• Used during normal shutdown (shutdown handler)

Force Mode ($force = true)

• Send shutdown command
• Wait 1 second
• If still running, send SIGTERM
• Clean up socket file
• Used when detecting stale server at startup

Why two modes?

At startup, we need to ensure old server is gone before starting new one (force mode). During normal shutdown, we don't need to wait - the server will handle shutdown and PHP exit won't block (normal mode).

Node.js Server Implementation

Key Design Decisions

1. Line-Delimited JSON

Simple, no complex framing needed:

socket.on('data', (data) => {
    buffer += data.toString();
    let newlineIndex;
    while ((newlineIndex = buffer.indexOf('\n')) !== -1) {
        const line = buffer.substring(0, newlineIndex);
        buffer = buffer.substring(newlineIndex + 1);
        handleRequest(line);
    }
});

2. Synchronous File Processing

Parse files sequentially in single request handler - simpler than async complexity for this use case.

3. Graceful Shutdown Handling

if (request.method === 'shutdown') {
    socket.end();
    server.close(() => {
        fs.unlinkSync(socketPath);
        process.exit(0);
    });
}

4. Signal Handlers

Ensure socket cleanup on unexpected termination:

process.on('SIGTERM', () => {
    server.close(() => {
        fs.unlinkSync(socketPath);
        process.exit(0);
    });
});

Cache Integration

The RPC server integrates with existing cache system:

public static function parse($file_path)
{
    // Check cache first
    if (file_exists($cache_file)) {
        return json_decode(file_get_contents($cache_file), true);
    }

    // Use RPC if server running, else fallback to single-file
    if (static::$rpc_server_process !== null) {
        return static::parse_via_rpc([$file_path])[$file_path];
    }

    return static::_parse_without_cache($file_path);
}

Batch Optimization (Future):

Collect uncached files and send in batches to RPC:

$uncached_files = array_filter($files, fn($f) => !cache_exists($f));
$results = static::parse_via_rpc($uncached_files);
foreach ($results as $file => $data) {
    cache_result($file, $data);
}

Error Handling

Fatal Error Philosophy

RPC server failure is a fatal error - no fallback to single-file mode. This is intentional:

Why fatal?

1. Server startup failure indicates serious system issue (Node.js missing, permissions, etc)
2. Failing loudly during development catches problems immediately
3. Simpler code - no complex fallback logic
4. Performance: fallback would still be slow, defeating the purpose

When does it fail?

• Node.js not installed
• Socket permission issues
• Server crashes during startup
• Timeout waiting for ping (10 seconds)

How to debug:

# Manual server test
node js-parser-server.js --socket=/tmp/test.sock

# Socket connection test
echo '{"id":1,"method":"ping"}' | nc -U /tmp/test.sock

Applying This Pattern Elsewhere

This RPC architecture can be reused for other expensive Node.js operations:

Example: SCSS Compilation

class Scss_Compiler
{
    protected const RPC_SOCKET = 'storage/rsx-tmp/scss-compiler.sock';
    protected static $rpc_server_process = null;

    public static function compile_batch(array $files): array
    {
        if (static::$rpc_server_process === null) {
            static::start_rpc_server();
        }

        $sock = stream_socket_client('unix://' . base_path(self::RPC_SOCKET));
        fwrite($sock, json_encode([
            'id' => 1,
            'method' => 'compile',
            'files' => $files
        ]) . "\n");

        $response = fgets($sock);
        fclose($sock);

        return json_decode($response, true)['results'];
    }
}

Pattern Checklist

When implementing RPC server for another operation:

1. ✅ Socket path: Use storage/rsx-tmp/{name}.sock
2. ✅ Protocol: Line-delimited JSON
3. ✅ Lazy start: Only start when first operation needed
4. ✅ Stale cleanup: Force-stop existing server on startup
5. ✅ Ping/pong: Wait with timeout, fatal on failure
6. ✅ Graceful shutdown: Register shutdown handler
7. ✅ Force parameter: Support both normal and force shutdown
8. ✅ Signal handlers: Clean up socket on SIGTERM/SIGINT
9. ✅ Error handling: Fatal error, no silent fallback
10. ✅ Batch support: Process multiple items per request

Key Implementation Files

Reference these when implementing pattern elsewhere:

• PHP Server Management: /system/app/RSpade/Core/JavaScript/Js_Parser.php

  • Methods: start_rpc_server(), stop_rpc_server(), ping_rpc_server()
  • Lines: 528-680

• Node.js RPC Server: /system/app/RSpade/Core/JavaScript/resource/js-parser-server.js

  • Full example server with line-delimited JSON handling
  • Socket cleanup, signal handlers, graceful shutdown

Performance Characteristics

Clean Build (No Cache)

• Before: 1,200 process spawns = 60-180s overhead
• After: 1 process spawn + RPC calls = 1-2s overhead
• Speedup: ~30-90x for process management alone

Incremental Build (With Cache)

• Most files cached, few parse needed
• RPC overhead minimal (single server already running)
• Similar performance to single-file mode

Memory Usage

• Node.js server: ~50-100MB RAM
• Lives only during manifest build (seconds to minutes)
• Cleaned up automatically after build

Future Enhancements

1. Batch Processing: Send 50 files per RPC call instead of 1
2. Parallel Parsing: Node.js worker threads for CPU-bound parsing
3. Persistent Server: Keep server running between builds (with stale detection)
4. Shared RPC Framework: Abstract common RPC patterns into reusable library
5. Protocol Upgrade: msgpack or protobuf if JSON parsing becomes bottleneck

Debugging

Check Server Status

# Is server running?
ps aux | grep js-parser-server

# Does socket exist?
ls -lh storage/rsx-tmp/js-parser-server.sock

# Can we connect?
echo '{"id":1,"method":"ping"}' | nc -U storage/rsx-tmp/js-parser-server.sock

Manual Server Test

# Start server manually
node system/app/RSpade/Core/JavaScript/resource/js-parser-server.js \
    --socket=/tmp/test.sock

# In another terminal:
echo '{"id":1,"method":"ping"}' | nc -U /tmp/test.sock
# Should return: {"id":1,"result":"pong"}

Common Issues

Server won't start:

• Check Node.js installed: node --version
• Check socket directory writable: ls -ld storage/rsx-tmp
• Check for port/socket conflicts: lsof -U | grep js-parser

Timeout waiting for ping:

• Server crashed during startup (check stderr)
• Socket permissions issue
• Node.js interpreter not in PATH

Stale socket after crash:

• Handled automatically (force-stop on next start)
• Manual cleanup: rm storage/rsx-tmp/js-parser-server.sock

Security Considerations

Socket Permissions:

• Unix socket in storage/rsx-tmp owned by PHP process user
• No external access (not network socket)
• Cleaned up automatically

Input Validation:

• Server validates JSON requests
• File paths should be validated before sending to RPC
• No arbitrary code execution risk (parsing pre-validated JS files)

Resource Limits:

• Node.js process limited by OS
• No explicit memory limits (processes ~1-2 files per second)
• Terminates after manifest build completes

Conclusion

The RPC server architecture provides massive performance improvements for operations requiring many Node.js process invocations. The pattern is clean, maintainable, and reusable across different parts of the framework.

Key benefits:

• Performance: 30-90x faster for clean builds
• Reliability: Fatal errors catch problems immediately
• Maintainability: Clear lifecycle, graceful shutdown
• Reusability: Pattern applicable to SCSS, TypeScript, etc.

This README serves as the reference implementation for future RPC servers in RSpade.