# JavaScript Parser - RPC Server Architecture

## Problem Statement

RSpade's manifest build system needs to parse ~1,200+ JavaScript files to extract metadata (classes, methods, decorators, etc). The original implementation spawned a new Node.js process for each file:

```php
foreach ($js_files as $file) {
    $process = new Process(['node', 'js-parser.js', $file]);
    $process->run();
    // ... parse output
}
```

**Performance Impact:**
- 1,200+ process spawns per clean build
- Each spawn: ~50-150ms overhead
- Total overhead: 60-180 seconds just for process management
- Plus Node.js interpreter startup time per file

## Solution: Long-Running RPC Server

Replace N process spawns with ONE long-running Node.js server that handles all parse requests via RPC over a Unix domain socket.

**Benefits:**
- Single Node.js interpreter startup
- No process spawn overhead per file
- Batch processing support
- ~25-50x speedup for clean builds

## Architecture

### Components

#### 1. PHP Client (`Js_Parser.php`)
- Manages RPC server lifecycle
- Provides static methods for server operations
- Integrates with existing parse/cache logic

#### 2. Node.js RPC Server (`js-parser-server.js`)
- Long-running process listening on Unix socket
- Handles JSON-RPC style requests
- Processes multiple files per request (batch support)

#### 3. Unix Domain Socket
- Path: `storage/rsx-tmp/js-parser-server.sock`
- Protocol: Line-delimited JSON
- No port conflicts, no network stack overhead

### Communication Protocol

**Request Format:**
```json
{"id": 1, "method": "parse", "files": ["path1.js", "path2.js"]}\n
```

**Response Format:**
```json
{"id": 1, "results": {"path1.js": {...}, "path2.js": {...}}}\n
```

**Methods:**
- `ping` → `{id: N, result: "pong"}` - Health check
- `parse` → `{id: N, results: {file: data, ...}}` - Parse files
- `shutdown` → `{id: N, result: "shutting down"}` - Graceful stop

Line-delimited JSON allows simple stream parsing without complex framing.

## Server Lifecycle

### 1. Lazy Initialization

Server only starts when first JS file needs parsing (during manifest build Phase 2):

```php
public static function extract_metadata(string $file_path): array
{
    // Start RPC server on first parse (lazy initialization)
    if (static::$rpc_server_process === null) {
        static::start_rpc_server();
    }
    // ... continue parsing
}
```

### 2. Startup Procedure

When `start_rpc_server()` is called:

1. **Check for stale server:**
   ```php
   if (file_exists($socket_path)) {
       static::stop_rpc_server(force: true);
   }
   ```
   Force-stop any existing server to ensure clean slate.

2. **Spawn Node process:**
   ```php
   $process = new Process([
       'node',
       base_path('app/RSpade/Core/JavaScript/resource/js-parser-server.js'),
       '--socket=' . $socket_path
   ]);
   $process->start();
   ```

3. **Wait for ready (ping/pong):**
   ```php
   for ($i = 0; $i < 200; $i++) {  // 10 seconds max
       usleep(50000);  // 50ms
       if (static::ping_rpc_server()) {
           // Server ready!
           break;
       }
   }
   ```

4. **Fatal error on timeout:**
   ```php
   if (!$ready) {
       throw new \RuntimeException('Failed to start RPC server');
   }
   ```

5. **Register shutdown handler:**
   ```php
   register_shutdown_function([self::class, 'stop_rpc_server']);
   ```

### 3. Normal Operation

During manifest build, all JS parsing goes through the RPC server:

```php
// Future implementation:
$results = static::parse_via_rpc([
    'file1.js',
    'file2.js',
    // ... batch of 50 files
]);
```

Cache is checked first - only uncached files sent to RPC.

### 4. Graceful Shutdown

When manifest build completes:

```php
public static function stop_rpc_server(bool $force = false): void
{
    // Send shutdown command
    $sock = stream_socket_client("unix://{$socket_path}");
    fwrite($sock, json_encode(['method' => 'shutdown']) . "\n");
    fclose($sock);

    if ($force) {
        // Wait and force kill if needed
        sleep(1);
        if ($process->isRunning()) {
            $process->stop(3, SIGTERM);
        }
    }
    // Otherwise just send command and return immediately
}
```

## Force Stop Parameter

`stop_rpc_server($force = false)` has two modes:

### Normal Mode (`$force = false`)
- Send shutdown command
- Return immediately
- Trust server will exit gracefully
- Used during normal shutdown (shutdown handler)

### Force Mode (`$force = true`)
- Send shutdown command
- Wait 1 second
- If still running, send SIGTERM
- Clean up socket file
- Used when detecting stale server at startup

**Why two modes?**

At startup, we need to ensure old server is gone before starting new one (force mode). During normal shutdown, we don't need to wait - the server will handle shutdown and PHP exit won't block (normal mode).

## Node.js Server Implementation

### Key Design Decisions

**1. Line-Delimited JSON**

Simple, no complex framing needed:

```javascript
socket.on('data', (data) => {
    buffer += data.toString();
    let newlineIndex;
    while ((newlineIndex = buffer.indexOf('\n')) !== -1) {
        const line = buffer.substring(0, newlineIndex);
        buffer = buffer.substring(newlineIndex + 1);
        handleRequest(line);
    }
});
```

**2. Synchronous File Processing**

Parse files sequentially in single request handler - simpler than async complexity for this use case.

**3. Graceful Shutdown Handling**

```javascript
if (request.method === 'shutdown') {
    socket.end();
    server.close(() => {
        fs.unlinkSync(socketPath);
        process.exit(0);
    });
}
```

**4. Signal Handlers**

Ensure socket cleanup on unexpected termination:

```javascript
process.on('SIGTERM', () => {
    server.close(() => {
        fs.unlinkSync(socketPath);
        process.exit(0);
    });
});
```

## Cache Integration

The RPC server integrates with existing cache system:

```php
public static function parse($file_path)
{
    // Check cache first
    if (file_exists($cache_file)) {
        return json_decode(file_get_contents($cache_file), true);
    }

    // Use RPC if server running, else fallback to single-file
    if (static::$rpc_server_process !== null) {
        return static::parse_via_rpc([$file_path])[$file_path];
    }

    return static::_parse_without_cache($file_path);
}
```

**Batch Optimization (Future):**

Collect uncached files and send in batches to RPC:

```php
$uncached_files = array_filter($files, fn($f) => !cache_exists($f));
$results = static::parse_via_rpc($uncached_files);
foreach ($results as $file => $data) {
    cache_result($file, $data);
}
```

## Error Handling

### Fatal Error Philosophy

RPC server failure is a **fatal error** - no fallback to single-file mode. This is intentional:

**Why fatal?**
1. Server startup failure indicates serious system issue (Node.js missing, permissions, etc)
2. Failing loudly during development catches problems immediately
3. Simpler code - no complex fallback logic
4. Performance: fallback would still be slow, defeating the purpose

**When does it fail?**
- Node.js not installed
- Socket permission issues
- Server crashes during startup
- Timeout waiting for ping (10 seconds)

**How to debug:**
```bash
# Manual server test
node js-parser-server.js --socket=/tmp/test.sock

# Socket connection test
echo '{"id":1,"method":"ping"}' | nc -U /tmp/test.sock
```

## Applying This Pattern Elsewhere

This RPC architecture can be reused for other expensive Node.js operations:

### Example: SCSS Compilation

```php
class Scss_Compiler
{
    protected const RPC_SOCKET = 'storage/rsx-tmp/scss-compiler.sock';
    protected static $rpc_server_process = null;

    public static function compile_batch(array $files): array
    {
        if (static::$rpc_server_process === null) {
            static::start_rpc_server();
        }

        $sock = stream_socket_client('unix://' . base_path(self::RPC_SOCKET));
        fwrite($sock, json_encode([
            'id' => 1,
            'method' => 'compile',
            'files' => $files
        ]) . "\n");

        $response = fgets($sock);
        fclose($sock);

        return json_decode($response, true)['results'];
    }
}
```

### Pattern Checklist

When implementing RPC server for another operation:

1. ✅ **Socket path:** Use `storage/rsx-tmp/{name}.sock`
2. ✅ **Protocol:** Line-delimited JSON
3. ✅ **Lazy start:** Only start when first operation needed
4. ✅ **Stale cleanup:** Force-stop existing server on startup
5. ✅ **Ping/pong:** Wait with timeout, fatal on failure
6. ✅ **Graceful shutdown:** Register shutdown handler
7. ✅ **Force parameter:** Support both normal and force shutdown
8. ✅ **Signal handlers:** Clean up socket on SIGTERM/SIGINT
9. ✅ **Error handling:** Fatal error, no silent fallback
10. ✅ **Batch support:** Process multiple items per request

### Key Implementation Files

Reference these when implementing pattern elsewhere:

- **PHP Server Management:** `/system/app/RSpade/Core/JavaScript/Js_Parser.php`
  - Methods: `start_rpc_server()`, `stop_rpc_server()`, `ping_rpc_server()`
  - Lines: 528-680

- **Node.js RPC Server:** `/system/app/RSpade/Core/JavaScript/resource/js-parser-server.js`
  - Full example server with line-delimited JSON handling
  - Socket cleanup, signal handlers, graceful shutdown

## Performance Characteristics

### Clean Build (No Cache)
- **Before:** 1,200 process spawns = 60-180s overhead
- **After:** 1 process spawn + RPC calls = 1-2s overhead
- **Speedup:** ~30-90x for process management alone

### Incremental Build (With Cache)
- Most files cached, few parse needed
- RPC overhead minimal (single server already running)
- Similar performance to single-file mode

### Memory Usage
- Node.js server: ~50-100MB RAM
- Lives only during manifest build (seconds to minutes)
- Cleaned up automatically after build

## Future Enhancements

1. **Batch Processing:** Send 50 files per RPC call instead of 1
2. **Parallel Parsing:** Node.js worker threads for CPU-bound parsing
3. **Persistent Server:** Keep server running between builds (with stale detection)
4. **Shared RPC Framework:** Abstract common RPC patterns into reusable library
5. **Protocol Upgrade:** msgpack or protobuf if JSON parsing becomes bottleneck

## Debugging

### Check Server Status
```bash
# Is server running?
ps aux | grep js-parser-server

# Does socket exist?
ls -lh storage/rsx-tmp/js-parser-server.sock

# Can we connect?
echo '{"id":1,"method":"ping"}' | nc -U storage/rsx-tmp/js-parser-server.sock
```

### Manual Server Test
```bash
# Start server manually
node system/app/RSpade/Core/JavaScript/resource/js-parser-server.js \
    --socket=/tmp/test.sock

# In another terminal:
echo '{"id":1,"method":"ping"}' | nc -U /tmp/test.sock
# Should return: {"id":1,"result":"pong"}
```

### Common Issues

**Server won't start:**
- Check Node.js installed: `node --version`
- Check socket directory writable: `ls -ld storage/rsx-tmp`
- Check for port/socket conflicts: `lsof -U | grep js-parser`

**Timeout waiting for ping:**
- Server crashed during startup (check stderr)
- Socket permissions issue
- Node.js interpreter not in PATH

**Stale socket after crash:**
- Handled automatically (force-stop on next start)
- Manual cleanup: `rm storage/rsx-tmp/js-parser-server.sock`

## Security Considerations

**Socket Permissions:**
- Unix socket in `storage/rsx-tmp` owned by PHP process user
- No external access (not network socket)
- Cleaned up automatically

**Input Validation:**
- Server validates JSON requests
- File paths should be validated before sending to RPC
- No arbitrary code execution risk (parsing pre-validated JS files)

**Resource Limits:**
- Node.js process limited by OS
- No explicit memory limits (processes ~1-2 files per second)
- Terminates after manifest build completes

## Conclusion

The RPC server architecture provides massive performance improvements for operations requiring many Node.js process invocations. The pattern is clean, maintainable, and reusable across different parts of the framework.

Key benefits:
- **Performance:** 30-90x faster for clean builds
- **Reliability:** Fatal errors catch problems immediately
- **Maintainability:** Clear lifecycle, graceful shutdown
- **Reusability:** Pattern applicable to SCSS, TypeScript, etc.

This README serves as the reference implementation for future RPC servers in RSpade.