Refactor filename naming system and apply convention-based renames

Standardize settings file naming and relocate documentation files
Fix code quality violations from rsx:check
Reorganize user_management directory into logical subdirectories
Move Quill Bundle to core and align with Tom Select pattern
Simplify Site Settings page to focus on core site information
Complete Phase 5: Multi-tenant authentication with login flow and site selection
Add route query parameter rule and synchronize filename validation logic
Fix critical bug in UpdateNpmCommand causing missing JavaScript stubs
Implement filename convention rule and resolve VS Code auto-rename conflict
Implement js-sanitizer RPC server to eliminate 900+ Node.js process spawns
Implement RPC server architecture for JavaScript parsing
WIP: Add RPC server infrastructure for JS parsing (partial implementation)
Update jqhtml terminology from destroy to stop, fix datagrid DOM preservation
Add JQHTML-CLASS-01 rule and fix redundant class names
Improve code quality rules and resolve violations
Remove legacy fatal error format in favor of unified 'fatal' error type
Filter internal keys from window.rsxapp output
Update button styling and comprehensive form/modal documentation
Add conditional fly-in animation for modals
Fix non-deterministic bundle compilation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
root
2025-11-13 19:10:02 +00:00
parent fc494c1e08
commit 77b4d10af8
28155 changed files with 2191860 additions and 12967 deletions

View File

@@ -0,0 +1,91 @@
# JavaScript Core Systems - RPC Server Architecture
## Overview
JavaScript parsing and transformation use long-running Node.js RPC servers via Unix sockets to avoid spawning 1000+ Node processes during builds.
## JS Parser - RPC Server
### Components
- `Js_Parser.php` - PHP client, manages parser server lifecycle
- `js-parser-server.js` - Node.js RPC server, processes batch parse requests
- `js-parser.js` - Legacy single-file parser (kept for compatibility)
### Server Lifecycle
1. **Lazy start:** Server spawns on first JS file parse during manifest build
2. **Startup:** Checks for stale socket, force-kills if found, starts fresh server
3. **Wait:** Polls socket with ping (50ms intervals, 10s max), fatal error if timeout
4. **Usage:** All JS parsing during build goes through RPC (batched when possible)
5. **Shutdown:** Graceful shutdown on manifest build completion (registered shutdown handler)
### Socket
- **Path:** `storage/rsx-tmp/js-parser-server.sock`
- **Protocol:** Line-delimited JSON over Unix domain socket
### RPC Methods
- `ping``"pong"` - Health check
- `parse``{file: result, ...}` - Batch parse multiple files
- `shutdown` → Graceful server termination
### PHP API
```php
Js_Parser::start_rpc_server(); // Lazy init, auto-called
Js_Parser::stop_rpc_server($force); // Clean shutdown
Js_Parser::parse_via_rpc($files); // Batch parse (future)
```
## JS Transformer (Babel) - RPC Server
### Components
- `Js_Transformer.php` - PHP client, manages transformer server lifecycle
- `js-transformer-server.js` - Node.js RPC server, processes batch Babel transformations
- `js-transformer.js` - Legacy single-file transformer (kept for compatibility)
### Server Lifecycle
1. **Lazy start:** Server spawns on first JS transformation during bundle compilation
2. **Startup:** Checks for stale socket, force-kills if found, starts fresh server
3. **Wait:** Polls socket with ping (50ms intervals, 10s max), fatal error if timeout
4. **Usage:** All JS transformations during bundle builds go through RPC
5. **Shutdown:** Graceful shutdown when bundle compilation completes (registered shutdown handler)
### Socket
- **Path:** `storage/rsx-tmp/js-transformer-server.sock`
- **Protocol:** Line-delimited JSON over Unix domain socket
### RPC Methods
- `ping``"pong"` - Health check
- `transform``{file: {status, result, hash}, ...}` - Batch transform multiple files
- `shutdown` → Graceful server termination
### PHP API
```php
Js_Transformer::start_rpc_server(); // Lazy init, auto-called
Js_Transformer::stop_rpc_server($force); // Clean shutdown
Js_Transformer::_transform_via_rpc(...); // Internal RPC transformation
```
### Transformation Details
- Preprocesses `@decorator` on standalone functions
- Applies Babel transformations: decorators, class properties, optional chaining, nullish coalescing
- Prefixes generated helper functions with file hash to prevent namespace collisions
- Uses target presets: `modern`, `es6`, `es5`
- Generates inline source maps for debugging
## Common RPC Pattern
### Force Parameter
`stop_rpc_server($force = false)`:
- `false` (default): Send shutdown command, return immediately
- `true`: Send shutdown + wait + SIGTERM if needed (used for stale server cleanup)
### Cache Integration
Cache checked before RPC call - only uncached files sent to server for processing.
### Error Handling
Server failure → fatal error (no fallback). Server must start or build/compilation fails.
### Performance Impact
**Before RPC:** N Node.js process spawns (where N = number of files needing processing)
**After RPC:** Single persistent Node.js process per server type (~1-2s startup overhead)
## Future: Other RPC Servers
This pattern is reusable for other expensive Node operations. See implementation guide in project summary.

View File

@@ -19,11 +19,36 @@ class Js_Parser
*/
protected const PARSER_SCRIPT = 'app/RSpade/Core/JavaScript/resource/js-parser.js';
/**
* Node.js RPC server script path
*/
protected const RPC_SERVER_SCRIPT = 'app/RSpade/Core/JavaScript/resource/js-parser-server.js';
/**
* Unix socket path for RPC server
*/
protected const RPC_SOCKET = 'storage/rsx-tmp/js-parser-server.sock';
/**
* Cache directory for parsed JavaScript files
*/
protected const CACHE_DIR = 'storage/rsx-tmp/persistent/js_parser';
/**
* RPC server process
*/
protected static $rpc_server_process = null;
/**
* RPC server socket connection
*/
protected static $rpc_socket = null;
/**
* Request ID counter
*/
protected static $request_id = 0;
/**
* Parse a JavaScript file using Node.js AST parser with caching
*/
@@ -58,42 +83,60 @@ class Js_Parser
*/
protected static function _parse_without_cache($file_path)
{
// Always use advanced parser for decorator support
// The simple parser is deprecated and doesn't support modern ES features
$parser_path = base_path(self::PARSER_SCRIPT);
if (!File::exists($parser_path)) {
throw new \RuntimeException("No JavaScript parser available. Please ensure Node.js and babel parser are installed.");
}
// Always use RPC server - if not running, that's a fatal error
return static::_parse_via_rpc($file_path);
}
$process = new Process([
'node',
$parser_path,
'--json', // Use JSON output for structured error reporting
$file_path
]);
// Set working directory to base path to find node_modules
$process->setWorkingDirectory(base_path());
$process->setTimeout(10); // 10 second timeout
/**
* Parse via RPC server
*/
protected static function _parse_via_rpc($file_path)
{
$socket_path = base_path(self::RPC_SOCKET);
try {
$process->run();
$output = $process->getOutput();
if (empty($output)) {
throw new \RuntimeException("JavaScript parser returned empty output for {$file_path}");
$sock = @stream_socket_client("unix://{$socket_path}", $errno, $errstr, 0.5);
if (!$sock) {
throw new \RuntimeException("Failed to connect to RPC server: {$errstr}");
}
// Parse JSON output
$result = @json_decode($output, true);
// Set blocking mode for reliable reads
stream_set_blocking($sock, true);
if (!$result || !is_array($result)) {
// Handle non-JSON output (shouldn't happen with --json flag)
throw new \RuntimeException(
"JavaScript parser returned invalid JSON for {$file_path}:\n" . $output
);
// Send parse request
$request = [
'id' => ++static::$request_id,
'method' => 'parse',
'files' => [$file_path]
];
fwrite($sock, json_encode($request) . "\n");
// Read response
$response = fgets($sock);
fclose($sock);
if (!$response) {
throw new \RuntimeException("No response from RPC server");
}
$data = json_decode($response, true);
if (!$data || !is_array($data)) {
throw new \RuntimeException("Invalid JSON response from RPC server");
}
if (isset($data['error'])) {
throw new \RuntimeException("RPC error: " . $data['error']);
}
if (!isset($data['results'][$file_path])) {
throw new \RuntimeException("No result for file in RPC response");
}
$result = $data['results'][$file_path];
// Handle parse result
if ($result['status'] === 'success') {
return $result['result'];
}
@@ -118,7 +161,7 @@ class Js_Parser
);
case 'CodeOutsideAllowed':
$error_msg = "JavaScript files without classes may only contain function declarations and comments.";
$error_msg = "JavaScript files without classes may only contain function declarations, const variables with static values, and comments.";
if ($code) {
$error_msg .= "\nFound: {$code}";
}
@@ -146,6 +189,9 @@ class Js_Parser
$line
);
case 'FileReadError':
throw new \RuntimeException("File read error: " . $message);
default:
// Clean up the message - remove redundant file path info
$message = preg_replace('/^Parse error:\s*/', '', $message);
@@ -168,36 +214,17 @@ class Js_Parser
// Unknown response format
throw new \RuntimeException(
"JavaScript parser returned unexpected response for {$file_path}:\n" .
"JavaScript parser RPC returned unexpected response for {$file_path}:\n" .
json_encode($result, JSON_PRETTY_PRINT)
);
} catch (ProcessFailedException $e) {
// Process failed to run at all
$error_output = $process->getErrorOutput();
// Check for missing Node.js
if (str_contains($e->getMessage(), 'No such file or directory') &&
str_contains($e->getMessage(), 'node')) {
throw new \RuntimeException(
"Node.js is not installed or not in PATH.\n" .
"Install Node.js to parse JavaScript files."
);
}
// Generic process failure
throw new \RuntimeException(
"Failed to run JavaScript parser for {$file_path}:\n" .
$e->getMessage() . "\n" .
"Error output: " . $error_output
);
} catch (Js_Exception $e) {
// Re-throw JavaScript exceptions
throw $e;
} catch (\Exception $e) {
// Wrap other exceptions
throw new \RuntimeException(
"JavaScript parser error for {$file_path}: " . $e->getMessage()
"JavaScript parser RPC error for {$file_path}: " . $e->getMessage()
);
}
}
@@ -419,6 +446,11 @@ JS;
*/
public static function extract_metadata(string $file_path): array
{
// Start RPC server on first parse (lazy initialization)
if (static::$rpc_server_process === null) {
static::start_rpc_server();
}
$data = [];
// Use static parser to get raw parsed data
$parsed = static::parse($file_path);
@@ -499,4 +531,136 @@ JS;
return $data;
}
/**
* Start the RPC server
*/
public static function start_rpc_server(): void
{
// Check if server already running
$socket_path = base_path(self::RPC_SOCKET);
if (file_exists($socket_path)) {
// Server might be running, force stop it
console_debug('JS_PARSER', 'Found existing socket, forcing shutdown');
static::stop_rpc_server(force: true);
}
// Start new server
$server_script = base_path(self::RPC_SERVER_SCRIPT);
console_debug('JS_PARSER', 'Starting RPC server: ' . $server_script);
$process = new Process([
'node',
$server_script,
'--socket=' . $socket_path
]);
$process->start();
static::$rpc_server_process = $process;
// Register shutdown handler
register_shutdown_function([self::class, 'stop_rpc_server']);
// Wait for server to be ready (ping/pong up to 10 seconds)
console_debug('JS_PARSER', 'Waiting for RPC server to be ready...');
$max_attempts = 200; // 10 seconds (50ms * 200)
$ready = false;
for ($i = 0; $i < $max_attempts; $i++) {
usleep(50000); // 50ms
if (file_exists($socket_path)) {
// Try to ping
if (static::__ping_rpc_server()) {
$ready = true;
console_debug('JS_PARSER', 'RPC server ready after ' . ($i * 50) . 'ms');
break;
}
}
}
if (!$ready) {
static::stop_rpc_server();
throw new \RuntimeException('Failed to start JS Parser RPC server - timeout after 10 seconds');
}
console_debug('JS_PARSER', 'RPC server started successfully');
}
/**
* Ping the RPC server
*/
protected static function __ping_rpc_server(): bool
{
$socket_path = base_path(self::RPC_SOCKET);
try {
$sock = @stream_socket_client("unix://{$socket_path}", $errno, $errstr, 0.5);
if (!$sock) {
return false;
}
// Set blocking mode for reliable reads
stream_set_blocking($sock, true);
// Send ping
fwrite($sock, json_encode(['id' => ++static::$request_id, 'method' => 'ping']) . "\n");
// Read response
$response = fgets($sock);
fclose($sock);
if (!$response) {
return false;
}
$data = json_decode($response, true);
return isset($data['result']) && $data['result'] === 'pong';
} catch (\Exception $e) {
return false;
}
}
/**
* Stop the RPC server
*/
public static function stop_rpc_server(bool $force = false): void
{
$socket_path = base_path(self::RPC_SOCKET);
// Try graceful shutdown
if (file_exists($socket_path)) {
try {
$sock = @stream_socket_client("unix://{$socket_path}", $errno, $errstr, 0.5);
if ($sock) {
fwrite($sock, json_encode(['id' => 0, 'method' => 'shutdown']) . "\n");
fclose($sock);
}
} catch (\Exception $e) {
// Ignore errors
}
}
// Only wait and force kill if $force = true
if ($force && static::$rpc_server_process && static::$rpc_server_process->isRunning()) {
console_debug('JS_PARSER', 'Force stopping RPC server');
// Wait for graceful shutdown
sleep(1);
// Force kill if still running
if (static::$rpc_server_process->isRunning()) {
static::$rpc_server_process->stop(3, SIGTERM);
}
}
// Clean up socket file
if (file_exists($socket_path)) {
@unlink($socket_path);
}
}
}

View File

@@ -20,7 +20,12 @@ use RuntimeException;
class Js_Transformer
{
/**
* Node.js transformer script path
* Node.js transformer script path (RPC server)
*/
protected const RPC_SERVER_SCRIPT = 'app/RSpade/Core/JavaScript/resource/js-transformer-server.js';
/**
* Transformer script path for availability checking (RPC server used for actual transformations)
*/
protected const TRANSFORMER_SCRIPT = 'app/RSpade/Core/JavaScript/resource/js-transformer.js';
@@ -29,6 +34,21 @@ class Js_Transformer
*/
protected const CACHE_DIR = 'storage/rsx-tmp/babel_cache';
/**
* RPC server socket path
*/
protected const RPC_SOCKET = 'storage/rsx-tmp/js-transformer-server.sock';
/**
* RPC server process
*/
protected static $rpc_server_process = null;
/**
* RPC request ID counter
*/
protected static $request_id = 0;
/**
* Transform a JavaScript file using Babel
*
@@ -95,114 +115,8 @@ class Js_Transformer
*/
protected static function _transform_without_cache(string $file_path, string $target, ?string $original_path = null): string
{
$transformer_path = base_path(self::TRANSFORMER_SCRIPT);
if (!File::exists($transformer_path)) {
throw new RuntimeException("Babel transformer script not found at {$transformer_path}");
}
// Use original path for hash generation if provided (for temp files)
$hash_path = $original_path ?: $file_path;
$process = new Process([
'node',
$transformer_path,
'--json', // Use JSON output for structured error reporting
$file_path,
$target,
$hash_path // Pass the path for hash generation
]);
// Set working directory to base path to find node_modules
$process->setWorkingDirectory(base_path());
$process->setTimeout(30); // 30 second timeout
try {
$process->run();
$output = $process->getOutput();
if (empty($output)) {
throw new RuntimeException("Babel transformer returned empty output for {$file_path}");
}
// Parse JSON output
$result = @json_decode($output, true);
if (!$result || !is_array($result)) {
throw new RuntimeException(
"Babel transformer returned invalid JSON for {$file_path}:\n" . $output
);
}
if ($result['status'] === 'success') {
return $result['result'];
}
// Handle error response
if ($result['status'] === 'error' && isset($result['error'])) {
$error = $result['error'];
$message = $error['message'] ?? 'Unknown error';
$line = $error['line'] ?? null;
$column = $error['column'] ?? null;
$suggestion = $error['suggestion'] ?? null;
// Build error message
$error_msg = "JavaScript transformation failed";
if ($line && $column) {
$error_msg .= " at line {$line}, column {$column}";
}
$error_msg .= " in {$file_path}:\n{$message}";
if ($suggestion) {
$error_msg .= "\n\n{$suggestion}";
}
// Check for specific error types
if (str_contains($message, 'Cannot find module')) {
throw new RuntimeException(
"Babel packages not installed.\n" .
"Run: npm install\n" .
"Error: {$message}"
);
}
if (str_contains($message, 'No such file or directory') &&
str_contains($message, 'node')) {
throw new RuntimeException(
"Node.js is not installed or not in PATH.\n" .
"Install Node.js to use JavaScript transformation."
);
}
throw new RuntimeException($error_msg);
}
// Unknown response format
throw new RuntimeException(
"Babel transformer returned unexpected response for {$file_path}:\n" .
json_encode($result, JSON_PRETTY_PRINT)
);
} catch (ProcessFailedException $e) {
// Process failed to run at all
$error_output = $process->getErrorOutput();
// Check for missing Node.js
if (str_contains($e->getMessage(), 'No such file or directory') &&
str_contains($e->getMessage(), 'node')) {
throw new RuntimeException(
"Node.js is not installed or not in PATH.\n" .
"Install Node.js to use JavaScript transformation."
);
}
// Generic process failure
throw new RuntimeException(
"Failed to run Babel transformer for {$file_path}:\n" .
$e->getMessage() . "\n" .
"Error output: " . $error_output
);
}
// Use RPC server for transformation
return static::_transform_via_rpc($file_path, $target, $original_path);
}
/**
@@ -287,4 +201,297 @@ class Js_Transformer
'@babel/plugin-proposal-decorators' => '^7.24.0'
];
}
/**
* Start RPC server for JavaScript transformation
* Lazy initialization - called automatically on first transform
*/
public static function start_rpc_server(): void
{
$socket_path = base_path(self::RPC_SOCKET);
$server_script = base_path(self::RPC_SERVER_SCRIPT);
if (!File::exists($server_script)) {
throw new RuntimeException("JS Transformer RPC server script not found at {$server_script}");
}
// If socket exists, check if it's stale
if (file_exists($socket_path)) {
// Try to ping the existing server
if (static::ping_rpc_server()) {
// Server already running and responsive
return;
}
// Socket exists but server not responding - force cleanup
static::stop_rpc_server(force: true);
}
// Ensure socket directory exists
$socket_dir = dirname($socket_path);
if (!is_dir($socket_dir)) {
mkdir($socket_dir, 0755, true);
}
// Start RPC server
$process = new Process([
'node',
$server_script,
'--socket=' . $socket_path
]);
$process->setWorkingDirectory(base_path());
$process->setTimeout(null); // No timeout for long-running server
$process->start();
static::$rpc_server_process = $process;
// Wait for server to be ready (ping until it responds)
$max_wait_ms = 10000; // 10 seconds
$wait_interval_ms = 50;
$iterations = $max_wait_ms / $wait_interval_ms;
for ($i = 0; $i < $iterations; $i++) {
usleep($wait_interval_ms * 1000); // Convert to microseconds
if (static::ping_rpc_server()) {
// Register shutdown handler to clean up server
register_shutdown_function([self::class, 'stop_rpc_server']);
return;
}
}
// Timeout waiting for server
throw new RuntimeException(
"JS Transformer RPC server failed to start within {$max_wait_ms}ms.\n" .
"Check that Node.js and Babel dependencies are installed."
);
}
/**
* Ping the RPC server to check if it's alive
*
* @return bool True if server responds, false otherwise
*/
public static function ping_rpc_server(): bool
{
$socket_path = base_path(self::RPC_SOCKET);
if (!file_exists($socket_path)) {
return false;
}
try {
$socket = @stream_socket_client('unix://' . $socket_path, $errno, $errstr, 1);
if (!$socket) {
return false;
}
// Set blocking mode for reliable reads/writes
stream_set_blocking($socket, true);
static::$request_id++;
$request = json_encode([
'id' => static::$request_id,
'method' => 'ping'
]) . "\n";
fwrite($socket, $request);
$response = fgets($socket);
fclose($socket);
if (!$response) {
return false;
}
$result = json_decode($response, true);
return isset($result['result']) && $result['result'] === 'pong';
} catch (\Exception $e) {
return false;
}
}
/**
* Stop the RPC server
*
* @param bool $force If true, forcefully kill the server (used for cleanup)
*/
public static function stop_rpc_server(bool $force = false): void
{
$socket_path = base_path(self::RPC_SOCKET);
if ($force) {
// Force cleanup: send shutdown, wait briefly, then SIGTERM if needed
if (file_exists($socket_path)) {
try {
$socket = @stream_socket_client('unix://' . $socket_path, $errno, $errstr, 1);
if ($socket) {
// Set blocking mode for reliable writes
stream_set_blocking($socket, true);
static::$request_id++;
$request = json_encode([
'id' => static::$request_id,
'method' => 'shutdown'
]) . "\n";
fwrite($socket, $request);
fclose($socket);
}
} catch (\Exception $e) {
// Ignore errors during force shutdown
}
// Wait briefly for graceful shutdown
usleep(100000); // 100ms
// Force remove socket
@unlink($socket_path);
}
// Kill process if still running
if (static::$rpc_server_process && static::$rpc_server_process->isRunning()) {
static::$rpc_server_process->stop(0, SIGTERM);
}
} else {
// Graceful shutdown: just send shutdown command
if (file_exists($socket_path)) {
try {
$socket = @stream_socket_client('unix://' . $socket_path, $errno, $errstr, 1);
if ($socket) {
// Set blocking mode for reliable writes
stream_set_blocking($socket, true);
static::$request_id++;
$request = json_encode([
'id' => static::$request_id,
'method' => 'shutdown'
]) . "\n";
fwrite($socket, $request);
fclose($socket);
}
} catch (\Exception $e) {
// Ignore errors during graceful shutdown
}
}
}
static::$rpc_server_process = null;
}
/**
* Transform file via RPC server
*
* @param string $file_path Path to file to transform
* @param string $target Target environment
* @param string|null $original_path Original file path for hash (if using temp file)
* @return string Transformed code
*/
protected static function _transform_via_rpc(string $file_path, string $target, ?string $original_path = null): string
{
// Start RPC server on first use (lazy initialization)
if (static::$rpc_server_process === null) {
static::start_rpc_server();
}
$socket_path = base_path(self::RPC_SOCKET);
if (!file_exists($socket_path)) {
throw new RuntimeException("JS Transformer RPC server socket not found at {$socket_path}");
}
// Connect to RPC server
$socket = @stream_socket_client('unix://' . $socket_path, $errno, $errstr, 5);
if (!$socket) {
throw new RuntimeException("Failed to connect to JS Transformer RPC server: {$errstr}");
}
// Set blocking mode for reliable reads/writes
stream_set_blocking($socket, true);
// Use original path for hash generation if provided (for temp files)
$hash_path = $original_path ?: $file_path;
// Send transform request
static::$request_id++;
$request = json_encode([
'id' => static::$request_id,
'method' => 'transform',
'files' => [
[
'path' => $file_path,
'target' => $target,
'hash_path' => $hash_path
]
]
]) . "\n";
fwrite($socket, $request);
// Read response
$response = fgets($socket);
fclose($socket);
if (!$response) {
throw new RuntimeException("JS Transformer RPC server returned empty response for {$file_path}");
}
$result = json_decode($response, true);
if (!$result || !isset($result['results'])) {
throw new RuntimeException(
"JS Transformer RPC server returned invalid response for {$file_path}:\n" . $response
);
}
$file_result = $result['results'][$file_path] ?? null;
if (!$file_result) {
throw new RuntimeException("JS Transformer RPC server did not return result for {$file_path}");
}
// Handle error response
if ($file_result['status'] === 'error' && isset($file_result['error'])) {
$error = $file_result['error'];
$message = $error['message'] ?? 'Unknown error';
$line = $error['line'] ?? null;
$column = $error['column'] ?? null;
$suggestion = $error['suggestion'] ?? null;
// Build error message
$error_msg = "JavaScript transformation failed";
if ($line && $column) {
$error_msg .= " at line {$line}, column {$column}";
}
$error_msg .= " in {$file_path}:\n{$message}";
if ($suggestion) {
$error_msg .= "\n\n{$suggestion}";
}
// Check for specific error types
if (str_contains($message, 'Cannot find module')) {
throw new RuntimeException(
"Babel packages not installed.\n" .
"Run: npm install\n" .
"Error: {$message}"
);
}
throw new RuntimeException($error_msg);
}
// Success - return transformed code
if ($file_result['status'] === 'success' && isset($file_result['result'])) {
return $file_result['result'];
}
// Unknown response format
throw new RuntimeException(
"JS Transformer RPC server returned unexpected response for {$file_path}:\n" .
json_encode($file_result, JSON_PRETTY_PRINT)
);
}
}

View File

@@ -0,0 +1,452 @@
# JavaScript Parser - RPC Server Architecture
## Problem Statement
RSpade's manifest build system needs to parse ~1,200+ JavaScript files to extract metadata (classes, methods, decorators, etc). The original implementation spawned a new Node.js process for each file:
```php
foreach ($js_files as $file) {
$process = new Process(['node', 'js-parser.js', $file]);
$process->run();
// ... parse output
}
```
**Performance Impact:**
- 1,200+ process spawns per clean build
- Each spawn: ~50-150ms overhead
- Total overhead: 60-180 seconds just for process management
- Plus Node.js interpreter startup time per file
## Solution: Long-Running RPC Server
Replace N process spawns with ONE long-running Node.js server that handles all parse requests via RPC over a Unix domain socket.
**Benefits:**
- Single Node.js interpreter startup
- No process spawn overhead per file
- Batch processing support
- ~25-50x speedup for clean builds
## Architecture
### Components
#### 1. PHP Client (`Js_Parser.php`)
- Manages RPC server lifecycle
- Provides static methods for server operations
- Integrates with existing parse/cache logic
#### 2. Node.js RPC Server (`js-parser-server.js`)
- Long-running process listening on Unix socket
- Handles JSON-RPC style requests
- Processes multiple files per request (batch support)
#### 3. Unix Domain Socket
- Path: `storage/rsx-tmp/js-parser-server.sock`
- Protocol: Line-delimited JSON
- No port conflicts, no network stack overhead
### Communication Protocol
**Request Format:**
```json
{"id": 1, "method": "parse", "files": ["path1.js", "path2.js"]}\n
```
**Response Format:**
```json
{"id": 1, "results": {"path1.js": {...}, "path2.js": {...}}}\n
```
**Methods:**
- `ping``{id: N, result: "pong"}` - Health check
- `parse``{id: N, results: {file: data, ...}}` - Parse files
- `shutdown``{id: N, result: "shutting down"}` - Graceful stop
Line-delimited JSON allows simple stream parsing without complex framing.
## Server Lifecycle
### 1. Lazy Initialization
Server only starts when first JS file needs parsing (during manifest build Phase 2):
```php
public static function extract_metadata(string $file_path): array
{
// Start RPC server on first parse (lazy initialization)
if (static::$rpc_server_process === null) {
static::start_rpc_server();
}
// ... continue parsing
}
```
### 2. Startup Procedure
When `start_rpc_server()` is called:
1. **Check for stale server:**
```php
if (file_exists($socket_path)) {
static::stop_rpc_server(force: true);
}
```
Force-stop any existing server to ensure clean slate.
2. **Spawn Node process:**
```php
$process = new Process([
'node',
base_path('app/RSpade/Core/JavaScript/resource/js-parser-server.js'),
'--socket=' . $socket_path
]);
$process->start();
```
3. **Wait for ready (ping/pong):**
```php
for ($i = 0; $i < 200; $i++) { // 10 seconds max
usleep(50000); // 50ms
if (static::ping_rpc_server()) {
// Server ready!
break;
}
}
```
4. **Fatal error on timeout:**
```php
if (!$ready) {
throw new \RuntimeException('Failed to start RPC server');
}
```
5. **Register shutdown handler:**
```php
register_shutdown_function([self::class, 'stop_rpc_server']);
```
### 3. Normal Operation
During manifest build, all JS parsing goes through the RPC server:
```php
// Future implementation:
$results = static::parse_via_rpc([
'file1.js',
'file2.js',
// ... batch of 50 files
]);
```
Cache is checked first - only uncached files sent to RPC.
### 4. Graceful Shutdown
When manifest build completes:
```php
public static function stop_rpc_server(bool $force = false): void
{
// Send shutdown command
$sock = stream_socket_client("unix://{$socket_path}");
fwrite($sock, json_encode(['method' => 'shutdown']) . "\n");
fclose($sock);
if ($force) {
// Wait and force kill if needed
sleep(1);
if ($process->isRunning()) {
$process->stop(3, SIGTERM);
}
}
// Otherwise just send command and return immediately
}
```
## Force Stop Parameter
`stop_rpc_server($force = false)` has two modes:
### Normal Mode (`$force = false`)
- Send shutdown command
- Return immediately
- Trust server will exit gracefully
- Used during normal shutdown (shutdown handler)
### Force Mode (`$force = true`)
- Send shutdown command
- Wait 1 second
- If still running, send SIGTERM
- Clean up socket file
- Used when detecting stale server at startup
**Why two modes?**
At startup, we need to ensure old server is gone before starting new one (force mode). During normal shutdown, we don't need to wait - the server will handle shutdown and PHP exit won't block (normal mode).
## Node.js Server Implementation
### Key Design Decisions
**1. Line-Delimited JSON**
Simple, no complex framing needed:
```javascript
socket.on('data', (data) => {
buffer += data.toString();
let newlineIndex;
while ((newlineIndex = buffer.indexOf('\n')) !== -1) {
const line = buffer.substring(0, newlineIndex);
buffer = buffer.substring(newlineIndex + 1);
handleRequest(line);
}
});
```
**2. Synchronous File Processing**
Parse files sequentially in single request handler - simpler than async complexity for this use case.
**3. Graceful Shutdown Handling**
```javascript
if (request.method === 'shutdown') {
socket.end();
server.close(() => {
fs.unlinkSync(socketPath);
process.exit(0);
});
}
```
**4. Signal Handlers**
Ensure socket cleanup on unexpected termination:
```javascript
process.on('SIGTERM', () => {
server.close(() => {
fs.unlinkSync(socketPath);
process.exit(0);
});
});
```
## Cache Integration
The RPC server integrates with existing cache system:
```php
public static function parse($file_path)
{
// Check cache first
if (file_exists($cache_file)) {
return json_decode(file_get_contents($cache_file), true);
}
// Use RPC if server running, else fallback to single-file
if (static::$rpc_server_process !== null) {
return static::parse_via_rpc([$file_path])[$file_path];
}
return static::_parse_without_cache($file_path);
}
```
**Batch Optimization (Future):**
Collect uncached files and send in batches to RPC:
```php
$uncached_files = array_filter($files, fn($f) => !cache_exists($f));
$results = static::parse_via_rpc($uncached_files);
foreach ($results as $file => $data) {
cache_result($file, $data);
}
```
## Error Handling
### Fatal Error Philosophy
RPC server failure is a **fatal error** - no fallback to single-file mode. This is intentional:
**Why fatal?**
1. Server startup failure indicates serious system issue (Node.js missing, permissions, etc)
2. Failing loudly during development catches problems immediately
3. Simpler code - no complex fallback logic
4. Performance: fallback would still be slow, defeating the purpose
**When does it fail?**
- Node.js not installed
- Socket permission issues
- Server crashes during startup
- Timeout waiting for ping (10 seconds)
**How to debug:**
```bash
# Manual server test
node js-parser-server.js --socket=/tmp/test.sock
# Socket connection test
echo '{"id":1,"method":"ping"}' | nc -U /tmp/test.sock
```
## Applying This Pattern Elsewhere
This RPC architecture can be reused for other expensive Node.js operations:
### Example: SCSS Compilation
```php
class Scss_Compiler
{
protected const RPC_SOCKET = 'storage/rsx-tmp/scss-compiler.sock';
protected static $rpc_server_process = null;
public static function compile_batch(array $files): array
{
if (static::$rpc_server_process === null) {
static::start_rpc_server();
}
$sock = stream_socket_client('unix://' . base_path(self::RPC_SOCKET));
fwrite($sock, json_encode([
'id' => 1,
'method' => 'compile',
'files' => $files
]) . "\n");
$response = fgets($sock);
fclose($sock);
return json_decode($response, true)['results'];
}
}
```
### Pattern Checklist
When implementing RPC server for another operation:
1. ✅ **Socket path:** Use `storage/rsx-tmp/{name}.sock`
2. ✅ **Protocol:** Line-delimited JSON
3. ✅ **Lazy start:** Only start when first operation needed
4. ✅ **Stale cleanup:** Force-stop existing server on startup
5. ✅ **Ping/pong:** Wait with timeout, fatal on failure
6. ✅ **Graceful shutdown:** Register shutdown handler
7. ✅ **Force parameter:** Support both normal and force shutdown
8. ✅ **Signal handlers:** Clean up socket on SIGTERM/SIGINT
9. ✅ **Error handling:** Fatal error, no silent fallback
10. ✅ **Batch support:** Process multiple items per request
### Key Implementation Files
Reference these when implementing pattern elsewhere:
- **PHP Server Management:** `/system/app/RSpade/Core/JavaScript/Js_Parser.php`
- Methods: `start_rpc_server()`, `stop_rpc_server()`, `ping_rpc_server()`
- Lines: 528-680
- **Node.js RPC Server:** `/system/app/RSpade/Core/JavaScript/resource/js-parser-server.js`
- Full example server with line-delimited JSON handling
- Socket cleanup, signal handlers, graceful shutdown
## Performance Characteristics
### Clean Build (No Cache)
- **Before:** 1,200 process spawns = 60-180s overhead
- **After:** 1 process spawn + RPC calls = 1-2s overhead
- **Speedup:** ~30-90x for process management alone
### Incremental Build (With Cache)
- Most files cached, few parse needed
- RPC overhead minimal (single server already running)
- Similar performance to single-file mode
### Memory Usage
- Node.js server: ~50-100MB RAM
- Lives only during manifest build (seconds to minutes)
- Cleaned up automatically after build
## Future Enhancements
1. **Batch Processing:** Send 50 files per RPC call instead of 1
2. **Parallel Parsing:** Node.js worker threads for CPU-bound parsing
3. **Persistent Server:** Keep server running between builds (with stale detection)
4. **Shared RPC Framework:** Abstract common RPC patterns into reusable library
5. **Protocol Upgrade:** msgpack or protobuf if JSON parsing becomes bottleneck
## Debugging
### Check Server Status
```bash
# Is server running?
ps aux | grep js-parser-server
# Does socket exist?
ls -lh storage/rsx-tmp/js-parser-server.sock
# Can we connect?
echo '{"id":1,"method":"ping"}' | nc -U storage/rsx-tmp/js-parser-server.sock
```
### Manual Server Test
```bash
# Start server manually
node system/app/RSpade/Core/JavaScript/resource/js-parser-server.js \
--socket=/tmp/test.sock
# In another terminal:
echo '{"id":1,"method":"ping"}' | nc -U /tmp/test.sock
# Should return: {"id":1,"result":"pong"}
```
### Common Issues
**Server won't start:**
- Check Node.js installed: `node --version`
- Check socket directory writable: `ls -ld storage/rsx-tmp`
- Check for port/socket conflicts: `lsof -U | grep js-parser`
**Timeout waiting for ping:**
- Server crashed during startup (check stderr)
- Socket permissions issue
- Node.js interpreter not in PATH
**Stale socket after crash:**
- Handled automatically (force-stop on next start)
- Manual cleanup: `rm storage/rsx-tmp/js-parser-server.sock`
## Security Considerations
**Socket Permissions:**
- Unix socket in `storage/rsx-tmp` owned by PHP process user
- No external access (not network socket)
- Cleaned up automatically
**Input Validation:**
- Server validates JSON requests
- File paths should be validated before sending to RPC
- No arbitrary code execution risk (parsing pre-validated JS files)
**Resource Limits:**
- Node.js process limited by OS
- No explicit memory limits (processes ~1-2 files per second)
- Terminates after manifest build completes
## Conclusion
The RPC server architecture provides massive performance improvements for operations requiring many Node.js process invocations. The pattern is clean, maintainable, and reusable across different parts of the framework.
Key benefits:
- **Performance:** 30-90x faster for clean builds
- **Reliability:** Fatal errors catch problems immediately
- **Maintainability:** Clear lifecycle, graceful shutdown
- **Reusability:** Pattern applicable to SCSS, TypeScript, etc.
This README serves as the reference implementation for future RPC servers in RSpade.

View File

@@ -0,0 +1,982 @@
#!/usr/bin/env node
const fs = require('fs');
const parser = require('@babel/parser');
const traverse = require('@babel/traverse').default;
const t = require('@babel/types');
const generate = require('@babel/generator').default;
const net = require('net');
// Parse command line arguments
let mode = 'cli'; // 'cli' or 'server'
let socketPath = null;
let filePath = null;
let jsonOutput = false;
for (let i = 2; i < process.argv.length; i++) {
const arg = process.argv[i];
if (arg.startsWith('--socket=')) {
mode = 'server';
socketPath = arg.substring('--socket='.length);
} else if (arg === '--json') {
jsonOutput = true;
} else if (!filePath) {
filePath = arg;
}
}
// =============================================================================
// SHARED PARSING LOGIC
// =============================================================================
// Error helper for JSON output
function outputError(error, filePath, jsonOutput) {
if (jsonOutput) {
const errorObj = {
status: 'error',
error: {
type: error.constructor.name,
message: error.message,
line: error.loc ? error.loc.line : null,
column: error.loc ? error.loc.column : null,
file: filePath,
context: null,
suggestion: null
}
};
// Add suggestions for common errors
if (error.message.includes('Unexpected token')) {
errorObj.error.suggestion = 'Check JavaScript syntax - may have invalid characters or missing punctuation';
} else if (error.message.includes('Unterminated')) {
errorObj.error.suggestion = 'Check for unclosed strings, comments, or brackets';
}
return errorObj;
} else {
return {
status: 'error',
message: `Parse error: ${error.message}`
};
}
}
// Custom error for structure violations
function structureError(type, message, line, code, filePath) {
return {
status: 'error',
error: {
type: type,
message: message,
line: line,
code: code,
file: filePath
}
};
}
// Helper function to extract decorator information in PHP-compatible compact format
function extractDecorators(decorators) {
if (!decorators || decorators.length === 0) {
return [];
}
return decorators.map(decorator => {
const expr = decorator.expression;
// Handle simple decorators like @readonly
if (t.isIdentifier(expr)) {
return [expr.name, []];
}
// Handle decorators with arguments like @Route('/api')
if (t.isCallExpression(expr)) {
const name = t.isIdentifier(expr.callee) ? expr.callee.name :
t.isMemberExpression(expr.callee) ? getMemberExpressionName(expr.callee) :
'unknown';
const args = expr.arguments.map(arg => extractArgumentValue(arg));
return [name, args];
}
// Handle member expression decorators like @Namespace.Decorator
if (t.isMemberExpression(expr)) {
return [getMemberExpressionName(expr), []];
}
return ['unknown', []];
});
}
// Helper to get full name from member expression
function getMemberExpressionName(node) {
if (t.isIdentifier(node.object) && t.isIdentifier(node.property)) {
return `${node.object.name}.${node.property.name}`;
}
if (t.isIdentifier(node.property)) {
return node.property.name;
}
return 'unknown';
}
// Helper to extract argument values
function extractArgumentValue(node) {
if (t.isStringLiteral(node)) {
return node.value;
}
if (t.isNumericLiteral(node)) {
return node.value;
}
if (t.isBooleanLiteral(node)) {
return node.value;
}
if (t.isNullLiteral(node)) {
return null;
}
if (t.isIdentifier(node)) {
return { identifier: node.name };
}
if (t.isObjectExpression(node)) {
const obj = {};
node.properties.forEach(prop => {
if (t.isObjectProperty(prop) && t.isIdentifier(prop.key)) {
obj[prop.key.name] = extractArgumentValue(prop.value);
}
});
return obj;
}
if (t.isArrayExpression(node)) {
return node.elements.map(el => el ? extractArgumentValue(el) : null);
}
if (t.isTemplateLiteral(node) && node.expressions.length === 0) {
// Simple template literal without expressions
return node.quasis[0].value.raw;
}
// For complex expressions, store the type
return { type: node.type };
}
// Helper to get snippet of code for error messages
function getCodeSnippet(node) {
try {
const generated = generate(node, { compact: true });
let code = generated.code;
// Truncate long code snippets
if (code.length > 60) {
code = code.substring(0, 60) + '...';
}
return code;
} catch (e) {
return node.type;
}
}
// Helper to check if a value is static (no function calls)
function isStaticValue(node) {
if (!node) return false;
// Literals are always static
if (t.isStringLiteral(node) || t.isNumericLiteral(node) ||
t.isBooleanLiteral(node) || t.isNullLiteral(node)) {
return true;
}
// Template literals without expressions are static
if (t.isTemplateLiteral(node) && node.expressions.length === 0) {
return true;
}
// Binary expressions with static operands (e.g., 3 * 365)
if (t.isBinaryExpression(node)) {
return isStaticValue(node.left) && isStaticValue(node.right);
}
// Unary expressions with static operands (e.g., -5, !true)
if (t.isUnaryExpression(node)) {
return isStaticValue(node.argument);
}
// Arrays with all static elements
if (t.isArrayExpression(node)) {
return node.elements.every(el => el === null || isStaticValue(el));
}
// Objects with all static properties
if (t.isObjectExpression(node)) {
return node.properties.every(prop => {
if (t.isObjectProperty(prop)) {
return isStaticValue(prop.value);
}
return false; // Methods are not static
});
}
// Everything else (identifiers, function calls, etc.) is not static
return false;
}
/**
* Helper function to check if a JSDoc comment contains @decorator tag
*/
function hasDecoratorTag(commentValue) {
if (!commentValue) return false;
// Parse JSDoc-style comment
// Look for @decorator anywhere in the comment
const lines = commentValue.split('\n');
for (const line of lines) {
// Remove leading asterisk and whitespace
const cleanLine = line.replace(/^\s*\*\s*/, '').trim();
// Check if line contains @decorator tag
if (cleanLine === '@decorator' || cleanLine.startsWith('@decorator ')) {
return true;
}
}
return false;
}
/**
* Extract JSDoc decorators from comments (e.g., @Instantiatable, @Monoprogenic)
* Returns compact format matching PHP: [[name, [args]], ...]
*/
function extractJSDocDecorators(leadingComments) {
const decorators = [];
if (!leadingComments) return decorators;
for (const comment of leadingComments) {
if (comment.type !== 'CommentBlock') continue;
const lines = comment.value.split('\n');
for (const line of lines) {
// Remove leading asterisk and whitespace
const cleanLine = line.replace(/^\s*\*\s*/, '').trim();
// Match @DecoratorName pattern (capital letter start, no spaces)
const match = cleanLine.match(/^@([A-Z][A-Za-z0-9_]*)\s*$/);
if (match) {
const decoratorName = match[1];
// Store in compact format matching PHP: [name, [args]]
decorators.push([decoratorName, []]);
}
}
}
return decorators;
}
// Helper functions
function getValueType(node) {
if (t.isStringLiteral(node)) return `"${node.value}"`;
if (t.isNumericLiteral(node)) return node.value;
if (t.isBooleanLiteral(node)) return node.value;
if (t.isNullLiteral(node)) return null;
if (t.isIdentifier(node)) return node.name;
if (t.isArrayExpression(node)) return 'array';
if (t.isObjectExpression(node)) return 'object';
if (t.isFunctionExpression(node) || t.isArrowFunctionExpression(node)) return 'function';
return node.type;
}
function getExportType(node) {
if (t.isClassDeclaration(node)) return 'class';
if (t.isFunctionDeclaration(node)) return 'function';
if (t.isIdentifier(node)) return 'identifier';
if (t.isCallExpression(node)) return 'expression';
return node.type;
}
/**
* Parse a single JavaScript file
* @param {string} content - File content
* @param {string} filePath - File path for error reporting
* @param {boolean} jsonOutput - Whether to output JSON format
* @returns {object} Parse result or error object
*/
function parseFileContent(content, filePath, jsonOutput) {
// Parse result object
const result = {
classes: {},
functions: {},
exports: {},
imports: [],
globalFunctions: [], // All global function names
globalConstants: [], // All global const names
functionsWithDecorators: {}, // Global functions that have decorators
instanceMethodsWithDecorators: {} // Instance methods with decorators (for error detection)
};
// Tracking for validation
let hasES6Class = false;
let moduleExportsFound = null;
let codeOutsideAllowed = [];
try {
// Parse with Babel
const ast = parser.parse(content, {
sourceType: 'module',
attachComment: true, // Attach comments to AST nodes for /** @decorator */ detection
plugins: [
'jsx',
'typescript',
'classProperties',
'classPrivateProperties',
'classPrivateMethods',
['decorators', { decoratorsBeforeExport: true }],
'dynamicImport',
'exportDefaultFrom',
'exportNamespaceFrom',
'asyncGenerators',
'objectRestSpread',
'optionalCatchBinding',
'optionalChaining',
'nullishCoalescingOperator',
'classStaticBlock'
]
});
// First pass: Check for classes and track top-level statements
ast.program.body.forEach(node => {
if (t.isClassDeclaration(node) || (t.isExportNamedDeclaration(node) && t.isClassDeclaration(node.declaration))) {
hasES6Class = true;
}
});
// Traverse AST
traverse(ast, {
// Class declarations
ClassDeclaration(path) {
const className = path.node.id.name;
// Extract both ES decorators and JSDoc decorators, merge them
const esDecorators = extractDecorators(path.node.decorators);
const jsdocDecorators = extractJSDocDecorators(path.node.leadingComments);
const allDecorators = [...esDecorators, ...jsdocDecorators];
const classInfo = {
name: className,
extends: path.node.superClass ? path.node.superClass.name : null,
public_instance_methods: {},
public_static_methods: {},
properties: {},
staticProperties: {},
decorators: allDecorators
};
// Extract methods and properties
path.node.body.body.forEach(member => {
if (t.isClassMethod(member)) {
// Check for @decorator in JSDoc comments on the method
let hasDecoratorComment = false;
if (member.leadingComments) {
hasDecoratorComment = member.leadingComments.some(comment => {
if (comment.type !== 'CommentBlock') return false;
return hasDecoratorTag(comment.value);
});
}
// Determine visibility: private if starts with #, otherwise public
const methodName = member.key.name || (t.isPrivateName(member.key) ? '#' + member.key.id.name : 'unknown');
const visibility = methodName.startsWith('#') ? 'private' : 'public';
const methodInfo = {
// PHP-compatible fields
name: methodName,
static: member.static,
visibility: visibility,
line: member.loc ? member.loc.start.line : null,
// JavaScript-specific fields (keep existing)
params: member.params.map(p => {
if (t.isIdentifier(p)) return p.name;
if (t.isRestElement(p) && t.isIdentifier(p.argument)) return '...' + p.argument.name;
if (t.isObjectPattern(p)) return '{object}';
if (t.isArrayPattern(p)) return '[array]';
return p.type;
}),
// PHP-compatible parameters structure
parameters: member.params.map(p => {
const paramInfo = {};
if (t.isIdentifier(p)) {
paramInfo.name = p.name;
} else if (t.isRestElement(p) && t.isIdentifier(p.argument)) {
paramInfo.name = '...' + p.argument.name;
} else if (t.isObjectPattern(p)) {
paramInfo.name = '{object}';
} else if (t.isArrayPattern(p)) {
paramInfo.name = '[array]';
} else {
paramInfo.name = p.type;
}
return paramInfo;
}),
async: member.async,
generator: member.generator,
kind: member.kind,
decorators: extractDecorators(member.decorators),
isDecoratorFunction: hasDecoratorComment
};
if (member.static) {
classInfo.public_static_methods[member.key.name] = methodInfo;
// Track static methods that are decorator functions
if (hasDecoratorComment) {
if (!result.functionsWithDecorators[className]) {
result.functionsWithDecorators[className] = {};
}
result.functionsWithDecorators[className][member.key.name] = {
decorators: [['decorator', []]],
line: member.loc ? member.loc.start.line : null
};
}
} else {
classInfo.public_instance_methods[member.key.name] = methodInfo;
}
} else if (t.isClassProperty(member) || t.isClassPrivateProperty(member)) {
const propName = t.isIdentifier(member.key) ? member.key.name :
t.isPrivateName(member.key) ? '#' + member.key.id.name :
'unknown';
const propInfo = {
name: propName,
static: member.static,
value: member.value ? getValueType(member.value) : null,
decorators: extractDecorators(member.decorators)
};
if (member.static) {
classInfo.staticProperties[propName] = propInfo;
} else {
classInfo.properties[propName] = propInfo;
}
}
});
result.classes[className] = classInfo;
},
// Function declarations
FunctionDeclaration(path) {
if (path.node.id) {
const funcName = path.node.id.name;
// Add to global functions list
result.globalFunctions.push(funcName);
// Check if it has decorators (either @decorator syntax or /** @decorator */ comment)
const decorators = extractDecorators(path.node.decorators);
// Check for @decorator in JSDoc comments
let hasDecoratorComment = false;
if (path.node.leadingComments) {
hasDecoratorComment = path.node.leadingComments.some(comment => {
if (comment.type !== 'CommentBlock') return false;
// Check if JSDoc contains @decorator tag
return hasDecoratorTag(comment.value);
});
}
if (decorators || hasDecoratorComment) {
result.functionsWithDecorators[funcName] = {
decorators: hasDecoratorComment ? [['decorator', []]] : decorators,
line: path.node.loc ? path.node.loc.start.line : null
};
}
result.functions[funcName] = {
name: funcName,
params: path.node.params.map(p => {
if (t.isIdentifier(p)) return p.name;
if (t.isRestElement(p) && t.isIdentifier(p.argument)) return '...' + p.argument.name;
if (t.isObjectPattern(p)) return '{object}';
if (t.isArrayPattern(p)) return '[array]';
return p.type;
}),
async: path.node.async,
generator: path.node.generator,
decorators: decorators
};
}
},
// Variable declarations (check for function expressions)
VariableDeclaration(path) {
// Only check top-level variables (directly in Program body)
if (path.parent && path.parent.type === 'Program') {
path.node.declarations.forEach(decl => {
if (t.isIdentifier(decl.id)) {
const varName = decl.id.name;
// Check if it's a function expression
if (decl.init && (t.isFunctionExpression(decl.init) || t.isArrowFunctionExpression(decl.init))) {
// Add to global functions list
result.globalFunctions.push(varName);
// Check for decorators (function expressions don't support decorators directly)
// But we still track them as functions
result.functions[varName] = {
name: varName,
params: decl.init.params ? decl.init.params.map(p => {
if (t.isIdentifier(p)) return p.name;
if (t.isRestElement(p) && t.isIdentifier(p.argument)) return '...' + p.argument.name;
if (t.isObjectPattern(p)) return '{object}';
if (t.isArrayPattern(p)) return '[array]';
return p.type;
}) : [],
async: decl.init.async || false,
generator: decl.init.generator || false,
decorators: null
};
} else if (path.node.kind === 'const' && decl.init && isStaticValue(decl.init)) {
// Const with static value - track it
result.globalConstants.push(varName);
} else if (!hasES6Class) {
// Non-allowed variable in a file without classes
// (not a function, not a const with static value)
codeOutsideAllowed.push({
line: path.node.loc ? path.node.loc.start.line : null,
code: getCodeSnippet(decl)
});
}
}
});
}
},
// Check for module.exports
AssignmentExpression(path) {
const left = path.node.left;
if (t.isMemberExpression(left)) {
if ((t.isIdentifier(left.object) && left.object.name === 'module') &&
(t.isIdentifier(left.property) && left.property.name === 'exports')) {
moduleExportsFound = path.node.loc ? path.node.loc.start.line : 1;
}
}
},
// Check for exports.something =
MemberExpression(path) {
if (path.parent && t.isAssignmentExpression(path.parent) && path.parent.left === path.node) {
if (t.isIdentifier(path.node.object) && path.node.object.name === 'exports') {
moduleExportsFound = path.node.loc ? path.node.loc.start.line : 1;
}
}
},
// Imports
ImportDeclaration(path) {
const importInfo = {
source: path.node.source.value,
specifiers: []
};
path.node.specifiers.forEach(spec => {
if (t.isImportDefaultSpecifier(spec)) {
importInfo.specifiers.push({
type: 'default',
local: spec.local.name
});
} else if (t.isImportSpecifier(spec)) {
importInfo.specifiers.push({
type: 'named',
imported: spec.imported.name,
local: spec.local.name
});
} else if (t.isImportNamespaceSpecifier(spec)) {
importInfo.specifiers.push({
type: 'namespace',
local: spec.local.name
});
}
});
result.imports.push(importInfo);
},
// Exports
ExportNamedDeclaration(path) {
if (path.node.declaration) {
if (t.isClassDeclaration(path.node.declaration) && path.node.declaration.id) {
result.exports[path.node.declaration.id.name] = 'class';
} else if (t.isFunctionDeclaration(path.node.declaration) && path.node.declaration.id) {
result.exports[path.node.declaration.id.name] = 'function';
} else if (t.isVariableDeclaration(path.node.declaration)) {
path.node.declaration.declarations.forEach(decl => {
if (t.isIdentifier(decl.id)) {
result.exports[decl.id.name] = 'variable';
}
});
}
}
// Handle export specifiers
if (path.node.specifiers) {
path.node.specifiers.forEach(spec => {
if (t.isExportSpecifier(spec)) {
result.exports[spec.exported.name] = 'named';
}
});
}
},
ExportDefaultDeclaration(path) {
result.exports.default = getExportType(path.node.declaration);
},
Program: {
exit(path) {
// After traversal, check structure violations
// Check for module.exports
if (moduleExportsFound) {
const error = structureError(
'ModuleExportsFound',
'Module exports detected. JavaScript files are concatenated, use direct class references.',
moduleExportsFound,
null,
filePath
);
throw error;
}
// Check if this is a compiled/generated file (not originally a .js file)
// These files are generated from other sources (.jqhtml, etc) and should be exempt
const firstLine = content.split('\n')[0];
if (firstLine && firstLine.includes('Compiled from:')) {
// This is a compiled file, skip all structure validation
return;
}
// TEMPORARY: Exempt specific utility files from strict validation
// TODO: Replace this with a proper @RULE comment-based system
const fileName = filePath.split('/').pop();
const exemptFiles = ['functions.js'];
if (exemptFiles.includes(fileName)) {
// Skip validation for these files temporarily
return;
}
// Check structure based on whether file has ES6 class
if (hasES6Class) {
// Files with classes should only have class and comments
let invalidCode = null;
path.node.body.forEach(node => {
// Allow comments (handled by parser)
if (t.isClassDeclaration(node)) {
// Class is allowed
return;
}
if (t.isExportNamedDeclaration(node) && t.isClassDeclaration(node.declaration)) {
// Exported class is allowed
return;
}
if (t.isExportDefaultDeclaration(node) && t.isClassExpression(node.declaration)) {
// Export default class is allowed
return;
}
if (t.isImportDeclaration(node)) {
// Imports are allowed (they're removed during bundling)
return;
}
// Anything else is not allowed
if (!invalidCode) {
invalidCode = {
line: node.loc ? node.loc.start.line : null,
code: getCodeSnippet(node)
};
}
});
if (invalidCode) {
const error = structureError(
'CodeOutsideClass',
'JavaScript files with classes may only contain one class declaration and comments.',
invalidCode.line,
invalidCode.code,
filePath
);
throw error;
}
} else {
// Files without classes should only have functions and comments
let invalidCode = null;
path.node.body.forEach(node => {
// Allow function declarations
if (t.isFunctionDeclaration(node)) {
return;
}
// Allow variable declarations that are functions or static const values
if (t.isVariableDeclaration(node)) {
const allAllowed = node.declarations.every(decl => {
// Functions are always allowed
if (decl.init && (
t.isFunctionExpression(decl.init) ||
t.isArrowFunctionExpression(decl.init)
)) {
return true;
}
// Const declarations with static values are allowed
if (node.kind === 'const' && decl.init && isStaticValue(decl.init)) {
return true;
}
return false;
});
if (allAllowed) {
return;
}
}
// Allow imports (they're removed during bundling)
if (t.isImportDeclaration(node)) {
return;
}
// Allow exports that wrap functions
if (t.isExportNamedDeclaration(node)) {
if (t.isFunctionDeclaration(node.declaration)) {
return;
}
if (!node.declaration && node.specifiers) {
// Export of existing identifiers
return;
}
}
// Anything else is not allowed
if (!invalidCode) {
invalidCode = {
line: node.loc ? node.loc.start.line : null,
code: getCodeSnippet(node)
};
}
});
if (invalidCode) {
const error = structureError(
'CodeOutsideAllowed',
'JavaScript files without classes may only contain function declarations, const variables with static values, and comments.',
invalidCode.line,
invalidCode.code,
filePath
);
throw error;
}
}
}
}
});
return {
status: 'success',
result: result,
file: filePath
};
} catch (error) {
// If error is already a structure error object, return it
if (error.status === 'error') {
return error;
}
// Parse Babel error location if available
if (!error.loc && error.message) {
// Try to extract from message (e.g., "Unexpected token (10:5)")
const match = error.message.match(/\((\d+):(\d+)\)/);
if (match) {
error.loc = {
line: parseInt(match[1]),
column: parseInt(match[2])
};
}
}
return outputError(error, filePath, jsonOutput);
}
}
// =============================================================================
// MODE HANDLING: CLI or RPC Server
// =============================================================================
if (mode === 'server') {
// RPC Server Mode
if (!socketPath) {
console.error('Server mode requires --socket=/path/to/socket');
process.exit(1);
}
// Remove socket if exists
if (fs.existsSync(socketPath)) {
fs.unlinkSync(socketPath);
}
function handleRequest(data) {
try {
const request = JSON.parse(data);
switch (request.method) {
case 'ping':
return JSON.stringify({
id: request.id,
result: 'pong'
}) + '\n';
case 'parse':
const results = {};
for (const file of request.files) {
try {
const content = fs.readFileSync(file, 'utf8');
const parseResult = parseFileContent(content, file, true);
results[file] = parseResult;
} catch (error) {
results[file] = {
status: 'error',
error: {
type: 'FileReadError',
message: error.message
}
};
}
}
return JSON.stringify({
id: request.id,
results: results
}) + '\n';
case 'shutdown':
return JSON.stringify({
id: request.id,
result: 'shutting down'
}) + '\n';
default:
return JSON.stringify({
id: request.id,
error: 'Unknown method: ' + request.method
}) + '\n';
}
} catch (error) {
return JSON.stringify({
error: 'Invalid JSON request: ' + error.message
}) + '\n';
}
}
const server = net.createServer((socket) => {
let buffer = '';
socket.on('data', (data) => {
buffer += data.toString();
let newlineIndex;
while ((newlineIndex = buffer.indexOf('\n')) !== -1) {
const line = buffer.substring(0, newlineIndex);
buffer = buffer.substring(newlineIndex + 1);
if (line.trim()) {
const response = handleRequest(line);
socket.write(response);
try {
const request = JSON.parse(line);
if (request.method === 'shutdown') {
socket.end();
server.close(() => {
if (fs.existsSync(socketPath)) {
fs.unlinkSync(socketPath);
}
process.exit(0);
});
}
} catch (e) {
// Ignore
}
}
}
});
socket.on('error', (err) => {
console.error('Socket error:', err);
});
});
server.listen(socketPath, () => {
console.log('JS Parser RPC server listening on ' + socketPath);
});
server.on('error', (err) => {
console.error('Server error:', err);
process.exit(1);
});
process.on('SIGTERM', () => {
server.close(() => {
if (fs.existsSync(socketPath)) {
fs.unlinkSync(socketPath);
}
process.exit(0);
});
});
process.on('SIGINT', () => {
server.close(() => {
if (fs.existsSync(socketPath)) {
fs.unlinkSync(socketPath);
}
process.exit(0);
});
});
} else {
// CLI Mode
if (!filePath) {
console.error('Usage: node js-parser-server.js [--json] <file>');
console.error(' or: node js-parser-server.js --socket=/path/to/socket');
process.exit(1);
}
// Read file
let content;
try {
content = fs.readFileSync(filePath, 'utf8');
} catch (error) {
if (jsonOutput) {
console.log(JSON.stringify({
status: 'error',
error: {
type: 'FileReadError',
message: `Error reading file: ${error.message}`,
file: filePath
}
}));
} else {
console.error(`Error reading file: ${error.message}`);
}
process.exit(1);
}
// Parse file
const result = parseFileContent(content, filePath, jsonOutput);
// Output result
if (jsonOutput) {
console.log(JSON.stringify(result));
} else {
if (result.status === 'error') {
console.error('Parse error:', result.error || result.message);
process.exit(1);
} else {
console.log(JSON.stringify(result.result, null, 2));
}
}
}

View File

@@ -1,815 +0,0 @@
const fs = require('fs');
const parser = require('@babel/parser');
const traverse = require('@babel/traverse').default;
const t = require('@babel/types');
const generate = require('@babel/generator').default;
// Parse command line arguments
let filePath = null;
let jsonOutput = false;
for (let i = 2; i < process.argv.length; i++) {
const arg = process.argv[i];
if (arg === '--json') {
jsonOutput = true;
} else if (!filePath) {
filePath = arg;
}
}
// Error helper for JSON output
function outputError(error) {
if (jsonOutput) {
const errorObj = {
status: 'error',
error: {
type: error.constructor.name,
message: error.message,
line: error.loc ? error.loc.line : null,
column: error.loc ? error.loc.column : null,
file: filePath,
context: null,
suggestion: null
}
};
// Add suggestions for common errors
if (error.message.includes('Unexpected token')) {
errorObj.error.suggestion = 'Check JavaScript syntax - may have invalid characters or missing punctuation';
} else if (error.message.includes('Unterminated')) {
errorObj.error.suggestion = 'Check for unclosed strings, comments, or brackets';
}
console.log(JSON.stringify(errorObj));
} else {
console.error(`Parse error: ${error.message}`);
}
}
// Custom error for structure violations
function structureError(type, message, line, code = null) {
if (jsonOutput) {
console.log(JSON.stringify({
status: 'error',
error: {
type: type,
message: message,
line: line,
code: code,
file: filePath
}
}));
} else {
console.error(`${type}: ${message} at line ${line}`);
if (code) {
console.error(` Code: ${code}`);
}
}
process.exit(1);
}
if (!filePath) {
if (jsonOutput) {
console.log(JSON.stringify({
status: 'error',
error: {
type: 'ArgumentError',
message: 'No input file specified',
suggestion: 'Usage: node js-parser.js [--json] <file-path>'
}
}));
} else {
console.error('Usage: node js-parser.js [--json] <file-path>');
}
process.exit(1);
}
// Read file
let content;
try {
content = fs.readFileSync(filePath, 'utf8');
} catch (error) {
error.message = `Error reading file: ${error.message}`;
outputError(error);
process.exit(1);
}
// Parse result object
const result = {
classes: {},
functions: {},
exports: {},
imports: [],
globalFunctions: [], // All global function names
globalConstants: [], // All global const names
functionsWithDecorators: {}, // Global functions that have decorators
instanceMethodsWithDecorators: {} // Instance methods with decorators (for error detection)
};
// Tracking for validation
let hasES6Class = false;
let moduleExportsFound = null;
let codeOutsideAllowed = [];
// Helper function to extract decorator information in PHP-compatible compact format
function extractDecorators(decorators) {
if (!decorators || decorators.length === 0) {
return [];
}
return decorators.map(decorator => {
const expr = decorator.expression;
// Handle simple decorators like @readonly
if (t.isIdentifier(expr)) {
return [expr.name, []];
}
// Handle decorators with arguments like @Route('/api')
if (t.isCallExpression(expr)) {
const name = t.isIdentifier(expr.callee) ? expr.callee.name :
t.isMemberExpression(expr.callee) ? getMemberExpressionName(expr.callee) :
'unknown';
const args = expr.arguments.map(arg => extractArgumentValue(arg));
return [name, args];
}
// Handle member expression decorators like @Namespace.Decorator
if (t.isMemberExpression(expr)) {
return [getMemberExpressionName(expr), []];
}
return ['unknown', []];
});
}
// Helper to get full name from member expression
function getMemberExpressionName(node) {
if (t.isIdentifier(node.object) && t.isIdentifier(node.property)) {
return `${node.object.name}.${node.property.name}`;
}
if (t.isIdentifier(node.property)) {
return node.property.name;
}
return 'unknown';
}
// Helper to extract argument values
function extractArgumentValue(node) {
if (t.isStringLiteral(node)) {
return node.value;
}
if (t.isNumericLiteral(node)) {
return node.value;
}
if (t.isBooleanLiteral(node)) {
return node.value;
}
if (t.isNullLiteral(node)) {
return null;
}
if (t.isIdentifier(node)) {
return { identifier: node.name };
}
if (t.isObjectExpression(node)) {
const obj = {};
node.properties.forEach(prop => {
if (t.isObjectProperty(prop) && t.isIdentifier(prop.key)) {
obj[prop.key.name] = extractArgumentValue(prop.value);
}
});
return obj;
}
if (t.isArrayExpression(node)) {
return node.elements.map(el => el ? extractArgumentValue(el) : null);
}
if (t.isTemplateLiteral(node) && node.expressions.length === 0) {
// Simple template literal without expressions
return node.quasis[0].value.raw;
}
// For complex expressions, store the type
return { type: node.type };
}
// Helper to get snippet of code for error messages
function getCodeSnippet(node) {
try {
const generated = generate(node, { compact: true });
let code = generated.code;
// Truncate long code snippets
if (code.length > 60) {
code = code.substring(0, 60) + '...';
}
return code;
} catch (e) {
return node.type;
}
}
// Helper to check if a value is static (no function calls)
function isStaticValue(node) {
if (!node) return false;
// Literals are always static
if (t.isStringLiteral(node) || t.isNumericLiteral(node) ||
t.isBooleanLiteral(node) || t.isNullLiteral(node)) {
return true;
}
// Template literals without expressions are static
if (t.isTemplateLiteral(node) && node.expressions.length === 0) {
return true;
}
// Binary expressions with static operands (e.g., 3 * 365)
if (t.isBinaryExpression(node)) {
return isStaticValue(node.left) && isStaticValue(node.right);
}
// Unary expressions with static operands (e.g., -5, !true)
if (t.isUnaryExpression(node)) {
return isStaticValue(node.argument);
}
// Arrays with all static elements
if (t.isArrayExpression(node)) {
return node.elements.every(el => el === null || isStaticValue(el));
}
// Objects with all static properties
if (t.isObjectExpression(node)) {
return node.properties.every(prop => {
if (t.isObjectProperty(prop)) {
return isStaticValue(prop.value);
}
return false; // Methods are not static
});
}
// Everything else (identifiers, function calls, etc.) is not static
return false;
}
/**
* Helper function to check if a JSDoc comment contains @decorator tag
*/
function hasDecoratorTag(commentValue) {
if (!commentValue) return false;
// Parse JSDoc-style comment
// Look for @decorator anywhere in the comment
const lines = commentValue.split('\n');
for (const line of lines) {
// Remove leading asterisk and whitespace
const cleanLine = line.replace(/^\s*\*\s*/, '').trim();
// Check if line contains @decorator tag
if (cleanLine === '@decorator' || cleanLine.startsWith('@decorator ')) {
return true;
}
}
return false;
}
/**
* Extract JSDoc decorators from comments (e.g., @Instantiatable, @Monoprogenic)
* Returns compact format matching PHP: [[name, [args]], ...]
*/
function extractJSDocDecorators(leadingComments) {
const decorators = [];
if (!leadingComments) return decorators;
for (const comment of leadingComments) {
if (comment.type !== 'CommentBlock') continue;
const lines = comment.value.split('\n');
for (const line of lines) {
// Remove leading asterisk and whitespace
const cleanLine = line.replace(/^\s*\*\s*/, '').trim();
// Match @DecoratorName pattern (capital letter start, no spaces)
const match = cleanLine.match(/^@([A-Z][A-Za-z0-9_]*)\s*$/);
if (match) {
const decoratorName = match[1];
// Store in compact format matching PHP: [name, [args]]
decorators.push([decoratorName, []]);
}
}
}
return decorators;
}
try {
// No preprocessing needed - parse content directly
const processedContent = content;
// Parse with Babel
const ast = parser.parse(processedContent, {
sourceType: 'module',
attachComment: true, // Attach comments to AST nodes for /** @decorator */ detection
plugins: [
'jsx',
'typescript',
'classProperties',
'classPrivateProperties',
'classPrivateMethods',
['decorators', { decoratorsBeforeExport: true }],
'dynamicImport',
'exportDefaultFrom',
'exportNamespaceFrom',
'asyncGenerators',
'objectRestSpread',
'optionalCatchBinding',
'optionalChaining',
'nullishCoalescingOperator',
'classStaticBlock'
]
});
// First pass: Check for classes and track top-level statements
ast.program.body.forEach(node => {
if (t.isClassDeclaration(node) || (t.isExportNamedDeclaration(node) && t.isClassDeclaration(node.declaration))) {
hasES6Class = true;
}
});
// Traverse AST
traverse(ast, {
// Class declarations
ClassDeclaration(path) {
const className = path.node.id.name;
// Extract both ES decorators and JSDoc decorators, merge them
const esDecorators = extractDecorators(path.node.decorators);
const jsdocDecorators = extractJSDocDecorators(path.node.leadingComments);
const allDecorators = [...esDecorators, ...jsdocDecorators];
const classInfo = {
name: className,
extends: path.node.superClass ? path.node.superClass.name : null,
public_instance_methods: {},
public_static_methods: {},
properties: {},
staticProperties: {},
decorators: allDecorators
};
// Extract methods and properties
path.node.body.body.forEach(member => {
if (t.isClassMethod(member)) {
// Check for @decorator in JSDoc comments on the method
let hasDecoratorComment = false;
if (member.leadingComments) {
hasDecoratorComment = member.leadingComments.some(comment => {
if (comment.type !== 'CommentBlock') return false;
return hasDecoratorTag(comment.value);
});
}
// Determine visibility: private if starts with #, otherwise public
const methodName = member.key.name || (t.isPrivateName(member.key) ? '#' + member.key.id.name : 'unknown');
const visibility = methodName.startsWith('#') ? 'private' : 'public';
const methodInfo = {
// PHP-compatible fields
name: methodName,
static: member.static,
visibility: visibility,
line: member.loc ? member.loc.start.line : null,
// JavaScript-specific fields (keep existing)
params: member.params.map(p => {
if (t.isIdentifier(p)) return p.name;
if (t.isRestElement(p) && t.isIdentifier(p.argument)) return '...' + p.argument.name;
if (t.isObjectPattern(p)) return '{object}';
if (t.isArrayPattern(p)) return '[array]';
return p.type;
}),
// PHP-compatible parameters structure
parameters: member.params.map(p => {
const paramInfo = {};
if (t.isIdentifier(p)) {
paramInfo.name = p.name;
} else if (t.isRestElement(p) && t.isIdentifier(p.argument)) {
paramInfo.name = '...' + p.argument.name;
} else if (t.isObjectPattern(p)) {
paramInfo.name = '{object}';
} else if (t.isArrayPattern(p)) {
paramInfo.name = '[array]';
} else {
paramInfo.name = p.type;
}
return paramInfo;
}),
async: member.async,
generator: member.generator,
kind: member.kind,
decorators: extractDecorators(member.decorators),
isDecoratorFunction: hasDecoratorComment
};
if (member.static) {
classInfo.public_static_methods[member.key.name] = methodInfo;
// Track static methods that are decorator functions
if (hasDecoratorComment) {
if (!result.functionsWithDecorators[className]) {
result.functionsWithDecorators[className] = {};
}
result.functionsWithDecorators[className][member.key.name] = {
decorators: [['decorator', []]],
line: member.loc ? member.loc.start.line : null
};
}
} else {
classInfo.public_instance_methods[member.key.name] = methodInfo;
}
} else if (t.isClassProperty(member) || t.isClassPrivateProperty(member)) {
const propName = t.isIdentifier(member.key) ? member.key.name :
t.isPrivateName(member.key) ? '#' + member.key.id.name :
'unknown';
const propInfo = {
name: propName,
static: member.static,
value: member.value ? getValueType(member.value) : null,
decorators: extractDecorators(member.decorators)
};
if (member.static) {
classInfo.staticProperties[propName] = propInfo;
} else {
classInfo.properties[propName] = propInfo;
}
}
});
result.classes[className] = classInfo;
},
// Function declarations
FunctionDeclaration(path) {
if (path.node.id) {
const funcName = path.node.id.name;
// Add to global functions list
result.globalFunctions.push(funcName);
// Check if it has decorators (either @decorator syntax or /** @decorator */ comment)
const decorators = extractDecorators(path.node.decorators);
// Check for @decorator in JSDoc comments
let hasDecoratorComment = false;
if (path.node.leadingComments) {
hasDecoratorComment = path.node.leadingComments.some(comment => {
if (comment.type !== 'CommentBlock') return false;
// Check if JSDoc contains @decorator tag
return hasDecoratorTag(comment.value);
});
}
if (decorators || hasDecoratorComment) {
result.functionsWithDecorators[funcName] = {
decorators: hasDecoratorComment ? [['decorator', []]] : decorators,
line: path.node.loc ? path.node.loc.start.line : null
};
}
result.functions[funcName] = {
name: funcName,
params: path.node.params.map(p => {
if (t.isIdentifier(p)) return p.name;
if (t.isRestElement(p) && t.isIdentifier(p.argument)) return '...' + p.argument.name;
if (t.isObjectPattern(p)) return '{object}';
if (t.isArrayPattern(p)) return '[array]';
return p.type;
}),
async: path.node.async,
generator: path.node.generator,
decorators: decorators
};
}
},
// Variable declarations (check for function expressions)
VariableDeclaration(path) {
// Only check top-level variables (directly in Program body)
if (path.parent && path.parent.type === 'Program') {
path.node.declarations.forEach(decl => {
if (t.isIdentifier(decl.id)) {
const varName = decl.id.name;
// Check if it's a function expression
if (decl.init && (t.isFunctionExpression(decl.init) || t.isArrowFunctionExpression(decl.init))) {
// Add to global functions list
result.globalFunctions.push(varName);
// Check for decorators (function expressions don't support decorators directly)
// But we still track them as functions
result.functions[varName] = {
name: varName,
params: decl.init.params ? decl.init.params.map(p => {
if (t.isIdentifier(p)) return p.name;
if (t.isRestElement(p) && t.isIdentifier(p.argument)) return '...' + p.argument.name;
if (t.isObjectPattern(p)) return '{object}';
if (t.isArrayPattern(p)) return '[array]';
return p.type;
}) : [],
async: decl.init.async || false,
generator: decl.init.generator || false,
decorators: null
};
} else if (path.node.kind === 'const' && decl.init && isStaticValue(decl.init)) {
// Const with static value - track it
result.globalConstants.push(varName);
} else if (!hasES6Class) {
// Non-allowed variable in a file without classes
// (not a function, not a const with static value)
codeOutsideAllowed.push({
line: path.node.loc ? path.node.loc.start.line : null,
code: getCodeSnippet(decl)
});
}
}
});
}
},
// Check for module.exports
AssignmentExpression(path) {
const left = path.node.left;
if (t.isMemberExpression(left)) {
if ((t.isIdentifier(left.object) && left.object.name === 'module') &&
(t.isIdentifier(left.property) && left.property.name === 'exports')) {
moduleExportsFound = path.node.loc ? path.node.loc.start.line : 1;
}
}
},
// Check for exports.something =
MemberExpression(path) {
if (path.parent && t.isAssignmentExpression(path.parent) && path.parent.left === path.node) {
if (t.isIdentifier(path.node.object) && path.node.object.name === 'exports') {
moduleExportsFound = path.node.loc ? path.node.loc.start.line : 1;
}
}
},
// Imports
ImportDeclaration(path) {
const importInfo = {
source: path.node.source.value,
specifiers: []
};
path.node.specifiers.forEach(spec => {
if (t.isImportDefaultSpecifier(spec)) {
importInfo.specifiers.push({
type: 'default',
local: spec.local.name
});
} else if (t.isImportSpecifier(spec)) {
importInfo.specifiers.push({
type: 'named',
imported: spec.imported.name,
local: spec.local.name
});
} else if (t.isImportNamespaceSpecifier(spec)) {
importInfo.specifiers.push({
type: 'namespace',
local: spec.local.name
});
}
});
result.imports.push(importInfo);
},
// Exports
ExportNamedDeclaration(path) {
if (path.node.declaration) {
if (t.isClassDeclaration(path.node.declaration) && path.node.declaration.id) {
result.exports[path.node.declaration.id.name] = 'class';
} else if (t.isFunctionDeclaration(path.node.declaration) && path.node.declaration.id) {
result.exports[path.node.declaration.id.name] = 'function';
} else if (t.isVariableDeclaration(path.node.declaration)) {
path.node.declaration.declarations.forEach(decl => {
if (t.isIdentifier(decl.id)) {
result.exports[decl.id.name] = 'variable';
}
});
}
}
// Handle export specifiers
if (path.node.specifiers) {
path.node.specifiers.forEach(spec => {
if (t.isExportSpecifier(spec)) {
result.exports[spec.exported.name] = 'named';
}
});
}
},
ExportDefaultDeclaration(path) {
result.exports.default = getExportType(path.node.declaration);
},
Program: {
exit(path) {
// After traversal, check structure violations
// Check for module.exports
if (moduleExportsFound) {
structureError(
'ModuleExportsFound',
'Module exports detected. JavaScript files are concatenated, use direct class references.',
moduleExportsFound,
null
);
}
// Check if this is a compiled/generated file (not originally a .js file)
// These files are generated from other sources (.jqhtml, etc) and should be exempt
const fileContent = fs.readFileSync(filePath, 'utf8');
const firstLine = fileContent.split('\n')[0];
if (firstLine && firstLine.includes('Compiled from:')) {
// This is a compiled file, skip all structure validation
return;
}
// TEMPORARY: Exempt specific utility files from strict validation
// TODO: Replace this with a proper @RULE comment-based system
const fileName = filePath.split('/').pop();
const exemptFiles = ['functions.js'];
if (exemptFiles.includes(fileName)) {
// Skip validation for these files temporarily
return;
}
// Check structure based on whether file has ES6 class
if (hasES6Class) {
// Files with classes should only have class and comments
let invalidCode = null;
path.node.body.forEach(node => {
// Allow comments (handled by parser)
if (t.isClassDeclaration(node)) {
// Class is allowed
return;
}
if (t.isExportNamedDeclaration(node) && t.isClassDeclaration(node.declaration)) {
// Exported class is allowed
return;
}
if (t.isExportDefaultDeclaration(node) && t.isClassExpression(node.declaration)) {
// Export default class is allowed
return;
}
if (t.isImportDeclaration(node)) {
// Imports are allowed (they're removed during bundling)
return;
}
// Anything else is not allowed
if (!invalidCode) {
invalidCode = {
line: node.loc ? node.loc.start.line : null,
code: getCodeSnippet(node)
};
}
});
if (invalidCode) {
structureError(
'CodeOutsideClass',
'JavaScript files with classes may only contain one class declaration and comments.',
invalidCode.line,
invalidCode.code
);
}
} else {
// Files without classes should only have functions and comments
let invalidCode = null;
path.node.body.forEach(node => {
// Allow function declarations
if (t.isFunctionDeclaration(node)) {
return;
}
// Allow variable declarations that are functions or static const values
if (t.isVariableDeclaration(node)) {
const allAllowed = node.declarations.every(decl => {
// Functions are always allowed
if (decl.init && (
t.isFunctionExpression(decl.init) ||
t.isArrowFunctionExpression(decl.init)
)) {
return true;
}
// Const declarations with static values are allowed
if (node.kind === 'const' && decl.init && isStaticValue(decl.init)) {
return true;
}
return false;
});
if (allAllowed) {
return;
}
}
// Allow imports (they're removed during bundling)
if (t.isImportDeclaration(node)) {
return;
}
// Allow exports that wrap functions
if (t.isExportNamedDeclaration(node)) {
if (t.isFunctionDeclaration(node.declaration)) {
return;
}
if (!node.declaration && node.specifiers) {
// Export of existing identifiers
return;
}
}
// Anything else is not allowed
if (!invalidCode) {
invalidCode = {
line: node.loc ? node.loc.start.line : null,
code: getCodeSnippet(node)
};
}
});
if (invalidCode) {
structureError(
'CodeOutsideAllowed',
'JavaScript files without classes may only contain function declarations, const variables with static values, and comments.',
invalidCode.line,
invalidCode.code
);
}
}
}
}
});
} catch (error) {
// Parse Babel error location if available
if (!error.loc && error.message) {
// Try to extract from message (e.g., "Unexpected token (10:5)")
const match = error.message.match(/\((\d+):(\d+)\)/);
if (match) {
error.loc = {
line: parseInt(match[1]),
column: parseInt(match[2])
};
}
}
outputError(error);
process.exit(1);
}
// Helper functions
function getValueType(node) {
if (t.isStringLiteral(node)) return `"${node.value}"`;
if (t.isNumericLiteral(node)) return node.value;
if (t.isBooleanLiteral(node)) return node.value;
if (t.isNullLiteral(node)) return null;
if (t.isIdentifier(node)) return node.name;
if (t.isArrayExpression(node)) return 'array';
if (t.isObjectExpression(node)) return 'object';
if (t.isFunctionExpression(node) || t.isArrowFunctionExpression(node)) return 'function';
return node.type;
}
function getExportType(node) {
if (t.isClassDeclaration(node)) return 'class';
if (t.isFunctionDeclaration(node)) return 'function';
if (t.isIdentifier(node)) return 'identifier';
if (t.isCallExpression(node)) return 'expression';
return node.type;
}
// Output result
if (jsonOutput) {
console.log(JSON.stringify({
status: 'success',
result: result,
file: filePath
}));
} else {
console.log(JSON.stringify(result, null, 2));
}

View File

@@ -0,0 +1,470 @@
#!/usr/bin/env node
/**
* JavaScript Transformer RPC Server (Babel)
*
* This script transforms modern JavaScript (decorators, class properties) to browser-compatible code.
*
* Usage:
* CLI mode: node js-transformer-server.js [--json] <file-path> [target] [hash-path]
* Server mode: node js-transformer-server.js --socket=/path/to/socket
*/
const fs = require('fs');
const path = require('path');
const crypto = require('crypto');
const babel = require('@babel/core');
const net = require('net');
// Parse command line arguments
let mode = 'cli'; // 'cli' or 'server'
let socketPath = null;
let filePath = null;
let target = 'modern';
let hashPath = null;
let jsonOutput = false;
// Process arguments
for (let i = 2; i < process.argv.length; i++) {
const arg = process.argv[i];
if (arg.startsWith('--socket=')) {
mode = 'server';
socketPath = arg.substring('--socket='.length);
} else if (arg === '--json') {
jsonOutput = true;
} else if (!filePath) {
filePath = arg;
} else if (!target || target === 'modern') {
target = arg;
} else if (!hashPath) {
hashPath = arg;
}
}
// =============================================================================
// SHARED TRANSFORMATION LOGIC
// =============================================================================
// Error helper for JSON output
function outputError(error, filePath, jsonOutput) {
const errorObj = {
status: 'error',
error: {
type: error.constructor.name,
message: error.message,
line: error.loc ? error.loc.line : null,
column: error.loc ? error.loc.column : null,
file: filePath,
context: null,
suggestion: null
}
};
// Add suggestions for common errors
if (error.message.includes('Cannot find module')) {
errorObj.error.suggestion = 'Missing Babel dependencies. Run: npm install';
} else if (error.message.includes('Unexpected token')) {
errorObj.error.suggestion = 'Check JavaScript syntax in the source file';
} else if (error.message.includes('decorator')) {
errorObj.error.suggestion = 'Ensure decorators are properly formatted (e.g., @decorator before class/method)';
}
if (jsonOutput) {
return errorObj;
} else {
return {
status: 'error',
message: `Transformation error: ${error.message}`
};
}
}
/**
* Preprocessor to handle @decorator on standalone functions
* Converts @decorator to decorator comment when no ES6 classes are present
*/
function preprocessDecorators(content, filePath) {
// Check if file contains ES6 class declarations
// Using regex to avoid parsing errors from decorators
const es6ClassRegex = /^\s*class\s+[A-Z]\w*\s*(?:extends\s+\w+\s*)?\{/m;
const hasES6Class = es6ClassRegex.test(content);
if (hasES6Class) {
// File has ES6 classes, leave @decorator syntax unchanged
return content;
}
// No ES6 classes, convert @decorator to /** @decorator */
// Match @decorator at the start of a line (with optional whitespace)
// that appears before a function declaration
const decoratorRegex = /^(\s*)@decorator\s*\n(\s*(?:async\s+)?function\s+\w+)/gm;
const processed = content.replace(decoratorRegex, (match, indent, funcDecl) => {
return `${indent}/** @decorator */\n${funcDecl}`;
});
return processed;
}
// Target environment presets
const targetPresets = {
modern: {
targets: {
chrome: '90',
firefox: '88',
safari: '14',
edge: '90'
}
},
es6: {
targets: {
chrome: '60',
firefox: '60',
safari: '10.1',
edge: '15'
}
},
es5: {
targets: {
ie: '11'
}
}
};
/**
* Create custom plugin to prefix generated WeakMap variables and Babel helper functions
* This plugin runs AFTER all other transformations to catch Babel-generated helpers
*/
function createPrefixPlugin(fileHash) {
return function() {
return {
name: 'prefix-generated-variables',
post(file) {
// Run after all transformations are complete
const program = file.path;
// Track all top-level variables and functions that start with underscore
const generatedNames = new Set();
// First pass: collect all generated variable and function names at top level
for (const statement of program.node.body) {
if (statement.type === 'VariableDeclaration') {
for (const declarator of statement.declarations) {
const name = declarator.id?.name;
if (name && name.startsWith('_')) {
generatedNames.add(name);
}
}
} else if (statement.type === 'FunctionDeclaration') {
const name = statement.id?.name;
if (name && name.startsWith('_')) {
generatedNames.add(name);
}
}
}
// Second pass: rename all references
if (generatedNames.size > 0) {
program.traverse({
Identifier(idPath) {
if (generatedNames.has(idPath.node.name)) {
// Don't rename if it's already prefixed
if (!idPath.node.name.startsWith(`_${fileHash}`)) {
const newName = `_${fileHash}${idPath.node.name}`;
idPath.scope.rename(idPath.node.name, newName);
}
}
}
});
}
}
};
};
}
/**
* Transform a single JavaScript file
* @param {string} content - File content
* @param {string} filePath - File path for source mapping
* @param {string} target - Target environment (modern, es6, es5)
* @param {string} hashPath - Path to use for hash generation
* @param {boolean} jsonOutput - Whether to output JSON format
* @returns {object} Transform result or error object
*/
function transformFileContent(content, filePath, target, hashPath, jsonOutput) {
// Preprocess content before transformation
content = preprocessDecorators(content, filePath);
// Generate file hash for prefixing (HARDCODED - NOT CONFIGURABLE)
// This prevents namespace collisions when files are concatenated in bundles
const fileHash = crypto.createHash('md5')
.update(hashPath)
.digest('hex')
.substring(0, 8);
try {
// Configure Babel transformation
// Use relative path for sourcemap to match SCSS behavior
const relativeFilePath = path.relative(process.cwd(), filePath);
const result = babel.transformSync(content, {
filename: relativeFilePath,
sourceFileName: relativeFilePath, // Explicitly set source filename for sourcemap
sourceMaps: 'inline',
presets: [
['@babel/preset-env', targetPresets[target] || targetPresets.modern]
],
plugins: [
// Apply custom prefixing plugin first
createPrefixPlugin(fileHash),
// Transform decorators (Stage 3 proposal)
// Note: We're NOT transforming private fields - native support only
['@babel/plugin-proposal-decorators', {
version: '2023-11',
// Ensure decorators are transpiled to compatible code
}],
// Transform class properties
'@babel/plugin-transform-class-properties',
// Transform optional chaining and nullish coalescing
'@babel/plugin-transform-optional-chaining',
'@babel/plugin-transform-nullish-coalescing-operator'
]
});
if (!result || !result.code) {
const error = new Error('Babel transformation produced no output');
return outputError(error, filePath, jsonOutput);
}
// Output result (no banner - concat-js.js handles that)
return {
status: 'success',
result: result.code,
file: filePath,
hash: fileHash
};
} catch (error) {
// Parse Babel error location if available
if (error.loc) {
// Babel provides loc.line and loc.column
} else if (error.codeFrame) {
// Try to extract line/column from codeFrame
const lineMatch = error.codeFrame.match(/>\s*(\d+)\s*\|/);
const colMatch = error.codeFrame.match(/\n\s+\|\s+(\^+)/);
if (lineMatch) {
error.loc = {
line: parseInt(lineMatch[1]),
column: colMatch ? colMatch[1].indexOf('^') + 1 : 0
};
}
} else if (error.message) {
// Try to extract from message (e.g., "file.js: Unexpected token (10:5)")
const match = error.message.match(/\((\d+):(\d+)\)/);
if (match) {
error.loc = {
line: parseInt(match[1]),
column: parseInt(match[2])
};
}
}
return outputError(error, filePath, jsonOutput);
}
}
// =============================================================================
// MODE HANDLING: CLI or RPC Server
// =============================================================================
if (mode === 'server') {
// RPC Server Mode
if (!socketPath) {
console.error('Server mode requires --socket=/path/to/socket');
process.exit(1);
}
// Remove socket if exists
if (fs.existsSync(socketPath)) {
fs.unlinkSync(socketPath);
}
function handleRequest(data) {
try {
const request = JSON.parse(data);
switch (request.method) {
case 'ping':
return JSON.stringify({
id: request.id,
result: 'pong'
}) + '\n';
case 'transform':
const results = {};
for (const file of request.files) {
const fileTarget = file.target || 'modern';
const fileHashPath = file.hash_path || file.path;
try {
const content = fs.readFileSync(file.path, 'utf8');
const transformResult = transformFileContent(content, file.path, fileTarget, fileHashPath, true);
results[file.path] = transformResult;
} catch (error) {
results[file.path] = {
status: 'error',
error: {
type: 'FileReadError',
message: error.message
}
};
}
}
return JSON.stringify({
id: request.id,
results: results
}) + '\n';
case 'shutdown':
return JSON.stringify({
id: request.id,
result: 'shutting down'
}) + '\n';
default:
return JSON.stringify({
id: request.id,
error: 'Unknown method: ' + request.method
}) + '\n';
}
} catch (error) {
return JSON.stringify({
error: 'Invalid JSON request: ' + error.message
}) + '\n';
}
}
const server = net.createServer((socket) => {
let buffer = '';
socket.on('data', (data) => {
buffer += data.toString();
let newlineIndex;
while ((newlineIndex = buffer.indexOf('\n')) !== -1) {
const line = buffer.substring(0, newlineIndex);
buffer = buffer.substring(newlineIndex + 1);
if (line.trim()) {
const response = handleRequest(line);
socket.write(response);
try {
const request = JSON.parse(line);
if (request.method === 'shutdown') {
socket.end();
server.close(() => {
if (fs.existsSync(socketPath)) {
fs.unlinkSync(socketPath);
}
process.exit(0);
});
}
} catch (e) {
// Ignore
}
}
}
});
socket.on('error', (err) => {
console.error('Socket error:', err);
});
});
server.listen(socketPath, () => {
console.log('JS Transformer RPC server listening on ' + socketPath);
});
server.on('error', (err) => {
console.error('Server error:', err);
process.exit(1);
});
process.on('SIGTERM', () => {
server.close(() => {
if (fs.existsSync(socketPath)) {
fs.unlinkSync(socketPath);
}
process.exit(0);
});
});
process.on('SIGINT', () => {
server.close(() => {
if (fs.existsSync(socketPath)) {
fs.unlinkSync(socketPath);
}
process.exit(0);
});
});
} else {
// CLI Mode
// Default hashPath to filePath if not provided
if (!hashPath) {
hashPath = filePath;
}
if (!filePath) {
const error = new Error('No input file specified');
if (jsonOutput) {
console.log(JSON.stringify({
status: 'error',
error: {
type: 'ArgumentError',
message: error.message,
suggestion: 'Usage: node js-transformer-server.js [--json] <file-path> [target] [hash-path]'
}
}));
} else {
console.error('Usage: node js-transformer-server.js [--json] <file-path> [target] [hash-path]');
console.error(' or: node js-transformer-server.js --socket=/path/to/socket');
console.error('Targets: modern, es6, es5');
}
process.exit(1);
}
// Read file
let content;
try {
content = fs.readFileSync(filePath, 'utf8');
} catch (error) {
error.message = `Error reading file: ${error.message}`;
const errorOutput = outputError(error, filePath, jsonOutput);
if (jsonOutput) {
console.log(JSON.stringify(errorOutput));
} else {
console.error(errorOutput.message);
}
process.exit(1);
}
// Transform file
const result = transformFileContent(content, filePath, target, hashPath, jsonOutput);
// Output result
if (jsonOutput) {
console.log(JSON.stringify(result));
} else {
if (result.status === 'error') {
console.error(result.message || result.error.message);
process.exit(1);
} else {
console.log(result.result);
}
}
}