rspade_system/node_modules/@jqhtml/parser/README.md

# JQHTML Parser

The JQHTML parser converts template files into JavaScript functions that use the component system.

## Quick Start - Common Patterns

### Basic Component with innerHTML (Most Common)

```jqhtml
<!-- Define a component with default attributes -->
<Define:Card tag="article" class="card" style="padding: 10px">
  <div class="card-body">
    <%= content() %>  <!-- Outputs whatever innerHTML was passed -->
  </div>
</Define:Card>

<!-- Usage - just pass innerHTML like regular HTML -->
<Card class="featured">
  <h3>Card Title</h3>
  <p>This is the card content.</p>
  <button>Click me</button>
</Card>

<!-- Result: <article class="card featured Card Jqhtml_Component" style="padding: 10px"> -->
```

This `content()` pattern is the **primary way** to compose components in JQHTML. It works like regular HTML - the innerHTML you pass gets rendered where `content()` is called.

### Container Component Example

```jqhtml
<!-- A container with default size that can be overridden -->
<Define:Container $size="medium" class="container">
  <div class="container-inner <%= this.args.size %>">
    <%= content() %>  <!-- Simple innerHTML output -->
  </div>
</Define:Container>

<!-- Usage - override the default size -->
<Container $size="large">
  <p>Any content here</p>
  <UserList />
  <Footer />
</Container>
```

**Note:** Slots (`<#slotname>`) are an advanced feature only needed when you have multiple distinct content areas. For 95% of use cases, the simple `content()` pattern shown above is recommended.

### Define Tag Attributes (Component Configuration)

Define tags support three types of attributes for configuring components:

```jqhtml
<Define:Contacts_DataGrid
    extends="DataGrid_Abstract"
    $ajax_endpoint=Frontend_Contacts_Controller.datagrid_fetch
    $per_page=25
    class="card DataGrid">
  <div>Content here</div>
</Define:Contacts_DataGrid>
```

**1. `extends=""` - Template Inheritance**
Explicitly declare parent template for inheritance (without requiring a JavaScript class).

**2. `$property=value` - Default Args**
Set default values for `this.args` that component invocations can override:
- **Quoted**: `$user_id="123"` → String literal `"123"`
- **Unquoted**: `$handler=MyController.fetch` → Raw JavaScript expression
- These become defaults; invocations can override them

**3. Regular Attributes**
Standard HTML attributes like `class=""`, `tag=""` applied to root element.

**Generated Output:**
```javascript
{
  name: 'Contacts_DataGrid',
  tag: 'div',
  defaultAttributes: {"class": "card DataGrid"},
  defineArgs: {"ajax_endpoint": Frontend_Contacts_Controller.datagrid_fetch, "per_page": "25"},
  extends: 'DataGrid_Abstract',
  render: function() { ... }
}
```

### Slot-Based Template Inheritance

When a template contains **ONLY slots** (no HTML), it automatically inherits the parent class template:

```jqhtml
<!-- Parent: DataGrid_Abstract.jqhtml -->
<Define:DataGrid_Abstract>
  <table>
    <thead><tr><%= content('header') %></tr></thead>
    <tbody>
      <% for (let record of this.data.records) { %>
        <tr><%= content('row', record) %></tr>
      <% } %>
    </tbody>
  </table>
</Define:DataGrid_Abstract>

<!-- Child: Users_DataGrid.jqhtml (slot-only) -->
<Define:Users_DataGrid>
  <#header>
    <th>ID</th><th>Name</th><th>Email</th>
  </#header>
  <#row>
    <td><%= row.id %></td>
    <td><%= row.name %></td>
    <td><%= row.email %></td>
  </#row>
</Define:Users_DataGrid>
```

The parser detects slot-only templates and generates `{_slots: {...}}` format. Runtime walks prototype chain to find parent templates. **Data passing**: `content('row', record)` passes data to slot parameter `function(row) { ... }`. **Reserved words**: Slot names cannot be JavaScript keywords (`function`, `if`, etc.) - parser rejects with fatal error.

## Lexer (Task 1 - COMPLETE)

The lexer is the first stage of the parser. It converts raw JQHTML template text into a stream of tokens.

### Features

- **No regex** - Uses simple character scanning for maintainability
- **Position tracking** - Every token includes line, column, and absolute positions for source maps
- **Simple token types** - Clear, unambiguous token categories
- **Efficient scanning** - Single pass through the input

### Token Types

- `TEXT` - Plain HTML/text content
- `EXPRESSION_START` - `<%=` opening tag
- `CODE_START` - `<%` opening tag
- `TAG_END` - `%>` closing tag
- `DEFINE_START`, `DEFINE_END` - Component definition tags
- `COMPONENT_NAME` - Component identifier
- `JAVASCRIPT` - JavaScript code within tags

### Usage

```typescript
import { Lexer } from '@jqhtml/parser';

const template = `
<Define:MyComponent>
  <h1><%= this.data.title %></h1>
  <% if (this.data.show) { %>
    <p>Content here</p>
  <% } %>
</Define:MyComponent>
`;

const lexer = new Lexer(template);
const tokens = lexer.tokenize();

// tokens is an array of Token objects with:
// - type: TokenType
// - value: string
// - line: number
// - column: number  
// - start: number (absolute position)
// - end: number
```

### Testing

```bash
# Build the parser
npm run build

# Run the test suite
node test-lexer.js

# Run the demo
node demo-lexer.js
```

### Implementation Notes

The lexer uses a simple state machine approach:

1. **Text scanning** - Default mode, captures plain text
2. **Tag detection** - Looks for `<%`, `<%=`, `%>` sequences
3. **Keyword matching** - After `<%`, checks for control flow keywords
4. **JavaScript capture** - Captures code between tags
5. **Position tracking** - Updates line/column for every character

No regex patterns are used. All scanning is done with:
- `match_sequence()` - Check for exact string match
- `match_keyword()` - Check for keyword with word boundary
- Character-by-character advancement

## Parser/AST Builder (Task 2 - COMPLETE)

The parser takes the token stream from the lexer and builds an Abstract Syntax Tree (AST) representing the template structure.

### Features

- **Recursive descent parsing** - Simple, predictable parsing algorithm
- **Clear node types** - Each AST node represents a specific construct
- **Position preservation** - All nodes include source positions
- **Error reporting** - Clear messages with line/column information

### AST Node Types

- `Program` - Root node containing all top-level definitions
- `ComponentDefinition` - Component template definition
- `Text` - Plain text/HTML content
- `Expression` - JavaScript expressions (`<%= ... %>`)
- `IfStatement` - Conditional rendering with optional else
- `ForStatement` - Loop constructs  
- `CodeBlock` - Generic JavaScript code blocks

### Usage

```typescript
import { Lexer, Parser } from '@jqhtml/parser';

const template = `
<Define:Card>
  <div class="card">
    <h3><%= title %></h3>
    <% if (showContent) { %>
      <p><%= content %></p>
    <% } %>
  </div>
</Define:Card>
`;

// Tokenize
const lexer = new Lexer(template);
const tokens = lexer.tokenize();

// Parse to AST
const parser = new Parser(tokens);
const ast = parser.parse();

// AST structure:
// {
//   type: 'Program',
//   body: [{
//     type: 'ComponentDefinition',
//     name: 'Card',
//     body: [...]
//   }]
// }
```

### Testing

```bash
# Run the parser test suite
node test-parser.js

# Run the interactive demo
node demo-parser.js
```

### Implementation Notes

The parser uses straightforward techniques:

1. **Token consumption** - Advances through tokens one at a time
2. **Lookahead** - Peeks at upcoming tokens to decide parsing path
3. **Context tracking** - Knows when inside components, loops, etc.
4. **Error recovery** - Provides helpful error messages

No parser generators or complex algorithms - just simple recursive functions that build nodes.

## Code Generator (Task 3 - COMPLETE)

The code generator takes the AST and produces executable JavaScript functions that work with the JQHTML component system.

### Features

- **jQuery-based DOM manipulation** - Generates efficient jQuery code
- **Component render functions** - Each template becomes a render function
- **Control flow handling** - Properly handles if/else and for loops
- **Expression evaluation** - Safely evaluates and renders expressions
- **Colon syntax support** - Strips trailing colons from control statements

### Generated Code Structure

```javascript
// Each component gets a render function
jqhtml_components.set('ComponentName', {
  name: 'ComponentName',
  render: function render() {
    const $root = $('<div></div>');
    const $current = $root;
    
    // Generated DOM manipulation code here
    
    return $root.children();
  },
  dependencies: []
});
```

### Usage

```typescript
import { Lexer, Parser, CodeGenerator } from '@jqhtml/parser';

const template = `
<Define:MyComponent>
  <h1><%= this.data.title %></h1>
  <% if (this.data.items) { %>
    <ul>
      <% for (const item of this.data.items) { %>
        <li><%= item %></li>
      <% } %>
    </ul>
  <% } %>
</Define:MyComponent>
`;

// Generate the code
const lexer = new Lexer(template);
const tokens = lexer.tokenize();
const parser = new Parser(tokens);
const ast = parser.parse();
const generator = new CodeGenerator();
const result = generator.generate(ast);

// Use in a component
class MyComponent extends Component {
  async on_render() {
    const template = result.components.get('MyComponent');
    const elements = template.render.call(this);
    this.$.append(elements);
  }
}
```

### Testing

```bash
# Run the code generator test suite
node test-codegen.js

# Run the interactive demo
node demo-codegen.js

# View the integration example
# Open example-integration.html in a browser
```

### Implementation Notes

The code generator:

1. **Traverses the AST** - Visits each node and generates appropriate code
2. **Maintains context** - Tracks current jQuery element for appending
3. **Handles nesting** - For loops create temporary containers
4. **Strips syntax** - Removes trailing colons from control flow
5. **Escapes strings** - Properly escapes text content for JavaScript

### v1 JavaScript Compilation Analysis

A comprehensive analysis of how JQHTML v1 compiles templates to JavaScript has been documented in [JQHTML_V1_JAVASCRIPT_COMPILATION_PATTERNS.md](../../JQHTML_V1_JAVASCRIPT_COMPILATION_PATTERNS.md). This analysis reveals:

- **Instruction-based output**: Templates compile to arrays of rendering instructions
- **Three instruction types**: `{tag:...}`, `{comp:...}`, `{block:...}`
- **Context preservation**: The `_that = this` pattern for nested functions
- **Single-pass DOM construction**: Efficient rendering with one `$.html()` call
- **Deferred component initialization**: Two-phase process for component creation

These patterns inform v2 implementation decisions, particularly around maintaining the efficient instruction-based architecture while modernizing the JavaScript output.

## Implementation Status (December 2024)

### ✅ Completed Tasks

1. **Lexer** - Full tokenization with position tracking
2. **Parser** - AST generation with all v2 syntax
3. **Code Generator** - Instruction array generation
4. **Component Integration** - Runtime integration complete
5. **Slot System** - `<#name>` syntax fully implemented
6. **Nested Components** - `<ComponentName>` recognition
7. **Binding Syntax** - `:prop` and `@event` support

### Current Features

- **Template Syntax**: HTML with embedded JavaScript
- **Control Flow**: `if/else/elseif` and `for` loops (both colon and brace styles)
- **Components**: `<Define:Name>` definitions and `<ComponentName>` invocations
- **Slots**: `<#name>content</#name>` and `<#name />` syntax
- **Expressions**: `<%= expression %>` for output
- **Bindings**: `:text`, `:value`, `:class`, `:style` for data binding
- **Events**: `@click`, `@change`, etc. for event handlers
- **Scoped IDs**: `$id` attribute for component-scoped IDs

### Instruction Format

The parser generates v1-compatible instruction arrays:

```javascript
// HTML tags
{tag: ["div", {"class": "card"}, false]}  // open tag
{tag: ["div", {}, true]}                   // close tag

// Components
{comp: ["UserCard", {name: "John"}]}

// Slots
{slot: ["header", {}, (props) => {
  const _output = [];
  _output.push("Header content");
  return [_output, this];
}]}

// Text content
"Plain text"

// Expressions
{expr: () => this.data.title}
```

### Integration with Core

Templates compile to functions that return `[instructions, context]`:

```javascript
function render() {
  const _output = [];
  
  // Generate instructions...
  _output.push({tag: ["h1", {}, false]});
  _output.push("Hello World");
  _output.push({tag: ["h1", {}, true]});
  
  return [_output, this];
}
```

The core runtime's instruction processor handles these arrays to create DOM.

### Next Steps

1. **Build Tools** - Webpack loader for `.jqhtml` imports
2. **Source Maps** - Full debugging support
3. **VS Code Extension** - Syntax highlighting
4. **Autoformatter** - Code formatting for `.jqhtml` files