JSE v2.0 AI Output Specification

(AST Architecture + Static Scoping)

1. Purpose

This specification defines JSE v2.0, which introduces AST-based evaluation and static scoping while maintaining full backward compatibility with v1.0’s structural syntax.

Key changes from v1.0:

• AST Architecture: Expressions are parsed into an Abstract Syntax Tree before evaluation
• Static Scoping: Closures capture their definition environment, not their call environment
• Scope Chaining: Environments form parent-child chains for symbol resolution
• First-class Functions: Lambda expressions with proper lexical closure support

2. Architecture Overview

JSE v2.0 implements a two-phase execution model:

JSON Value → Parser → AST Node → Environment → Result

Components

Two-Environment Pattern

For static scoping, v2.0 distinguishes between:

• Construction Environment (get_env()): Where the AST node was created
• Evaluation Environment (apply(env)): Where the AST node is executed

This separation enables proper closures: lambdas capture their definition environment.

3. Environment Model

Scope Chain

Environment
├── parent: Environment | null
├── bindings: Map<string, Value>
└── functors: Map<string, Functor>

Symbol Resolution: When resolving a symbol:

1. Check current environment’s bindings
2. If not found, recursively search parent chain
3. If chain exhausted, return “Symbol not found” error

Registration

• register(name, value): Adds new binding (errors if exists)
• set(name, value): Sets or overwrites binding
• load(functors): Bulk-loads a functor module

Module System

Functors are organized into modules:

4. AST Node Types

4.1 LiteralNode

Represents primitive values: numbers, strings, booleans, null.

42
"hello"
true
null

4.2 SymbolNode

Represents variable references that are resolved at evaluation time.

"$myVariable"

4.3 ArrayNode

Represents function calls or literal arrays.

Function Call (first element is a symbol):

["$add", 1, 2]

Literal Array (first element is NOT a symbol):

[1, 2, 3]
["$literalValue", "not an operator"]

4.4 ObjectNode

Represents key-value mappings.

{ "key": "value", "number": 42 }

4.5 ObjectExpressionNode

Represents operator expressions in object form.

{
  "$pattern": ["$*", "author of", "$*"],
  "confidence": 0.95
}

Rules:

• Exactly one key must start with $ (the operator)
• Other keys are metadata (preserved in result)
• Metadata does not affect evaluation

4.6 QuoteNode

Special node that prevents evaluation of its content.

["$quote", ["$unevaluated", "expression"]]

4.7 LambdaNode

Represents anonymous functions with closure support.

{
  "$lambda": {
    "__lambda__": true,
    "params": ["x", "y"],
    "body": ["$add", "$x", "$y"]
  }
}

5. Operator Reference

5.1 Builtin Operators

5.2 Utils Operators

5.3 Lisp Operators

5.4 SQL Operators

The SQL module is not a utility module; it is only used to demonstrate that functors with the same name may have completely different behaviors in different scopes.

6. Backward Compatibility

v1.0 Compatibility

All valid v1.0 JSE expressions remain valid in v2.0:

// v1.0 Array Form - Still works
["$and", true, false]

// v1.0 Object Form - Still works
{
  "$if": ["$gt", 5, 3],
  "confidence": 0.98
}

// v1.0 Quote - Still works
["$quote", { "$raw": "data" }]

New Capabilities

v2.0 adds:

// First-class functions with closures
{
  "$def": "makeAdder",
  "$lambda": {
    "params": ["x"],
    "body": {
      "$lambda": {
        "params": ["y"],
        "body": ["$add", "$x", "$y"]
      }
    }
  }
}

// Static scoping preserves definition environment
{
  "$defn": "counter",
  [],
  ["$def", "count", 0,
   "$lambda", {
     "params": [],
     "body": ["$set", "count", ["$add", "$count", 1]]
   }]
}

7. AI Output Requirements (v2.0)

When generating JSE v2.0, an AI system MUST:

Structural Rules (from v1.0):

• Produce syntactically valid JSON
• Respect Symbol rules ($ prefix, $$ escape)
• Use $quote to prevent evaluation
• Never produce multiple Symbol keys in one object

New in v2.0:

• Understand that expressions form an AST, not direct evaluation
• Use $lambda for first-class functions
• Use $def/$defn for variable/function definitions
• Leverage static scoping for closures

8. Operator Whitelisting

Implementations MAY:

• Restrict available operators
• Add custom operators via env.load()
• Implement different operator sets per use case

Recommended minimal set: $quote, $eq, $cond

Recommended safe set: builtin + utils (no $def, $defn, $eval)

Full power set: builtin + utils + lisp

9. Execution Model

Phase 1: Parsing

parser = Parser(env)
ast_node = parser.parse(json_value)

Phase 2: Evaluation

result = ast_node.apply(env)

Error Handling

10. Migration from v1.0

For Implementers

1. Add Parser: Parse JSON into AST nodes
2. Implement Scope Chain: Add parent field to Environment
3. Add Lambda Support: Implement closures with captured environments
4. Maintain Compatibility: Keep v1.0 syntax working

For Users

No changes required! All v1.0 expressions continue to work.

To use new features:

• Use $def to define variables
• Use $defn to define functions
• Use $lambda to create closures

11. Example: Complete Program

{
  "$defn": "factorial",
  ["n"],
  ["$cond",
    ["$eq", "$n", 0],
    1,
    ["$mul", "$n", ["$factorial", ["$add", "$n", -1]]]
  ],

  "$def": "result",
  ["$factorial", 5],

  "$get", "$$result", "value"
}

This demonstrates:

• Function definition with $defn
• Recursive function calls
• Static scoping
• Variable binding with $def
• Escaped symbol with $$

12. Compliance Checklist

A JSE v2.0 implementation MUST:

• Parse JSON into AST nodes before evaluation
• Implement environment with scope chaining
• Support static scoping for lambdas
• Maintain v1.0 structural syntax compatibility
• Provide the three-phase modules (builtin, utils, lisp)
• Support the four core node types (Literal, Symbol, Array, Object)
• Implement ObjectExpression metadata support
• Provide proper error types

13. Summary of Changes from v1.0

Appendix A: Type Definitions (TypeScript)

type Value =
  | null
  | boolean
  | number
  | string
  | Value[]
  | { [key: string]: Value };

interface Environment {
  parent: Environment | null;
  bindings: Map<string, Value>;
  functors: Map<string, Functor>;
}

interface Functor {
  (env: Environment, args: Value[]): Result<Value, Error>;
}

interface AstNode {
  apply(env: Environment): Result<Value, AstError>;
  getEnv(): Environment;
  toJSON(): Value;
}

Appendix B: Evaluation Semantics

Given expression E and environment ENV:

1. Parse: AST = Parser.parse(ENV, E)
2. Evaluate: RESULT = AST.apply(ENV)
3. Return: RESULT

For closures (lambda):

• DEF_ENV = AST.getEnv() (environment where lambda was defined)
• CALL_ENV = ENV (environment where lambda is called)
• Body evaluation uses DEF_ENV for symbol resolution (static scoping)

Appendix C: Quick Reference