Refactor Expression Grammar to Resolve Parser Conflicts

[lwgray]

Problem

The Think parser currently has a shift/reduce conflict in state 49 due to ambiguous grammar rules around arithmetic expressions and primary expressions. This creates a warning during parser initialization and relies on PLY's default conflict resolution.

Changes

Grammar Restructuring

• Removed direct primary_expr path from expression rule
• Introduced proper arithmetic hierarchy:

  expression -> arithmetic_expr
  arithmetic_expr -> term
  term -> factor
  factor -> primary_expr
  

Rule Consolidation

• Simplified p_number rule to handle basic numbers only
• Moved unary minus handling to factor level
• Consolidated primary expression rules

Key File Changes

In parser.py:

- expression : arithmetic_expr | comparison_expr | primary_expr
+ expression : arithmetic_expr | comparison_expr

- arithmetic_expr : primary_expr | arithmetic_expr PLUS arithmetic_expr | ...
+ arithmetic_expr : term | arithmetic_expr PLUS term | ...

+ term : factor | term TIMES factor | term DIVIDE factor

- factor : IDENTIFIER | NUMBER | numeric_literal | ...
+ factor : primary_expr | MINUS factor %prec UMINUS

Testing

Must Test

1. Basic arithmetic expressions

   • Addition, subtraction
   • Multiplication, division
   • Mixed operations

2. Unary operations

   • Negative numbers
   • Nested unary minus

3. Complex expressions

   • Nested operations
   • Parenthesized expressions
   • Operator precedence

Edge Cases

• Multiple unary minus operators
• Mixed numeric types (int/float)
• Expressions with function calls
• Array indexing within expressions

Implementation Notes

• Maintains existing operator precedence rules
• No changes to AST structure or interpreter
• Preserves all existing language functionality
• Better aligns with standard parsing patterns

Migration

This is a parser-internal change that does not affect:

• Language syntax
• Runtime behavior
• AST structure
• User code

No migration steps are needed for existing Think programs.