Support composite and default tags

[ydah]

The lexer only recognized tags matching <\w+>, rejecting composite semantic types such as <int *> and <std::vector<int>> as well as Bison's special <*> and <> tags.

Replace the word-only matcher with a balanced tag pattern that accepts spaces, punctuation, empty tags, and nested angle brackets without stopping at an inner >.

[Copilot]

🟡 Changes recommended

The lexer now accepts tags (e.g., <int *>, <*>, <>) that appear incompatible with downstream tag handling/code generation and user-code $<...> reference parsing, which can lead to incorrect behavior.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR updates Lrama’s lexer to recognize more general Bison-style semantic value tags, including composite/nested angle-bracket forms and the special default tags (<>, <*>), and adds a spec plus an RBS constant to reflect the change.

Changes:

• Replace the word-only tag matcher (/<\w+>/) with a balanced tag regex that supports nesting and punctuation.
• Add an RSpec example covering composite and default tags.
• Expose TAG_PATTERN in the generated RBS signature.

File summaries

File	Description
spec/lrama/lexer_spec.rb	Adds coverage asserting the lexer emits :TAG tokens for composite/default tag strings.
sig/generated/lrama/lexer.rbs	Adds the TAG_PATTERN: Regexp constant to the generated type signature.
lib/lrama/lexer.rb	Introduces TAG_PATTERN and uses it in lex_token instead of /<\w+>/.

Review details

Suppressed comments (1)

lib/lrama/lexer.rb:131

• Lexer TAG_PATTERN is updated, but $&lt;...&gt; references inside user code are still parsed with the old word-only tag regex (see lexer/token/user_code.rb: the optional tag is "(&lt;[a-zA-Z0-9_]+&gt;)?"). This means composite/default tags will tokenize correctly as :TAG in declarations, but "$&lt;int >1" / "$<>$" / "$<>$" in actions will still not be recognized consistently.

      when @scanner.scan(TAG_PATTERN)
        return [:TAG, Lrama::Lexer::Token::Tag.new(s_value: @scanner.matched, location: location)]

• Files reviewed: 2/3 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.