Package org.bct.api

Class NestedTokenizer

java.lang.Object
org.bct.api.NestedTokenizer
class NestedTokenizer extends Object
Splits a nested comma-delimited string into a list of Token objects using a state machine parser.

This class parses complex nested structures with support for escaping and arbitrary nesting depth. The parser uses a finite state machine to handle different contexts during parsing.

Supported Syntax:
  • "foo" - Single value token
  • "foo,bar" - Multiple value tokens
  • "foo{a,b},bar" - Token with nested values
  • "foo{a{a1,a2}},bar" - Recursively nested values
  • "foo\\,bar" - Escaped comma in value
  • "foo\\{bar\\}" - Escaped braces in value
State Machine:

The parser operates in several states:

  • PARSING_VALUE: Reading a token value
  • PARSING_NESTED: Reading nested content within braces
  • IN_ESCAPE: Processing escaped character
Usage Examples:

// Simple tokens var tokens = NestedTokenizer.tokenize("foo,bar,baz"); // tokens = [Token{value="foo"}, Token{value="bar"}, Token{value="baz"}] // Nested tokens var nested = NestedTokenizer.tokenize("user{name,email},config{timeout,retries}"); // nested[0] = Token{value="user", nested=[Token{value="name"}, Token{value="email"}]} // nested[1] = Token{value="config", nested=[Token{value="timeout"}, Token{value="retries"}]}