Skip to content
  • Nikita Popov's avatar
    4b497045
    Move attribute handling into parser · 4b497045
    Nikita Popov authored
    The Lexer now only provides the tokens to the parser, while the
    parser is responsible for determining which attributes are placed
    on notes. This only needs to be done when the attributes are
    actually needed, rather than for all tokens.
    
    This removes the usedAttributes lexer option (and lexer options
    entirely). The attributes are now enabled unconditionally. They
    have less overhead now, and the need to explicitly enable them for
    some use cases (e.g. formatting-preserving printing) doesn't seem
    like a good tradeoff anymore.
    
    There are some additional changes to the Lexer interface that
    should be done after this, and the docs / upgrading guide haven't
    been adjusted yet.
    4b497045
    Move attribute handling into parser
    Nikita Popov authored
    The Lexer now only provides the tokens to the parser, while the
    parser is responsible for determining which attributes are placed
    on notes. This only needs to be done when the attributes are
    actually needed, rather than for all tokens.
    
    This removes the usedAttributes lexer option (and lexer options
    entirely). The attributes are now enabled unconditionally. They
    have less overhead now, and the need to explicitly enable them for
    some use cases (e.g. formatting-preserving printing) doesn't seem
    like a good tradeoff anymore.
    
    There are some additional changes to the Lexer interface that
    should be done after this, and the docs / upgrading guide haven't
    been adjusted yet.
Loading