Crate logos

Logos

Create ridiculously fast Lexers.

Logos has two goals:

• To make it easy to create a Lexer, so you can focus on more complex problems.
• To make the generated Lexer faster than anything you’d write by hand.

To achieve those, Logos:

• Combines all token definitions into a single deterministic state machine.
• Optimizes branches into lookup tables or jump tables.
• Prevents backtracking inside token definitions.
• Unwinds loops, and batches reads to minimize bounds checking.
• Does all of that heavy lifting at compile time.

See the Logos handbook for additional documentation and usage examples.

Re-exports

• pub use crate::source::Source;

Modules

• source
  This module contains a bunch of traits necessary for processing byte strings.

Structs

• Lexer
  Lexer is the main struct of the crate that allows you to read through a Source and produce tokens for enums implementing the Logos trait.
• Skip
  Type that can be returned from a callback, informing the Lexer, to skip current token match. See also logos::skip.
• SpannedIter
  Iterator that pairs tokens with their position in the source.

Enums

• Filter
  Type that can be returned from a callback, either producing a field for a token, or skipping it.
• FilterResult
  Type that can be returned from a callback, either producing a field for a token, skipping it, or emitting an error.

Traits

• Logos
  Trait implemented for an enum representing all tokens. You should never have to implement it manually, use the #[derive(Logos)] attribute on your enum.

Functions

• skip
  Predefined callback that will inform the Lexer to skip a definition.

Type Aliases

• Span
  Byte range in the source.

Derive Macros

• Logos