pub trait Source {
type Slice<'a>: PartialEq + Eq + Debug
where Self: 'a;
// Required methods
fn len(&self) -> usize;
fn read<'a, Chunk>(&'a self, offset: usize) -> Option<Chunk>
where Chunk: Chunk<'a>;
unsafe fn read_unchecked<'a, Chunk>(&'a self, offset: usize) -> Chunk
where Chunk: Chunk<'a>;
fn slice(&self, range: Range<usize>) -> Option<Self::Slice<'_>>;
unsafe fn slice_unchecked(&self, range: Range<usize>) -> Self::Slice<'_>;
fn is_boundary(&self, index: usize) -> bool;
// Provided method
fn find_boundary(&self, index: usize) -> usize { ... }
}
Expand description
Trait for types the Lexer
can read from.
Most notably this is implemented for &str
. It is unlikely you will
ever want to use this Trait yourself, unless implementing a new Source
the Lexer
can use.
Required Associated Types§
Required Methods§
sourcefn read<'a, Chunk>(&'a self, offset: usize) -> Option<Chunk>where
Chunk: Chunk<'a>,
fn read<'a, Chunk>(&'a self, offset: usize) -> Option<Chunk>where
Chunk: Chunk<'a>,
Read a chunk of bytes into an array. Returns None
when reading
out of bounds would occur.
This is very useful for matching fixed-size byte arrays, and tends to be very fast at it too, since the compiler knows the byte lengths.
use logos::Source;
let foo = "foo";
assert_eq!(foo.read(0), Some(b"foo")); // Option<&[u8; 3]>
assert_eq!(foo.read(0), Some(b"fo")); // Option<&[u8; 2]>
assert_eq!(foo.read(2), Some(b'o')); // Option<u8>
assert_eq!(foo.read::<&[u8; 4]>(0), None); // Out of bounds
assert_eq!(foo.read::<&[u8; 2]>(2), None); // Out of bounds
sourceunsafe fn read_unchecked<'a, Chunk>(&'a self, offset: usize) -> Chunkwhere
Chunk: Chunk<'a>,
unsafe fn read_unchecked<'a, Chunk>(&'a self, offset: usize) -> Chunkwhere
Chunk: Chunk<'a>,
Read a chunk of bytes into an array without doing bounds checks.
Safety
Offset should not exceed bounds.
sourcefn slice(&self, range: Range<usize>) -> Option<Self::Slice<'_>>
fn slice(&self, range: Range<usize>) -> Option<Self::Slice<'_>>
Get a slice of the source at given range. This is analogous to
slice::get(range)
.
use logos::Source;
let foo = "It was the year when they finally immanentized the Eschaton.";
assert_eq!(<str as Source>::slice(&foo, 51..59), Some("Eschaton"));
sourceunsafe fn slice_unchecked(&self, range: Range<usize>) -> Self::Slice<'_>
unsafe fn slice_unchecked(&self, range: Range<usize>) -> Self::Slice<'_>
Get a slice of the source at given range. This is analogous to
slice::get_unchecked(range)
.
Safety
Range should not exceed bounds.
use logos::Source;
let foo = "It was the year when they finally immanentized the Eschaton.";
unsafe {
assert_eq!(<str as Source>::slice_unchecked(&foo, 51..59), "Eschaton");
}
sourcefn is_boundary(&self, index: usize) -> bool
fn is_boundary(&self, index: usize) -> bool
Check if index
is valid for this Source
, that is:
- It’s not larger than the byte length of the
Source
. - (
str
only) It doesn’t land in the middle of a UTF-8 code point.
Provided Methods§
sourcefn find_boundary(&self, index: usize) -> usize
fn find_boundary(&self, index: usize) -> usize
For &str
sources attempts to find the closest char
boundary at which source
can be sliced, starting from index
.
For binary sources (&[u8]
) this should just return index
back.