Uses of Class
fr.umlv.tatoo.runtime.lexer.Tokenizer

Packages that use Tokenizer

fr.umlv.tatoo.runtime.lexer

Uses of Tokenizer in fr.umlv.tatoo.runtime.lexer

Subclasses of Tokenizer in fr.umlv.tatoo.runtime.lexer

class

Scanner<R,B extends LexerBuffer> Provide a simple scanner that implements Iterator and Iterable The buffer must support the read operation.

Methods in fr.umlv.tatoo.runtime.lexer that return Tokenizer

static <R,B extends LexerBuffer> Tokenizer<R,B>	Tokenizer.createTokenizer(LexerTable<R> lexerTable, B buffer) Creates a tokenizer that extracts character from a buffer and matches them against rules.
static <R,B extends LexerBuffer> Tokenizer<R,B>	Tokenizer.createTokenizer(LexerTable<R> lexerTable, B buffer, Iterable<? extends R> blanks) Creates a scanner that extracts character from a buffer and matches them against rules.