Lexer

nothrow

class Lexer {

Loc scanloc;

Loc prevloc;

const(char)* p;

Token token;

this(const(char)* filename, const(char)* base, size_t begoffset, size_t endoffset, bool doDocComment, bool commentToken, DiagnosticHandler handleDiagnostic);

Token* allocateToken();

TOK peekNext();

TOK peekNext2();

void scan(Token* t);

Token* peekPastParen(Token* tk);

static const(char)* combineComments(const(char)[] c1, const(char)[] c2, bool newParagraph);

}

Constructors

this this(const(char)* filename, const(char)* base, size_t begoffset, size_t endoffset, bool doDocComment, bool commentToken, DiagnosticHandler handleDiagnostic): Creates a Lexer for the source code base[begoffset..endoffset+1]. The last character, baseendoffset, must be null (0) or EOF (0x1A).

Members

Functions

allocateToken Token* allocateToken()
peekNext TOK peekNext(): Look ahead at next token's value.
peekNext2 TOK peekNext2(): Look 2 tokens ahead at value.
peekPastParen Token* peekPastParen(Token* tk): tk is on the opening (. Look ahead and return token that is past the closing ).
scan void scan(Token* t): Turn next token in buffer into a token.

Static functions

combineComments const(char)* combineComments(const(char)[] c1, const(char)[] c2, bool newParagraph): Combine two document comments into one, separated by an extra newline if newParagraph is true.

Meta

Source

See Implementation

dmd lexer classes

Lexer