Clarification about Grammars , Lexers and Parsers
Background info (May Skip): I am working on a task we have been set at uni in which we have to design a grammar for a DSL we have been provided with. The grammar must be in BNF or EBNF. As well as other thing we are being evaluated on the Lexical rules in the grammar and the Parsing rules – such as if rules are suitable for the language subset, how comprehensive these rules are, how clear the rules are ect.
Storing tokens during lexing stage
I am currently implementing a lexer that breaks XML files up into tokens, I’m considering ways of passing the tokens onto a parser to create a more useful data structure out of said tokens – my current plan is to store them in an arraylist and pass this to the parser , would a link list where each token points to the next be better suited? Or is being able to access tokens by index easier to make a parser for? Or is this all a terrible strategy?
Storing tokens during lexing stage
I am currently implementing a lexer that breaks XML files up into tokens, I’m considering ways of passing the tokens onto a parser to create a more useful data structure out of said tokens – my current plan is to store them in an arraylist and pass this to the parser , would a link list where each token points to the next be better suited? Or is being able to access tokens by index easier to make a parser for? Or is this all a terrible strategy?
Storing tokens during lexing stage
I am currently implementing a lexer that breaks XML files up into tokens, I’m considering ways of passing the tokens onto a parser to create a more useful data structure out of said tokens – my current plan is to store them in an arraylist and pass this to the parser , would a link list where each token points to the next be better suited? Or is being able to access tokens by index easier to make a parser for? Or is this all a terrible strategy?
Storing tokens during lexing stage
I am currently implementing a lexer that breaks XML files up into tokens, I’m considering ways of passing the tokens onto a parser to create a more useful data structure out of said tokens – my current plan is to store them in an arraylist and pass this to the parser , would a link list where each token points to the next be better suited? Or is being able to access tokens by index easier to make a parser for? Or is this all a terrible strategy?
Storing tokens during lexing stage
I am currently implementing a lexer that breaks XML files up into tokens, I’m considering ways of passing the tokens onto a parser to create a more useful data structure out of said tokens – my current plan is to store them in an arraylist and pass this to the parser , would a link list where each token points to the next be better suited? Or is being able to access tokens by index easier to make a parser for? Or is this all a terrible strategy?
Storing tokens during lexing stage
I am currently implementing a lexer that breaks XML files up into tokens, I’m considering ways of passing the tokens onto a parser to create a more useful data structure out of said tokens – my current plan is to store them in an arraylist and pass this to the parser , would a link list where each token points to the next be better suited? Or is being able to access tokens by index easier to make a parser for? Or is this all a terrible strategy?
Lexing: One token per operator, or one universal operator token?
When lexing, what would be the best way to tokenize operators? Would one just create a BinaryOperator token, or a separate token for every single binary operator? Examples: PlusOperator, MinusOperator, DivisionOperator …
Lexing: One token per operator, or one universal operator token?
When lexing, what would be the best way to tokenize operators? Would one just create a BinaryOperator token, or a separate token for every single binary operator? Examples: PlusOperator, MinusOperator, DivisionOperator …
Lexer/Parser for multidimensional Languages
How does Lexer/Parser work in a 2D programming language like Funciton in order to transform such an unusual source-code to the correct AST?