I code a new shell in C, that could be done in several ways: Flex/bison, finite state machine, abstract syntax tree or just a tokenizer in C. Now I’ve written a for-loop that changes the condition of the increment and I wonder if that is the same as a FSM? First I did it as a finite state machine but that code was very basic. Now I work on doing it with loops or recursion, hopefully I can make 2 methods that are mutually recursive in building up a pipeline, or that the for loop can handle all cases by dynamically building up an array an increasing the arguments. I still didn’t settle on a reasonable MAX_PIPELINE_LENGTH but that could also be dynamic. Now the part of the code that I’m writing about looks as follows and tokenizes input from the shell e.g. the parameter const char * cmd is a string that could be something like echo 'foobar blarg'|grep foo|awk '{print $1}'|wc but it can only parse the command up to 4 pipelines this far.
I’m wondering if it is even possible to solve the problem with a for loop that implements a FSM or an AST with a for loop and the way that I try to do it or do I need I new strategy where I “think about recursion” from the beginning rather that hoping to make the function recursive later? I already have code to check if a char is between quotations.
I think that between compound lists (which involve parenthesized expressions) and various operators (like |, && and ||, …) the shell grammar is fundamentally recursive in nature, so yes, you should consider recursion up front.
There are many ways to parse language, but for this grammar, any way you do it your parser needs to handle recursive expressions on input, and your parser output needs to be be a some kind of recursive construct.
See here.
That doesn’t necessarily mean you have to use a recursive algorithm to do the parsing, however, if you don’t you’ll most certainly need one or more growable stack’s to store intermediate parsing state, which kind of rules out a simple state machine.
1