DevDocs

Table of Contents

Bombadil

All that is gold does not glitter,
Not all those who wander are lost;
The old that is strong does not wither,
Deep roots are not reached by the frost.

From the ashes a fire shall be woken,
A light from the shadows shall spring;
Renewed shall be blade that was broken,
The crownless again shall be king.

- J.R.R. Tolkien, The Fellowship of the Ring

Welcome, to the home page of Marko Štajgár

My Crafts & Projects -> flow -> documentation

1. The Flow Shell - Developer Documentation

1.1. Introduction

Rough idea of what the flow lifecycle looks like with links to the corresponding modules documented further down bellow:

  1. Inicialize and parse the cli args
  2. Based on the choice of execution mode (REPL or from file) chosen by the user go into eighter FlowLoopFile or FlowLoopRepl mode
  3. Load Input
  4. Run Input through Lexer
  5. Run FlowLexer output through FlowParser
  6. Run FlowParser output through Executer
  7. If the end of input or source file is encountered (0 bytes read) or FlowState::Exit state is returned from the Executer then break out of the program loop and exit, otherwise go to step 3.

1.2. Modules

The Flow Shell codebase is divided into several rust modules that work together to compose the main program loop. Below is an overview of the modules:

1.2.1. reporters

Current location: ./src/reporters

1.2.1.1. Submodules
1.2.1.1.1. debugmodereporter

Current location: ./src/reporters/debugmodereporter

This submodule implements simple debug-mode reporting functions, printing enter/exit events and informational messages for instrumenting function calls.

1.2.1.1.1.1. Functions
1.2.1.1.1.1.1. flow debug mode function enter

Logs the entry of a function with its arguments.

Source: flowdebugmodereporter.rs

pub fn flow_debug_mode_function_enter(  
    module_name: &str,  
    function_name: &str,  
    args: Vec<&str>  
) -> ()

Arguments:

  1. module_name: &str - name of the module emitting the event.
  2. function_name: &str - name of the function being entered.
  3. args: Vec<&str> - vector of string slices representing the function arguments.

Behavior:

  • Prints a header line: FUNCTION_ENTER - module_name -> function_name -> with args:
  • Iterates over `args`, printing each with its index.

Return type:

  • Void
1.2.1.1.1.1.2. flow debug mode function exit

Logs the exit of a function with its return values.

Source: flowdebugmodereporter.rs

pub fn flow_debug_mode_function_exit(  
    module_name: &str,  
    function_name: &str,  
    return_values: Vec<&str>  
) -> () 

Arguments:

  1. module_name: &str - name of the module emitting the event.
  2. function_name: &str - name of the function being exited.
  3. return_values: Vec<&str> - vector of string slices for the return values.

Behavior:

  • Prints a header line: FUNCTION_EXIT - module_name -> function_name -> with return values:
  • Iterates over return_values, printing each with its index.

Return type:

  • Void
1.2.1.1.1.1.3. flow debug mode function info

Logs an informational message during function execution.

Source: flowdebugmodereporter.rs

pub fn flow_debug_mode_function_info(  
    module_name: &str,  
    function_name: &str,  
    message: &str  
) -> () 

Arguments:

  1. module_name: &str - name of the module emitting the event.
  2. function_name: &str - name of the function reporting info.
  3. message: &str - informational message to print.

Behavior:

  • Prints a single line: INFO - module_name -> function_name -> with message: "message"

Return type:

  • Void
1.2.1.1.2. flowerrorreporter

This source file implements error reporting by printing formatted error messages to standard error.

1.2.1.1.2.1. Functions
1.2.1.1.2.1.1. flow report error

Reports an error with its location and message.

Source: flowerrorreporter.rs

pub fn flow_report_error(  
    line: u32,  
    width: u32,  
    error_location: &str,  
    message: &str  
) -> ()

Arguments:

  1. line: u32 - line number where the error occurred (1-based).
  2. width: u32 - character position within the line (0 indicates the entire line).
  3. error_location: &str - identifier of the function or module reporting the error.
  4. message: &str - error message to display.

Behavior:

  • Prints to standard error in the format: [line {line}, char {width}] Error: {error_location} \u2192 {message}.

Return type:

  • Void

1.2.2. flowloop

Current location: ./src/flowloop

This module serves as the main composer of the input reading and interpreting.

1.2.2.1. Submodules
1.2.2.1.1. historyhandler

Current location: ./src/flowloop/historyhandler

This source file implements the shells history-logging functionality. It tracks commands entered during runtime and appends them to a history file upon exit or when explicitly called.

1.2.2.1.1.1. Structs
1.2.2.1.1.1.1. ShellHistoryLogger

Source: flowhistoryhandler.rs

#[derive(Debug)]  
pub struct ShellHistoryLogger {  
    hist_file_location: PathBuf,  
    runtime_shell_history: Vec<String>,  
}  

Fields:

  1. hist_file_location: PathBuf - filesystem path to the history file (either from the HISTLOCATION envar or defaulting to ~/.flow_history in the users home directory).
  2. runtime_shell_history: Vec<String> - in memory buffer of commands entered during the current session.
1.2.2.1.1.2. Methods
1.2.2.1.1.2.1. new

Constructs a new `ShellHistoryLogger`, determining and creating the history file if needed.

Source: flowhistoryhandler.rs

pub fn new() -> Result<Self, String>  

Arguments: None

Return type:

  • Returns Result<ShellHistoryLogger, String>
    • Ok: a logger with hist_file_location set and empty runtime_shell_history.
    • Err: an error message if home directory lookup, file creation, or custom-path validation fails.

Behavior:

  1. Checks HISTLOCATION envar for a custom path.
  2. If unset, uses HOME or USERPROFILE to derive ~/.flow_history and creates it if nonexistent.
  3. If set but the file doesnt exist, returns an error.
1.2.2.1.1.2.2. log all history

Appends all buffered entries to the history file.

Source: flowhistoryhandler.rs

pub fn log_all_history(&mut self) -> Result<(), String>  

Arguments:

  • &mut self - mutable reference to the logger.

Return type:

  • Returns Result<(), String>
    • Ok: entries successfully written.
    • Err: formatted I/O error.

Behavior:

  1. Opens self.hist_file_location with append mode.
  2. Iterates over runtime_shell_history, writing each entry on its own line.
1.2.2.1.1.2.3. update

Adds a new command to the inmemory history buffer.

Source: flowhistoryhandler.rs

pub fn update(&mut self, new_input: String) -> Result<(), String>  

Arguments:

  1. &mut self - mutable reference to the logger.
  2. new_input: String - the command string to record.

Return type:

  • Returns Result<(), String> always Ok(()), since in-memory push cannot fail.

Behavior: Pushes new_input into runtime_shell_history.

1.2.2.1.1.3. drop

Flushes the buffered history to disk, intended for shell exit.

Source: flowhistoryhandler.rs

pub fn drop(&mut self) -> Result<(), String>  

Arguments:

  • &mut self - mutable reference to the logger.

Return type:

  • Returns Result<(), String>
    • Ok: history successfully written.
    • Err: formatted error.

Behavior: Calls log_all_history() and propagates any error.

1.2.2.2. flowlooprepl

Current location: ./src/flowloop

This source file implements the REPL functionality of the shell. It reads user input, invokes the interpreter, logs history, and handles debug reporting.

1.2.2.2.1. Functions
1.2.2.2.1.1. Flow Read Line

Reads one line from a buffered reader after printing a prompt.

Source: flowlooprepl.rs

fn flow_read_line<R: BufRead>(  
    reader: &mut R,  
    prompt: &str,  
) -> io::Result<Option<String>>

Arguments:

  1. reader: &mut R - mutable reference to a generic that implements BufRead.
  2. prompt: &str - reference to the prompt string to print.

Return type:

  • Returns io::Result<Option<String>>:
    • Ok(Some(line)) with the input line (including newline).
    • Ok(None) on EOF.
    • Err on I/O failure.
1.2.2.2.1.2. Flow Loop REPL

Main loop of the shell REPL: reads input, interprets it, logs it to history, and exits on FlowState::Exit.

Source: flowlooprepl.rs

pub fn flow_loop_repl(  
    debug_mode: bool  
) -> Result<(), String>

Arguments:

  1. debug_mode: bool - flag indicating whether debug mode is enabled.

Return type:

  • Returns Result<(), String>:
    • Ok(() on normal exit.
    • Err(String) if errors occured.

Behavior:

  1. Determines prompt skin from PS1 or defaults to "> ".
  2. Initializes `TypeEnv` and `ShellHistoryLogger`.
  3. Loops while `running` is true:
    • Reads and trims input.
    • Calls flow_interpret; sets running = false on FlowState::Exit.
    • Updates in-memory history buffer.
  4. After loop, calls shell_history_logger.drop() to persist history.
1.2.2.3. flowloopfile

Current location: ./src/flowloop

This source file implements file-based REPL functionality. It reads s-expressions and non-s-expressions from a source file, invokes the interpreter on each expression, supports debug reporting, and stops on errors or exit signals.

1.2.2.3.1. Structs
1.2.2.3.1.1. SourceExpressionLoader

Accumulates tokens into complete expressions, handles comments, tracks nesting, and emits full expressions when ready.

Source: flowloopfile.rs

#[derive(Debug)]  
struct SourceExpressionLoader {  
    line_number: u32,  
    word_number: u32,  
    expression: String,  
    started_as_s_expression: bool,  
    in_comment: bool,  
    in_single_line_comment: bool,  
    new_expression: bool,  
    paren_count: i32,  
}  

Fields:

  1. line_number: u32 - current source-file line index (1-based).
  2. word_number: u32 - count of words processed in the current expression.
  3. expression: String - accumulated tokens forming the current expression.
  4. started_as_s_expression: bool - true if the expression began with (, indicating an s-expression.
  5. in_comment: bool - true when inside any comment.
  6. in_single_line_comment: bool - true when inside a ; comment until end of line.
  7. new_expression: bool - true when no tokens have yet been added to the current expression.
  8. paren_count: i32 - nesting depth of parentheses for s-expressions.
1.2.2.3.2. Methods
1.2.2.3.2.1. new

Initializes a fresh loader with zeroed counters and empty state.

Source: flowloopfile.rs

fn new() -> Self  

Arguments: None

Returns:

  • A `SourceExpressionLoader` with all fields reset.
1.2.2.3.2.2. clear last expression

Resets the loader state to begin accumulating a new expression.

Source: flowloopfile.rs

fn clear_last_expression(&mut self)  

Arguments:

  1. &mut self - mutable reference to the loader instance.

Behavior: Clears `expression`, resets flags (started_as_s_expression, in_comment, in_single_line_comment, new_expression), and sets paren_count back to zero.

1.2.2.3.2.3. handle load word

Processes a single whitespace-separated word, updating comment state, parentheses count, and expression buffer. Emits a complete expression when balanced.

Source: flowloopfile.rs

fn handle_load_word(&mut self, word: &str) -> Option<String>  

Arguments:

  1. &mut self - mutable reference to the loader.
  2. word: &str - the next token from the source.

Returns:

  • Some(String) when a full expression is complete (either an s-expression with balanced parentheses or a non-s-expression at line end). Expression includes all tokens and spacing.
  • None otherwise.

Behavior:

  1. If `word` starts with `;`, enters single-line comment mode.
  2. If in any comment, ignores the rest of the word.
  3. Appends `word` to `expression` (with space if not new_expression), increments word_number.
  4. If `word` starts with `(`, sets or continues s-expression state and increments paren_count.
  5. If `word` ends with `)`, decrements paren_count, when it returns to zero in an s-expression, returns the accumulated `expression` and clears.
1.2.2.3.2.4. line end

Called at the end of each source line to emit expressions that did not start with parentheses. Also resets single-line comment state.

Source: flowloopfile.rs

fn line_end(&mut self) -> Option<String>  

Arguments:

  1. &mut self - mutable reference to the loader.

Returns:

  • Some(String) when `expression` is non-empty and not an s-expression, containing the full expression.
  • None otherwise.

Behavior:

  1. Increments line_number.
  2. If in single-line comment, clears comment flags and returns `None`.
  3. If `expression` is non-empty and not an s-expression, returns it and clears state.
1.2.2.3.3. Functions
1.2.2.3.3.1. flow loop file

Reads and interprets expressions from a file until EOF, errors, or an `Exit` signal.

Source: flowloopfile.rs

pub fn flow_loop_file(  
    path: &PathBuf,  
    debug_mode: bool,  
) -> io::Result<()>  

Arguments:

  1. path: &PathBuf - filesystem path to the source file.
  2. debug_mode: bool - enable debug reporting.

Return type:

  • Ok(()) on normal completion.
  • Err(io::Error) if file I/O fails.

Behavior:

  1. Emits debug-enter event if enabled.
  2. Opens the file at `path`, on error, reports via flow_report_error and returns the I/O error.
  3. Wraps file in BufReader and initializes SourceExpressionLoader and a fresh TypeEnv.
  4. Iterates over each line: a. Splits line into whitespace tokens. b. For each `word`, calls handle_load_word, on Some(expression), emits debug-info and calls flow_interpret, stopping if Exit or ErrorsOccured. c. After tokens, calls line_end, on Some(expression) interprets.
  5. After EOF, if any tokens remain, flush them.

1.2.3. interpreter

Current location: ./src/interpreter

Contains the source for the flow lisp language interpreter.

1.2.3.1. Submodules
1.2.3.1.1. flowlexer

Current location: ./src/interpreter/lexer

This source file implements the lexer for the shells interpreter. It splits input lines into tokens, classifies identifiers, integers, string literals, and parentheses, and supports debug tracing.

1.2.3.1.1.1. Enums
1.2.3.1.1.1.1. TokenType

Represents the kinds of lexemes recognized by the lexer.

Source: flowlexer.rs

#[derive(Debug, Clone, PartialEq, Eq)]  
pub enum TokenType {  
    LeftParen,    // (  
    RightParen,   // )  
    Identifier,  
    StringLiteral,  
    Integer,  
}  

Fields:

  1. LeftParen - single-character `(`.
  2. RightParen - single-character `)`.
  3. Identifier - names and symbols not matching other types.
  4. StringLiteral - text enclosed in double quotes.
  5. Integer - sequences of ASCII digits.
1.2.3.1.1.2. Structs
1.2.3.1.1.2.1. Token

Holds a lexeme with its type and source location.

Source: flowlexer.rs

#[derive(Debug, Clone, PartialEq)]  
pub struct Token {  
    pub token_type: TokenType,  
    pub lexeme: String,  
    pub line_number: u32,  
    pub token_start_number: u32,  
}  

Fields:

  1. token_type: TokenType - kind of the token.
  2. lexeme: String - the exact substring from the source line.
  3. line_number: u32 - 1-based line index where the token was found.
  4. token_start_number: u32 - 0-based character index of the tokens start.
1.2.3.1.1.3. Functions
1.2.3.1.1.3.1. push identifier

Classifies a raw substring as `Identifier` or `Integer` and appends it to the token list.

Source: flowlexer.rs

fn push_identifier(  
    tokens: &mut Vec<Token>,  
    text: &str,  
    line_number: u32,  
    start: u32  
) -> () 

Arguments:

  1. tokens: &mut Vec<Token> - buffer to receive the new token.
  2. text: &str - the substring to classify.
  3. line_number: u32 - source line number.
  4. start: u32 - character index of `text` start.

Behavior:

  • If `text` is all ASCII digits, token type is `Integer`, otherwise `Identifier`.
  • Creates and pushes a `Token` with the given location and lexeme.
1.2.3.1.1.3.2. flow tokenize

Splits a line of source into a vector of `Token`, handling strings, comments, and grouping.

Source: flowlexer.rs

pub fn flow_tokenize(  
    line: &str,  
    line_number: u32,  
    debug_mode: bool  
) -> Vec<Token>

Arguments:

  1. line: &str - the raw source line.
  2. line_number: u32 - 1-based index of the line.
  3. debug_mode: bool - if `true`, emits debug-enter and debug-exit events.

Return type:

  • Vec<Token> a list of tokens extracted from `line`.

Behavior:

  1. Iterates over characters, maintaining state for:
    • in_string for text inside `" "`.
    • in_comment and in_single_line_comment for `;` comments.
    • current buffer accumulating identifier/number text.
    • Positions string_start and token_start.
  2. On encountering delimiters, flushes current via push_identifier as needed, pushes single-char tokens, or enters/exits string/comment modes.
  3. After the loop, flushes any remaining identifier/number token.
  4. Returns the token vector.
1.2.3.1.2. typeenv

Current location: ./src/interpreter/typeenv

This source file implements the type system and environment for the shell interpreter. It defines a `Type` enum for expression types, a scoped `TypeEnv` for bindings, and functions to check and record variable definitions.

1.2.3.1.2.1. Enums
1.2.3.1.2.1.1. Type

Represents the inferred or annotated type of an expression.

Source: flowtypeenv.rs

#[derive(Debug, Clone, PartialEq)]  
pub enum Type {  
    Integer(i64),        // integer literal with its value  
    Boolean(bool),       // boolean literal  
    String(String),      // string literal  
    Identifier(String),  // user-defined identifier type  
    Function(Vec<Type>, Box<Type>), // function: arg types and return type  
    Alias(String),       // alias name for another type  
    Unknown,             // runtime-inferred or undetermined type  
}  

Variants:

  1. Integer(i64) - literal 64 bit integer value.
  2. Boolean(bool) - literal boolean value.
  3. String(String) - literal string data.
  4. Identifier(String) - refers to a named type or variable.
  5. Function(Vec<Type>, Box<Type>) - function type signature.
  6. Alias(String) - named alias for another type.
  7. Unknown - placeholder for types to be inferred at runtime.
1.2.3.1.2.2. Structs
1.2.3.1.2.2.1. TypeEnv

Manages a stack of scopes mapping names to `Type`.

Source: typeenv.rs

#[derive(Debug)]  
pub struct TypeEnv {  
    scopes: Vec<HashMap<String, Type>>,  
}  

Fields:

  1. scopes: Vec<HashMap<String, Type>> - stack of hash maps for nested scopes.

Methods:

  • new() - initializes with a single empty scope.
  • enter_scope() - pushes a new, inner scope.
  • exit_scope() - pops the current scope.
  • insert(name: String, var_value: Type) - binds name to var_value in the current scope.
  • lookup(name: &str) -> Option<&Type> - searches from innermost to outermost scope.
1.2.3.1.2.3. Functions
1.2.3.1.2.3.1. handle define var

Validates an optional annotation against an inferred type and records the binding.

Source: typechecker.rs

pub fn handle_define_var(  
    name: String,  
    var_type: Option<Type>,  
    value: Type,  
    env: &mut TypeEnv,  
) -> Result<(), String>

Arguments:

  1. name: String - variable name.
  2. var_type: Option<Type> - optional user annotation.
  3. value: Type - inferred type of the initializer.
  4. env: &mut TypeEnv - environment to bind the variable.

Behavior:

  • If var_type is Some(annot), ensures value == annot, returning an `Err` on mismatch.
  • Inserts the binding into the current scope.
  • Returns Ok(()) on success or Err(String).
1.2.3.1.2.3.2. type check

Performs type inference and checking on an `Expression`.

Source: typechecker.rs

pub fn type_check(  
    expr: &Expression,  
    env: &mut TypeEnv  
) -> Result<Type, String> 

Arguments:

  1. expr: &Expression - AST node to check.
  2. env: &mut TypeEnv - current type environment.

Behavior:

  • For `Integer`, `StringLiteral`, and `Bool`, returns the corresponding `Type` variant.
  • For Identifier(name), looks up `name` in `env`, cloning and returning the bound `Type`, or `Err` if unbound.
  • Returns `Err` for unimplemented AST variants.
1.2.3.1.3. flowparser

Current location: ./src/interpreter/parser

This source file implements the parser for the shells interpreter. It consumes a token stream produced by the lexer, builds an abstract syntax tree (AST) of `Expression` nodes, and prepares constructs for type checking and evaluation. It handles literals, variable/environment/alias definitions, commands with piping and redirection, lists of expressions, and error recovery.

1.2.3.1.3.1. Enums
1.2.3.1.3.1.1. ExpressionType

Represents the different kinds of expressions the parser recognizes.

Source: flowparser.rs

#[derive(Debug, Clone, PartialEq)]  
pub enum ExpressionType {  
    // atoms  
    Integer(i64),  
    StringLiteral(String),  
    Identifier(String),  
    Bool(bool),  

    // reserved keywords  
    DefineVar {  
        name: String,  
        var_type: Option<Type>,    // optional annotation, always inferred at runtime for now  
        value: Box<Expression>  
    },  
    DefineEnvar {  
        name: String,  
        value: Box<Expression>,  
    },  

    DefineAlias {  
        name: String,  
        value: Box<Expression>,  
    },  

    // for shell purposes  
    Command {  
        name: String,  
        params: Vec<Expression>,  
        pipe: Option<Box<Expression>>,  
        redirect: Option<(RedirectionType, PathBuf)>,  
    },  

    // represents a parsed file  
    List(Vec<Expression>),  

    // for partial evaluation and error aggregation  
    Error(ErrorExp),  
    Unknown,  
}  

Variants:

  1. Integer(i64) - numeric literal.
  2. StringLiteral(String) - text enclosed in quotes.
  3. Identifier(String) - names, variable references, and symbols.
  4. Bool(bool) - boolean literal (`true` or `false`).
  5. DefineVar - variable definition with optional type annotation and initializer.
  6. DefineEnvar - environment variable definition with initializer.
  7. DefineAlias - alias definition for commands.
  8. Command - shell command invocation, with parameters, optional pipe target, and optional redirection.
  9. List(Vec<Expression>) - top-level sequence of expressions
  10. Error(ErrorExp) - placeholder for a parse error, captures location and message.
  11. Unknown - fallback for unrecognized constructs.
1.2.3.1.3.1.2. RedirectionType

Specifies kinds of I/O redirection in a command.

Source: flowparser.rs

#[derive(Debug, Clone, PartialEq)]  
pub enum RedirectionType {  
    Input,   
    Output {  
        append: bool,  
        stderr: bool,  
        stdin: bool,  
    },  
}  

Variants:

  1. Input - `< file` redirect for stdin.
  2. Output - `>` or `>>` or variants:
    • append: if true, use `>>`;
    • stderr: if true, redirect stderr, use `2>`;
    • stdin: if true, redirect stdin as well, use `&>`;
1.2.3.1.3.1.3. KeywordDefineType

Categorizes the define-keyword forms.

Source: flowparser.rs

#[derive(Debug, Clone, PartialEq)]  
pub enum KeywordDefineType {  
    Var,  
    Envar,  
    Alias,  
}  

Variants:

  1. Var - define `var`.
  2. Envar - define `envar`.
  3. Alias - define `alias`.
1.2.3.1.3.1.4. KeywordRedirectType

Enumerates redirection keywords recognized during parsing.

Source: flowparser.rs

#[derive(Debug, Clone, PartialEq)]  
pub enum KeywordRedirectType {  
    Input,  
    Output,  
    OutputAppend,  
    OutputStderr,  
    OutputCombined,  
}  

Variants correspond to `<`, `>`, `>>`, `2>`, and `&>` syntax.

1.2.3.1.3.1.5. KeywordType

Higher-level classification for parser tokens corresponding to keywords.

Source: flowparser.rs

#[derive(Debug, Clone, PartialEq)]  
pub enum KeywordType {  
    Define(KeywordDefineType),  
    Command,  
    Pipe,  
    Redirect(KeywordRedirectType),  
}  

Variants:

  1. Define - wraps a `KeywordDefineType`.
  2. Command - invocation keyword.
  3. Pipe - `|`.
  4. Redirect - wraps a `KeywordRedirectType`.
1.2.3.1.3.2. Structs
1.2.3.1.3.2.1. Expression

Holds a parsed expression node and its source token for location tracking.

Source: flowparser.rs

#[derive(Debug, Clone, PartialEq)]  
pub struct Expression {  
    pub exp_start_token: Option<Token>,  
    pub exp_type: ExpressionType,  
}  

Fields:

  1. exp_start_token: Option<Token> - the first token of this expression (for error reporting).
  2. exp_type: ExpressionType - the actual AST node kind and data.
1.2.3.1.3.2.2. ErrorExp

Captures information about a parse-time error as an expression.

Source: flowparser.rs

#[derive(Debug, Clone, PartialEq)]  
pub struct ErrorExp {  
    pub token: Option<Token>,  
    pub error_location: String,  
    pub message: String,  
}  

Fields:

  1. token: Option<Token> - the token where the error was detected (if available).
  2. error_location: String - textual description of the error position
  3. message: String - human-readable error message.
1.2.3.1.3.3. Functions
1.2.3.1.3.3.1. parse s expression

First pass turns a flat &[Token] slice into a nested Expression::List structure, detecting parentheses depth and basic atom forms.

Source: flowparser.rs

fn parse_s_expression(  
    tokens: &[Token],  
    pos: usize,  
    has_errors: &mut bool  
) -> (Expression, usize)

Arguments:

  1. tokens: &[Token] - the token stream from the lexer.
  2. pos: usize - current parse position in `tokens`.
  3. has_errors: &mut bool - accumulates parse errors.

Behavior:

  • On `LeftParen`, enters a loop to parse sub-expressions until a matching `RightParen`.
  • Recursively calls itself for nested lists.
  • Converts `Identifier`, `Integer`, and `StringLiteral` tokens into their respective `ExpressionType` variants
  • Returns an `Expression` and the next token position.
  • Reports errors for unexpected tokens, unclosed parentheses, and EOF.
1.2.3.1.3.3.2. expand parsed s expression

Second pass: walks the raw Expression::List, recognizes keywords, and invokes specialized parsers (`parse_define`, `preproc_parse_cmd`). It also checks variable bindings and expands nested lists.

Source: flowparser.rs

fn expand_parsed_s_expression(  
    exp: Expression,  
    keyword_type_map: &HashMap<&str, KeywordType>,  
    type_env: &mut TypeEnv,  
    has_errors: &mut bool,  
) -> Expression 

Arguments:

  1. exp: Expression - the raw AST from parse_s_expression.
  2. keyword_type_map: &HashMap<&str, KeywordType> - maps symbols to parsing actions (define, command, pipe, redirect).
  3. type_env: &mut TypeEnv - for checking that identifiers are defined.
  4. has_errors: &mut bool - records any new errors.

Behavior:

  • Propagates existing `Error` expressions unchanged.
  • On ExpressionType::List(items):
    • If the first element is a keyword identifier, dispatches to parse_define or preproc_parse_cmd.
    • Otherwise, ensures the identifier is defined in type_env or emits an error.
    • Recursively expands each element to produce a new `List`.
  • Atoms (`Integer`, `StringLiteral`, `Identifier`, `Bool`) are returned as-is.
1.2.3.1.3.3.3. parse define

Parses a var/envar/alias definition S-expression into a corresponding `ExpressionType`.

Source: flowparser.rs

fn parse_define(  
    items: Vec<Expression>,  
    keyword_define_type: &KeywordDefineType,  
    keyword_type_map: &HashMap<&str, KeywordType>,  
    type_env: &mut TypeEnv,  
    has_errors: &mut bool,  
) -> Expression 

Arguments:

  1. items: Vec<Expression> - the three-element list: (define-keyword name expr value expr).
  2. keyword_define_type: &KeywordDefineType - which define form (Var, Envar, Alias).
  3. keyword_type_map: &HashMap<&str, KeywordType> - lookup for nested expansions.
  4. type_env: &mut TypeEnv - environment for type inference/checking.
  5. has_errors: &mut bool - flag to record parse or type errors.

Behavior:

  • Extracts and validates the identifier
  • Recursively expands the value expression via expand_parsed_s_expression.
  • Depending on keyword_define_type, constructs one of the corresponding ExpressionType.
  • Returns an Expression with exp_start_token from the keyword and the chosen ExpressionType.
1.2.3.1.3.3.4. parse cmd

Processes a flat list of expressions into a shell `Command` AST node, handling parameters, piping, and redirection.

Source: flowparser.rs

fn parse_cmd(  
    items: &Vec<Expression>,  
    index: usize,  
    keyword_type_map: &HashMap<&str, KeywordType>,  
    type_env: &mut TypeEnv,  
    has_errors: &mut bool,  
) -> Expression

Arguments:

  1. items: &Vec<Expression> - sequence of expanded subexpressions.
  2. index: usize - start index of the command name in `items`.
  3. keyword_type_map: &HashMap<&str, KeywordType> - kind of database that matches `keyword` to its `KeywordType`.
  4. type_env: &mut TypeEnv - for type checks of parameters.
  5. has_errors: &mut bool - accumulates any errors encountered.

Behavior:

  • Validates the command name is an `Identifier`.
  • Iterates over subsequent `items`, distinguishing:
    • Pipe tokens: recursing to parse the next command into `pipe`.
    • Redirection keywords: recording a `RedirectionType` and expecting the next token as a path.
    • Other expressions, expanded via expand_parsed_s_expression, collected into `params`.
  • Reports errors
  • Constructs ExpressionType::Command { name, params, pipe, redirect }.
1.2.3.1.3.3.5. preproc parse cmd

Initial wrapper that validates the S-expression for a command and invokes parse_cmd.

Source: flowparser.rs

fn preproc_parse_cmd(  
    raw_items: Vec<Expression>,  
    keyword_type_map: &HashMap<&str, KeywordType>,  
    type_env: &mut TypeEnv,  
    has_errors: &mut bool,  
) -> Expression 

Arguments:

  1. raw_items: Vec<Expression> - top-level S-expression where raw_items[1] should be a `List`.
  2. keyword_type_map: &HashMap<&str, KeywordType> - kind of database that matches `keyword` to its `KeywordType`.
  3. type_env: &mut TypeEnv - lookup for nested expansions.
  4. has_errors: &mut bool - flag to record parse or type errors.

Behavior:

  • Confirms raw_items contains at least two elements and that raw_items[1] is ExpressionType::List.
  • Extracts the inner Vec<Expression> and validates it is non-empty.
  • Delegates to parse_cmd with that vector starting at index `0`.
  • Returns an Error expression on invalid structure or command list.
1.2.3.1.3.3.6. flow parse

Top-level entry: orchestrates token-to-AST conversion, error collection, and optional debug tracing.

Source: flowparser.rs

pub fn flow_parse(  
    tokens: &[Token],  
    type_env: &mut TypeEnv,  
    debug_mode: bool,  
) -> (Expression, bool) { \u2026 }  

Arguments:

  1. tokens: &[Token] - input token stream.
  2. type_env: &mut TypeEnv - initial type environment.
  3. debug_mode: bool - if `true`, emits enter/exit debug events.

Behavior:

  • Builds a keyword_map of reserved words.
  • Calls parse_s_expression to get raw AST and updates has_errors.
  • Calls expand_parsed_s_expression to produce the final AST, dispatching to specialized parsers.
  • Returns the final `Expression` and has_errors flag.
1.2.3.1.4. executer

Current location: ./src/interpreter/executer

This source file implements the runtime execution engine: launching processes, managing pipelines, built-in commands, error reporting, glob expansion, redirections, and evaluation via `FlowState`.

1.2.3.1.4.1. Submodules
1.2.3.1.4.1.1. builtincommands

Current location: ./src/interpreter/executer/builtincommands

This source file implements detection and execution of built-in shell commands for the interpreter.

1.2.3.1.4.1.2. Enums
1.2.3.1.4.1.2.1. BuiltInCommandType

Identifies the supported built-in commands.

Source: flowbuiltincommands.rs

#[derive(PartialEq, Clone)]  
pub enum BuiltInCommandType {  
    Exit,        // exit the shell  
    Help,        // display help text  
    ChangeDir,   // cd <dir>  
    None,        // not a built-in  
}  

Variants:

  1. Exit - terminate the shell loop.
  2. Help - show usage information.
  3. ChangeDir - change the current working directory.
  4. None - indicates the command is not built-in.
1.2.3.1.4.1.3. Functions
1.2.3.1.4.1.3.1. is built in command

Maps a command name to its `BuiltInCommandType`.

Source: flowbuiltincommands.rs

pub fn is_built_in_command(name: &String) -> BuiltInCommandType {  
    match name.to_lowercase().as_str() {  
        "exit" => BuiltInCommandType::Exit,  
        "help" => BuiltInCommandType::Help,  
        "cd"   => BuiltInCommandType::ChangeDir,  
        _      => BuiltInCommandType::None,  
    }  
}  

Arguments:

  1. name: &String - command name string (case-insensitive).

Behavior: Returns the corresponding `BuiltInCommandType`.

1.2.3.1.4.1.3.2. execute built in command

Executes the specified built-in command with given parameters.

Source: flowbuiltincommands.rs

pub fn execute_built_in_command(  
    kind: &BuiltInCommandType,  
    params: Vec<String>,  
) -> Result<(), String> 

Arguments:

  1. kind: &BuiltInCommandType - which built-in to run.
  2. params: Vec<String> - command arguments.

Behavior:

  • Exit: returns Ok(()), caller interprets this as an exit signal.
  • Help: prints help text to stdout.
  • ChangeDir: expects one parameter, the target directory; calls env::set_current_dir.
  • None: does nothing and returns Ok(()).
1.2.3.1.4.2. Enums
1.2.3.1.4.2.1. FlowState

Differentiates shell-loop control flow and error handling.

Source: flowexecuter.rs

// differentiate shell-loop states
pub enum FlowState {
    FreeToMove,
    ErrorsOccured,
    Return(Type),
    Error {
        message: String,
        exp: Option<Expression>,
    },
    Exit,
}

Variants:

  1. FreeToMove - ready for the next command.
  2. ErrorsOccured - one or more errors have occurred, skip to error handler.
  3. Return(Type) - a return from a function or script, carrying its Type
  4. Error { message, exp } - runtime error with a message and optional Expression reference.
  5. Exit - terminate the shell loop and exit.
1.2.3.1.4.3. Functions
1.2.3.1.4.3.1. report all parse errors

Recursively reports all parse-time errors in an AST.

Source: flowexecuter.rs

fn report_all_parse_errors(
    parsed_exp: Expression,
) -> ()

Arguments:

  1. parsed_exp: Expression - AST to inspect.

Behavior:

  • On ExpressionType::Error(err), extracts line, column, location, and message then calls flow_report_error.
  • On ExpressionType::List(items), recurses into each sub-expression.
  • Ignores other.
1.2.3.1.4.3.2. expand globs

Performs filesystem globbing on command arguments.

Source: flowexecuter.rs

fn expand_globs(
    args: Vec<String>
) -> Result<Vec<String>, String>

Arguments:

  1. args: Vec<String> - raw argument strings.

Behavior:

  • For each argument containing `*`, `?`, or `[`, uses glob(&arg).
  • On matches, adds each paths string form.
  • Returns Ok(expanded_args) on success.
1.2.3.1.4.3.3. execute command

Forks and executes an external command with redirection and optional piping.

Source: flowexecuter.rs

fn execute_command(
    name: &str,
    params: Vec<String>,
    redirect_stderr: Option<PathBuf>,
    redirect_stdout: Option<PathBuf>,
    redirect_stdin: Option<PathBuf>,
    mut prev_pipe_output: Option<Stdio>,
    pipe_stdout: bool,
) -> Result<io::Result<Child>, String>

Arguments:

  1. name: &str - program to run.
  2. params: Vec<String> - command-line arguments.
  3. redirect_stderr: Option<PathBuf> - file for stderr.
  4. redirect_stdout: Option<PathBuf> - file for stdout.
  5. redirect_stdin: Option<PathBuf> - file for stdin.
  6. prev_pipe_output: Option<Stdio> - upstream pipe as stdin.
  7. pipe_stdout: bool - whether to pipe this processs stdout.

Behavior:

  • Constructs Command::new(name).
  • Sets up `stdin`, `stdout`, `stderr` based on redirection or piping.
  • Appends arguments and spawns the child process.
  • Returns nested `Result` to distinguish io errors and spawn errors.
1.2.3.1.4.3.4. handle command execution

Evaluates and executes a shell command AST node, handling built-ins, glob expansion, redirection, and pipelines.

Source: flowexecuter.rs

fn handle_command_execution(
    name: String,
    raw_params: Vec<String>,
    raw_redirect: Option<(RedirectionType, PathBuf)>,
    mut prev_pipe_output: Option<Stdio>,
    next_pipe: Option<Box<Expression>>,
    type_env: &mut TypeEnv,
) -> FlowState 

Arguments:

  1. name: String - command identifier.
  2. raw_params: Vec<String> - unexpanded argument patterns.
  3. raw_redirect: Option<(RedirectionType, PathBuf)> - optional redirection.
  4. prev_pipe_output: Option<Stdio> - pipe input from previous command.
  5. next_pipe: Option<Box<Expression>> - AST for next piped command.
  6. type_env: &mut TypeEnv - environment for var/envar lookup.

Behavior:

  • Calls expand_globs.
  • Detects built-in commands via is_built_in_command, dispatches to execute_built_in_command.
  • Parses raw_redirect into separate stdin/stdout/stderr paths, errors on conflicts.
  • Invokes execute_command, checks spawn and I/O errors
  • If next_pipe exists, capture stdout, evaluate its parameters, and recurses into handle_command_execution.
  • On success of command, returns FlowState::FreeToMove.
1.2.3.1.4.3.5. eval expression

Recursively evaluates any `Expression`, returning a `FlowState`.

Source: flowexecuter.rs

fn eval_expression(
    parsed_exp: Expression,
    type_env: &mut TypeEnv,
) -> FlowState

Arguments:

  1. parsed_exp: Expression - AST node to evaluate.
  2. type_env: &mut TypeEnv - for variable and type lookups.

Behavior:

  • Evaluates based on ExpressionType.
  • Propagates errors as FlowState::Error.
1.2.3.1.4.3.6. flow interpret

Top-level entry: tokenizes, parses, reports errors, and evaluates a single input line.

Source: flowexecuter.rs

pub fn flow_interpret(
    line: &str,
    line_number: u32,
    type_env: &mut TypeEnv,
    debug_mode: bool
) -> FlowState 

Arguments:

  1. line: &str - raw input text.
  2. line_number: u32 - for error reporting.
  3. type_env: &mut TypeEnv - current type environment.
  4. debug_mode: bool - toggles debug tracing.

Behavior:

  • Ignores lines starting with `;` (commends).
  • Wraps non-parenthesized lines into (exec ({line})) form.
  • Tokenizes via flow_tokenize, parses with flow_parse.
  • On parse errors, calls report_all_parse_errors and returns ErrorsOccured.
  • Otherwise, calls eval_expression, reports runtime errors.
  • Returns the resulting `FlowState`.

1.3. The Main Program Loop

1.3.0.0.1. main

Current location: ./src/

This source implements the shells startup lifecycle: argument parsing, .flowrc loading, interactive REPL, option to execute from a script.

1.3.0.0.1.1. Description

The main function drives the shell through four phases:

  • CLI argument processing
  • RC configuration file handling
  • Script file execution (if any)
  • REPL invocation otherwise
1.3.0.0.1.2. Functions
1.3.0.0.1.2.1. main

Entry point. Coordinates all phases of the shell lifecycle.

Source: main.rs

fn main()

Arguments: None - reads from environment and CLI.

Behavior:

  1. Parses CLI flags and file paths into local variables.
  2. Ensures the RC file exists, creating it if necessary.
  3. Loads and executes the RC file via flowloopfile.
  4. If script files were passed, runs each via flowloopfile in the order that they came in.
  5. Otherwise, starts the interactive REPL via flowlooprepl.

Author: Marko Štajgár MarkoStajgar

Last Updated: 2025-08-25 Mon 10:31

Emacs 29.4 (Org mode 9.6.15)

Validate