6.1.1.2.1.1.2.10. pycropml.transpiler.antlr_py.parse module

class pycropml.transpiler.antlr_py.parse.AliasNode(node: BaseNode, fields: Dict[str, Any] | None = None)

Bases: BaseNode

classmethod bind_to_transformer(transformer_cls: Type[BaseNodeTransformer], default_transform_method: str = 'from_spec')

classmethod from_spec(node: BaseNode)

classmethod get_path(node: BaseNode, path: List[str])

classmethod get_transformer(method_name: str)

Get method to bind to visitor

exception pycropml.transpiler.antlr_py.parse.AntlrException(msg, orig)

Bases: Exception

pycropml.transpiler.antlr_py.parse.AstNode

alias of BaseNode

class pycropml.transpiler.antlr_py.parse.AstNodeMeta

Bases: type

class pycropml.transpiler.antlr_py.parse.BaseAstVisitor(registry: BaseNodeRegistry)

Bases: ParseTreeVisitor

Visitor that creates a high level tree ~ ANTLR tree serializer + automatic node creation using field and label detection + alias nodes can work on tree without (ANTLR) visitor Used from BaseAstVisitor: visitTerminal, visitErrorNode .. todo:

- [done] support labels
- [done] make compatible with AST: _fields = () (should only every child once)
- [done] include child_index to filter unique elements + order
- [done] memoize dynamic classes, to have list + make instance checks work
- [done] tree simplification as part of AliasNode
- [done] flatten nested list (see select with dynamic clause ordering)
- combine terminals / error nodes
- serialize highlight info
- [done] make compatible with AstNode & AstModule in protowhat (+ shellwhat usage: bashlex + osh parser)
    - combining fields & labels dicts needed?
- use exact ANTLR names in _rules (capitalize name without changing other casing)
- add labels to _fields if not overlapping with fields from rules
- [done] eliminate overhead of alias parsing (store ref to child index, get children on alias access)
- [necessary?] grammar must use lexer or grammar rules for elements that should be in the tree
  and literals for elements that cannot
  currently:
  - Use AliasNode to add labels to _fields, define custom fields and omit fields
  - Use Transformer to replace a node by a combination of fields
- [rejected] alternative dynamic class naming:
  - pass parse start to visitor constructor, use as init for self.current_node
  - set self.current_node to field.__name__ before self.visit_field
  - use self.current_node to create dynamic classes
  (does not use #RuleAlias names in grammar)
  (other approach: transforming returned dict, needs more work for arrays + top level)

Higher order visitor (or integrated) - [alternative] allow node aliases (~ AstNode._rules) by dynamically creating a class inheriting from the dynamic node class

(multiple inheritance if node is alias for multiple nodes, class has combined _fields for AST compatibility

• [alternative] allow field aliases using .aliases property with defaultdict(list) (~ AstNode._fields_spec)

  • dynamic fields? (~ visit_path)

test code in parse:

tree = parse_ast(grammar, sql_text, start, **kwargs) field_tree = BaseAstVisitor().visit(tree) alias_tree = AliasVisitor(Transformer()).visit(field_tree) import ast nodes = [el for el in ast.walk(field_tree)] import json json_str = json.dumps(field_tree, default=lambda o: o.to_json())

visitChildren(node: ParserRuleContext, predicate=None, simplify=False) → BaseNode

visitErrorNode(node: ErrorNode)

visitTerminal(ctx: ParserRuleContext) → Terminal

Converts case insensitive keywords and identifiers to lowercase

class pycropml.transpiler.antlr_py.parse.BaseNode(children: list, field_references: Dict[str, int | List[int]], label_references: Dict[str, int | List[int]], ctx: ParserRuleContext | None = None, position: dict | None = None, text: str | None = None)

Bases: AST

AST is subclassed so we can use Python ast module visiting and walking on the custom AST

classmethod combine(*fields: BaseNode) → List[BaseNode]

Combine fields Creates a list field from other fields Filters None and combines other elements in a flat list Use in transformer methods.

classmethod create(ctx: ParserRuleContext, children: list | None = None, registry: BaseNodeRegistry | None = None) → BaseNode

classmethod create_cls(cls_name: str, field_names: tuple) → Type[BaseNode]

static extend_node_list(acc: List[BaseNode], new: List[BaseNode] | BaseNode) → List[BaseNode]

Extend accumulator with Node(s) from new

get_position() → Dict[str, int] | None

get_text(full_text: str = None) → str | None

class pycropml.transpiler.antlr_py.parse.BaseNodeRegistry

Bases: object

get_cls(cls_name: str, field_names: tuple) → Type[BaseNode]

isinstance(instance: BaseNode, class_name: str) → bool

Check if a BaseNode is an instance of a registered dynamic class

class pycropml.transpiler.antlr_py.parse.BaseNodeTransformer(registry: BaseNodeRegistry)

Bases: NodeTransformer

classmethod bind_alias_nodes(alias_classes: List[Type[AliasNode]])

visit(node: BaseNode)

Visit a node.

visit_Terminal(terminal: Terminal) → Terminal

Handle Terminal the same as other non-node types

class pycropml.transpiler.antlr_py.parse.CaseTransformInputStream(*args, transform=None, **kwargs)

Bases: InputStream

Support case insensitive languages https://github.com/antlr/antlr4/blob/master/doc/case-insensitive-lexing.md#custom-character-streams-approach

LOWER = 'lower'

UPPER = 'upper'

class pycropml.transpiler.antlr_py.parse.FieldSpec(name, origin)

Bases: tuple

name

Alias for field number 0

origin

Alias for field number 1

class pycropml.transpiler.antlr_py.parse.LexerErrorListener

Bases: ConsoleErrorListener

syntaxError(recognizer, offendingSymbol, line, column, msg, e)

class pycropml.transpiler.antlr_py.parse.Speaker(**cfg)

Bases: object

describe(node, fmt='{node_name}', field=None, **kwargs)

static get_info(node_cfg)

Return a tuple with the verbal name of a node, and a dict of field names.

class pycropml.transpiler.antlr_py.parse.StrictErrorListener

Bases: ErrorListener

reportAmbiguity(recognizer, dfa, startIndex, stopIndex, exact, ambigAlts, configs)

reportAttemptingFullContext(recognizer, dfa, startIndex, stopIndex, conflictingAlts, configs)

reportContextSensitivity(recognizer, dfa, startIndex, stopIndex, prediction, configs)

syntaxError(recognizer, badSymbol, line, col, msg, e)

class pycropml.transpiler.antlr_py.parse.Terminal(*args, **kwargs)

Bases: BaseNode

This is a thin node wrapper for a string. The node is transparent when not in debug mode. In debug mode, it keeps the link to the corresponding ANTLR node.

DEBUG = True

DEBUG_INSTANCES = ['using', 'System', ';', 'using', 'Models', '.', 'Core', ';', 'namespace', 'Models', '.', 'Interfaces', '{', 'public', 'interface', 'IWeather', '{', 'DateTime', 'StartDate', '{', 'get', ';', '}', 'DateTime', 'EndDate', '{', 'get', ';', '}', 'double', 'MaxT', '{', 'get', ';', 'set', ';', '}', 'double', 'MinT', '{', 'get', ';', 'set', ';', '}', 'double', 'MeanT', '{', 'get', ';', '}', 'double', 'VPD', '{', 'get', ';', '}', 'double', 'Rain', '{', 'get', ';', 'set', ';', '}', 'public', 'double', 'PanEvap', '{', 'get', ';', 'set', ';', '}', 'double', 'Radn', '{', 'get', ';', 'set', ';', '}', 'double', 'VP', '{', 'get', ';', 'set', ';', '}', 'double', 'Wind', '{', 'get', ';', 'set', ';', '}', 'double', 'CO2', '{', 'get', ';', 'set', ';', '}', 'double', 'AirPressure', '{', 'get', ';', 'set', ';', '}', 'double', 'DiffuseFraction', '{', 'get', ';', 'set', ';', '}', 'double', 'Latitude', '{', 'get', ';', '}', 'double', 'Longitude', '{', 'get', ';', '}', 'double', 'Tav', '{', 'get', ';', '}', 'double', 'Amp', '{', 'get', ';', '}', 'string', 'FileName', '{', 'get', ';', '}', 'double', 'CalculateDayLength', '(', 'double', 'Twilight', ')', ';', 'double', 'CalculateSunRise', '(', ')', ';', 'double', 'CalculateSunSet', '(', ')', ';', 'DailyMetDataFromFile', 'TomorrowsMetData', '{', 'get', ';', '}', 'DailyMetDataFromFile', 'YesterdaysMetData', '{', 'get', ';', '}', '}', '[', 'Serializable', ']', 'public', 'class', 'DailyMetDataFromFile', ':', 'Model', '{', 'public', 'double', 'MaxT', '{', 'get', ';', 'set', ';', '}', 'public', 'double', 'MinT', '{', 'get', ';', 'set', ';', '}', 'public', 'double', 'PanEvap', '{', 'get', ';', 'set', ';', '}', 'public', 'double', 'Rain', '{', 'get', ';', 'set', ';', '}', 'public', 'double', 'Radn', '{', 'get', ';', 'set', ';', '}', 'public', 'double', 'VP', '{', 'get', ';', 'set', ';', '}', 'public', 'double', 'Wind', '{', 'get', ';', 'set', ';', '}', 'public', 'double', 'RainfallHours', '{', 'get', ';', 'set', ';', '}', 'public', 'double', 'AirPressure', '{', 'get', ';', 'set', ';', '}', 'public', 'double', 'DiffuseFraction', '{', 'get', ';', 'set', ';', '}', 'public', 'double', 'DayLength', '{', 'get', ';', 'set', ';', '}', 'public', 'double', 'CO2', '{', 'get', ';', 'set', ';', '}', 'public', 'object', '[', ']', 'Raw', '{', 'get', ';', 'set', ';', '}', '}', '[', 'Serializable', ']', 'public', 'class', 'WeatherRecordEntry', '{', 'public', 'DateTime', 'Date', '{', 'get', ';', 'set', ';', '}', 'public', 'DailyMetDataFromFile', 'MetData', '{', 'get', ';', 'set', ';', '}', '}', '}', '<EOF>', 'using', 'System', ';', 'using', 'APSIM', '.', 'Shared', '.', 'Utilities', ';', 'using', 'Models', '.', 'Core', ';', 'using', 'Models', '.', 'Interfaces', ';', 'namespace', 'Models', '.', 'Toy', '{', '[', 'Serializable', ']', '[', 'ValidParent', '(', 'ParentType', '=', 'typeof', '(', 'Zone', ')', ')', ']', 'public', 'class', 'Toy1', ':', 'Model', '{', '[', 'Link', ']', 'private', 'IWeather', 'weather', '=', 'null', ';', '[', 'Description', '(', '"Vapour pressure deficit"', ')', ']', '[', 'Units', '(', '"hPa"', ')', ']', 'public', 'double', 'VPD', '{', 'get', ';', 'private', 'set', ';', '}', '[', 'EventSubscribe', '(', '"StartOfSimulation"', ')', ']', 'public', 'void', 'OnStartOfSimulation', '(', 'object', 'sender', ',', 'EventArgs', 'args', ')', '{', '}', '[', 'EventSubscribe', '(', '"StartOfDay"', ')', ']', 'public', 'void', 'OnStartOfDay', '(', 'object', 'sender', ',', 'EventArgs', 'args', ')', '{', 'const', 'double', 'SVPfrac', '=', '0.66', ';', 'double', 'VPDmint', '=', 'svp', '(', 'weather', '.', 'MinT', ')', '-', 'weather', '.', 'VP', ';', 'VPDmint', '=', 'Math', '.', 'Max', '(', 'VPDmint', ',', '0.0', ')', ';', 'double', 'VPDmaxt', '=', 'svp', '(', 'weather', '.', 'MaxT', ')', '-', 'weather', '.', 'VP', ';', 'VPDmaxt', '=', 'Math', '.', 'Max', '(', 'VPDmaxt', ',', '0.0', ')', ';', 'VPD', '=', 'SVPfrac', '*', 'VPDmaxt', '+', '(', '1', '-', 'SVPfrac', ')', '*', 'VPDmint', ';', '}', 'public', 'static', 'double', 'svp', '(', 'double', 'temp_c', ')', '{', 'return', '6.1078', '*', 'Math', '.', 'Exp', '(', '17.269', '*', 'temp_c', '/', '(', '237.3', '+', 'temp_c', ')', ')', ';', '}', '}', '}', '<EOF>']

classmethod from_text(text: str, ctx: ParserRuleContext | None = None)

class pycropml.transpiler.antlr_py.parse.TransformerHelper(registry: BaseNodeRegistry)

Bases: object

isinstance(*args)

pycropml.transpiler.antlr_py.parse.bind_to_transformer(transformer_cls: Type[BaseNodeTransformer], rule_name: str, transformer_method: Callable)

Assign AST node class constructors to parse tree visitors.

pycropml.transpiler.antlr_py.parse.dump_node(node, node_class=<class 'ast.AST'>)

pycropml.transpiler.antlr_py.parse.get_alias_nodes(items) → List[Type[BaseNode]]

pycropml.transpiler.antlr_py.parse.get_field(ctx: ParserRuleContext, field: str)

Helper to get the value of a field

pycropml.transpiler.antlr_py.parse.get_field_names(ctx: ParserRuleContext)

Get fields defined in an ANTLR context for a parser rule

pycropml.transpiler.antlr_py.parse.get_field_references(ctx: ParserRuleContext, field_names: List[str], simplify=False) → Dict[str, Any]

Create a mapping from fields to corresponding child indices

Parameters:

• ctx – ANTLR node

• field_names – list of strings

• simplify – if True, omits fields with empty lists or None this makes it easy to detect nodes that only use a single field but it requires more work to combine fields that can be empty

Returns:

mapping str -> int | int[]

pycropml.transpiler.antlr_py.parse.get_label_names(ctx: ParserRuleContext)

Get labels defined in an ANTLR context for a parser rule

pycropml.transpiler.antlr_py.parse.get_transformer_method_name(rule_name: str) → str

pycropml.transpiler.antlr_py.parse.langLexerParser(ant)

pycropml.transpiler.antlr_py.parse.materialize(reference_dict: Dict[str, int | List[int]], source: List[Any]) → Dict[str, Any]

Replace indices by actual elements in a reference mapping :param reference_dict: mapping str -> int | int[] :param source: list of elements :return: mapping str -> element | element[]

pycropml.transpiler.antlr_py.parse.parse_field_spec(spec: str) → FieldSpec

pycropml.transpiler.antlr_py.parse.parsef(code, language, start='compilation_unit', strict='False', transform: str | Callable = None, error_listener: ErrorListener = None)

pycropml.transpiler.antlr_py.parse.simplify_tree(tree, unpack_lists=True, in_list=False)

Recursively unpack single-item lists and objects where fields and labels only reference a single child :param tree: the tree to simplify (mutating!) :param unpack_lists: whether single-item lists should be replaced by that item :param in_list: this is used to prevent unpacking a node in a list as AST visit can’t handle nested lists