Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
calculate_python_expression [2026/08/18 20:55] hermann |
calculate_python_expression [2026/08/31 17:10] (current) hermann |
||
|---|---|---|---|
| Line 3: | Line 3: | ||
| ===== Description ===== | ===== Description ===== | ||
| - | This is a **[[ego_script#container_functors|container functor]]** that runs a Python instance with the user-defined expression. Like the other calculator functors, data is connected through hook functors placed inside its ''<nowiki>{{ … }}</nowiki>'' block. See also [[calculate_r_expression|Calculate R Expression]] for the equivalent functor using R. | + | This is a container functor that runs a Python instance with a user-defined expression. Like the other calculator functors, such as [[Calculate Map]] or [[Calculate Lookup Table]], data is connected through hook functors placed inside its block, rather than through regular input ports. See also [[Calculate R Expression]] for the equivalent functor using R. |
| ===== Inputs ===== | ===== Inputs ===== | ||
| - | ^ Name ^ Type ^ Description ^ | + | ^ Name ^ Type ^ Description ^ |
| - | | expression | [[code_type|Code]] | The expression that will run on Python. Written directly as a Code constant using its own raw string syntax — see [[#writing_the_expression_in_ego_script|Writing the expression in EGO Script]] below. | | + | | Expression | [[Code Type]] | The expression to run on Python. Written directly as a Code constant using its own raw string syntax -- see [[#writing_the_expression_in_ego_script|Writing the expression in EGO Script]] below. | |
| ===== Optional Inputs ===== | ===== Optional Inputs ===== | ||
| - | ^ Name ^ Type ^ Description ^ | + | ^ Name ^ Type ^ Description ^ Default Value ^ |
| - | | packages //(advanced)// | String | Required packages to be installed by PIP (one per line). Each package can be identified either by its name or by specifying a filename or URL pointing to the corresponding wheel file. Packages that are already installed will be ignored. | | + | | Packages | [[String Type]] | Packages to install with pip before the expression runs, one per line. Each package can be identified by name or by a filename or URL pointing to the corresponding wheel; a package already installed is skipped. This is an advanced port. | None | |
| ===== Outputs ===== | ===== Outputs ===== | ||
| - | ^ Name ^ Type ^ Description ^ | + | ^ Name ^ Type ^ Description ^ |
| - | | result | [[struct_type|Struct]] | A struct containing the output values generated by the expression. | | + | | Result | [[Struct Type]] | Struct containing the output values generated by the expression, one entry per key assigned into dinamica.outputs. | |
| + | |||
| + | ===== Group ===== | ||
| + | |||
| + | [[Functor List#Integration | Integration]] | ||
| ===== Notes ===== | ===== Notes ===== | ||
| Line 24: | Line 28: | ||
| ==== Expression inputs ==== | ==== Expression inputs ==== | ||
| - | Data is passed into the expression through **hook** functors placed inside the container's ''<nowiki>{{ … }}</nowiki>'' block — the same [[ego_script#verbose_form|verbose form]] hook mechanism used by the calculator functors: | + | Data is passed into the expression through hook functors placed inside the container's block -- the same verbose-form hook mechanism used by the calculator functors: |
| - | * Tables and lookup tables → [[Number Table]] → available in the expression as ''dinamica.inputs["t1"]'', ''dinamica.inputs["t2"]'', …, ''dinamica.inputs["t100"]'' | + | * Tables and lookup tables → [[Number Table]] → available in the expression as dinamica.inputs["t1"], dinamica.inputs["t2"], …, dinamica.inputs["t100"] |
| - | * Scalar values → [[Number Value]] → available as ''dinamica.inputs["v1"]'', ''dinamica.inputs["v2"]'', …, ''dinamica.inputs["v100"]'' | + | * Scalar values → [[Number Value]] → available as dinamica.inputs["v1"], dinamica.inputs["v2"], …, dinamica.inputs["v100"] |
| - | * Strings → [[Number String]] → available as ''dinamica.inputs["s1"]'', ''dinamica.inputs["s2"]'', …, ''dinamica.inputs["s100"]'' | + | * Strings → [[Number String]] → available as dinamica.inputs["s1"], dinamica.inputs["s2"], …, dinamica.inputs["s100"] |
| - | Maps cannot be connected — this functor has no cell context. There is no shorthand notation. See [[calculate_functors|Calculate Functors — Complete Operator Documentation]] for the general hook mechanism and syntax. | + | Maps cannot be connected -- this functor has no cell context. There is no shorthand notation. See [[Calculate Functors|Calculate Functors -- Complete Operator Documentation]] for the general hook mechanism and syntax. |
| - | Every table or lookup table arriving through ''dinamica.inputs'' is represented in Python as a **list of lists**: the first inner list contains the column names (the header row) and every subsequent inner list is a row of data. The header carries plain column names only — no ''*'' marking key columns, no ''#type'' annotation — so which columns were keys in the original table is information Python doesn't receive and can't recover from the input alone. Code that needs that distinction has to be told it separately, for instance by also connecting the key count as its own [[Number Value]] hook. | + | Every table or lookup table arriving through dinamica.inputs is represented in Python as a list of lists: the first inner list contains the column names (the header row) and every subsequent inner list is a row of data. The header carries plain column names only -- no asterisk marking key columns, no type annotation -- so which columns were keys in the original table is information Python doesn't receive and can't recover from the input alone. Code that needs that distinction has to be told it separately, for instance by also connecting the key count as its own [[Number Value]] hook. |
| ==== Expression outputs ==== | ==== Expression outputs ==== | ||
| - | Values are returned to Dinamica by assigning them into ''dinamica.outputs'', keyed by the desired output name. Every assigned value becomes an entry in the output ''result'' struct: | + | Values are returned to Dinamica by assigning them into dinamica.outputs, keyed by the desired output name. Every assigned value becomes an entry in the output Result struct: |
| <code python> | <code python> | ||
| - | // Scalar values are assigned directly | + | # Scalar values are assigned directly |
| dinamica.outputs["patchCount"] = 42 | dinamica.outputs["patchCount"] = 42 | ||
| dinamica.outputs["totalArea"] = 1530.5 | dinamica.outputs["totalArea"] = 1530.5 | ||
| </code> | </code> | ||
| - | Tables and lookup tables cannot be assigned directly — they must first be converted using the [[#utilities|utilities]] described below. If a table arriving from an input already carries ''*'' markers on its key column names, it can be assigned directly to an output without conversion. | + | Tables and lookup tables cannot be assigned directly -- they must first be converted using the utilities described below. If a table arriving from an input already carries asterisk markers on its key column names, it can be assigned directly to an output without conversion. |
| ==== Retrieving outputs ==== | ==== Retrieving outputs ==== | ||
| - | ''CalculatePythonExpression'' returns a single [[struct_type|Struct]] value (via its ''result'' output port) containing every entry assigned to ''dinamica.outputs''. To retrieve individual values from that struct, use the following functors from the //Integration// group: | + | Calculate Python Expression returns a single [[Struct Type]] value (via its Result output port) containing every entry assigned to dinamica.outputs. To retrieve individual values from that struct, use the following functors from the Integration group: |
| - | ^ Functor ^ Retrieves ^ | + | ^ Functor ^ Retrieves ^ |
| - | | [[Extract Struct Number]] | A numeric value (''int'' or ''float'' assigned to ''dinamica.outputs'') | | + | | [[Extract Struct Number]] | A numeric value (int or float assigned to dinamica.outputs) | |
| - | | [[Extract Struct String]] | A string value | | + | | [[Extract Struct String]] | A string value | |
| - | | [[Extract Struct Table]] | A table produced by ''dinamica.prepareTable()'' | | + | | [[Extract Struct Table]] | A table produced by dinamica.prepareTable() | |
| - | | [[Extract Struct Lookup Table]] | A lookup table produced by ''dinamica.prepareLookupTable()'' | | + | | [[Extract Struct Lookup Table]] | A lookup table produced by dinamica.prepareLookupTable() | |
| - | | [[Extract Struct Tuple]] | A tuple value | | + | | [[Extract Struct Tuple]] | A tuple value | |
| - | Each functor takes two inputs: the ''Struct'' returned by ''CalculatePythonExpression'', and the name of the entry to extract as a string constant. For example, to retrieve a numeric output named ''patchCount'' and a table output named ''filteredPatches'' — the ''$"(...)"'' raw string syntax wrapping the Python code is explained in [[#writing_the_expression_in_ego_script|Writing the expression in EGO Script]] below: | + | Each functor takes two inputs: the Struct returned by Calculate Python Expression, and the name of the entry to extract as a string constant. For example, to retrieve a numeric output named patchCount and a table output named filteredPatches: |
| <code> | <code> | ||
| Line 73: | Line 77: | ||
| === Installing packages === | === Installing packages === | ||
| - | Packages can be installed in two ways: calling ''dinamica.package(...)'' from within the expression, or listing package names on the //packages// input port. The two differ in timing: ''dinamica.package(...)'' runs **during** the expression, so it can be called conditionally; the //packages// port always runs **before** the expression executes. | + | Packages can be installed in two ways: calling dinamica.package(...) from within the expression, or listing package names on the Packages input port. The two differ in timing: dinamica.package(...) runs during the expression, so it can be called conditionally; the Packages port always runs before the expression executes. |
| **dinamica.package()** | **dinamica.package()** | ||
| - | ''dinamica.package(packageName, installPath=None, loadPath=None)'' installs (if needed via pip) and imports the requested module. Because it runs together with the script, it may fail if a package is already loaded in an incompatible version. | + | dinamica.package(packageName, installPath=None, loadPath=None) installs (if needed, via pip) and imports the requested module. Because it runs together with the script, it may fail if a package is already loaded in an incompatible version. |
| - | ^ Parameter ^ Type ^ Default ^ Description ^ | + | ^ Parameter ^ Type ^ Default ^ Description ^ |
| - | | ''packageName'' | str | — | Identifies the package. Used as the default for both ''installPath'' and ''loadPath'' when those are omitted. | | + | | packageName | str | -- | Identifies the package. Used as the default for both installPath and loadPath when those are omitted. | |
| - | | ''installPath'' | str | ''packageName'' | What is passed to ''pip install''. Can be a plain name, a version-pinned requirement, a wheel filename or URL, a ''git+'' URL, or a name followed by extra pip flags. | | + | | installPath | str | packageName | What is passed to pip install. Can be a plain name, a version-pinned requirement, a wheel filename or URL, a git+ URL, or a name followed by extra pip flags. | |
| - | | ''loadPath'' | str | ''packageName'' | The name used to ''import'' the module in Python. | | + | | loadPath | str | packageName | The name used to import the module in Python. | |
| + | |||
| + | Simply install and load numpy: | ||
| - | Simply install and load ''numpy'': | ||
| <code python> | <code python> | ||
| dinamica.package("numpy") | dinamica.package("numpy") | ||
| Line 90: | Line 95: | ||
| Specify a version: | Specify a version: | ||
| + | |||
| <code python> | <code python> | ||
| dinamica.package("numpy", "numpy==1.19.5") | dinamica.package("numpy", "numpy==1.19.5") | ||
| Line 95: | Line 101: | ||
| Install a package whose importable name differs from its pip name: | Install a package whose importable name differs from its pip name: | ||
| + | |||
| <code python> | <code python> | ||
| dinamica.package("segment_anything_py", "segment_anything_py", "segment_anything") | dinamica.package("segment_anything_py", "segment_anything_py", "segment_anything") | ||
| Line 100: | Line 107: | ||
| Use arbitrary pip parameters, such as installing from a remote wheel with a custom index: | Use arbitrary pip parameters, such as installing from a remote wheel with a custom index: | ||
| + | |||
| <code python> | <code python> | ||
| dinamica.package("segment_anything_py", "https://files.pythonhosted.org/packages/43/2f/dabe75d90a7eb54a0a609a0fc5c36d1933256319beaea5d6b2f176e213a2/segment_anything_py-1.0-py3-none-any.whl --index-url https://download.pytorch.org/whl/cu118", "segment_anything") | dinamica.package("segment_anything_py", "https://files.pythonhosted.org/packages/43/2f/dabe75d90a7eb54a0a609a0fc5c36d1933256319beaea5d6b2f176e213a2/segment_anything_py-1.0-py3-none-any.whl --index-url https://download.pytorch.org/whl/cu118", "segment_anything") | ||
| Line 105: | Line 113: | ||
| Chain several installs: | Chain several installs: | ||
| + | |||
| <code python> | <code python> | ||
| dinamica.package("cython") | dinamica.package("cython") | ||
| Line 111: | Line 120: | ||
| </code> | </code> | ||
| - | **The packages port** | + | **The Packages port** |
| + | |||
| + | The Packages input port takes one package identifier per line, in any form accepted by pip. Every listed package is installed before the expression runs. Unlike dinamica.package(), the port only installs -- it does not import anything. Importing is left entirely to the expression, using standard Python import syntax and each package's actual importable name, which may differ from the name given to pip. | ||
| - | The //packages// input port takes one package identifier per line, in any form accepted by ''pip''. Every listed package is installed before the expression runs. Unlike ''dinamica.package()'', the port only installs — it does not also import the module, and it does not support specifying a separate install name and import name. The expression must still import each package itself, using its actual importable name (which may differ from the name given to ''pip''). | + | The Packages port: |
| - | The //packages// port: | ||
| <code> | <code> | ||
| numpy | numpy | ||
| Line 123: | Line 133: | ||
| The expression: | The expression: | ||
| + | |||
| <code python> | <code python> | ||
| - | import numpy | + | import numpy as np |
| - | import requests | + | from requests import Session |
| - | import torchvision | + | from torchvision import transforms |
| </code> | </code> | ||
| === dinamica.prepareTable() === | === dinamica.prepareTable() === | ||
| - | Converts a list of lists into a table ready to be assigned to an output. The first inner list must be the header row. As with any table (see [[ego_script#constants|Constants]]), each column's type is inferred from its own values: a column of ''int''/''float'' values produces a Real column, a column of ''str'' values produces a String column. | + | Converts a list of lists into a table ready to be assigned to an output. The first inner list must be the header row. As with any table, each column's type is inferred from its own values: a column of int/float values produces a Real column, a column of str values produces a String column. |
| - | ^ Parameter ^ Type ^ Default ^ Description ^ | + | ^ Parameter ^ Type ^ Default ^ Description ^ |
| - | | ''inputTable'' | list of lists | — | The table data. First inner list is the header row; subsequent inner lists are data rows. | | + | | inputTable | list of lists | -- | The table data. First inner list is the header row; subsequent inner lists are data rows. | |
| - | | ''numKeys'' | int | — | Number of key columns. Key column names will have ''*'' appended in the output. | | + | | numKeys | int | -- | Number of key columns. Key column names will have an asterisk appended in the output. | |
| === dinamica.prepareLookupTable() === | === dinamica.prepareLookupTable() === | ||
| - | Converts a list of lists into a lookup table ready to be assigned to an output. The first inner list must be the header row. Lookup tables are always Real-typed on both key and value sides — there is no String option — so every value in ''lut'' must be numeric. | + | Converts a list of lists into a lookup table ready to be assigned to an output. The first inner list must be the header row. Lookup tables are always Real-typed on both key and value sides -- there is no String option -- so every value in lut must be numeric. |
| - | ^ Parameter ^ Type ^ Default ^ Description ^ | + | ^ Parameter ^ Type ^ Default ^ Description ^ |
| - | | ''lut'' | list of lists | — | The lookup table data. First inner list is the header row; subsequent inner lists are data rows. | | + | | lut | list of lists | -- | The lookup table data. First inner list is the header row; subsequent inner lists are data rows. | |
| === dinamica.toTable() === | === dinamica.toTable() === | ||
| - | Converts several Python data shapes into a valid Dinamica table for output. Column types are inferred the same way as ''dinamica.prepareTable()'' — except for a flat list, which always produces a Real-typed lookup table with sequential keys, matching the Real-only rule for lookup tables. | + | Converts several Python data shapes into a valid Dinamica table for output. Column types are inferred the same way as dinamica.prepareTable() -- except for a flat list, which always produces a Real-typed lookup table with sequential keys, matching the Real-only rule for lookup tables. |
| - | ^ Parameter ^ Type ^ Default ^ Description ^ | + | ^ Parameter ^ Type ^ Default ^ Description ^ |
| - | | ''inputTable'' | list of lists; dict of lists; list of tuples; flat list; ''pandas.DataFrame''; ''numpy.array'' | — | The data to convert. A flat list produces a lookup table with sequential keys. A ''numpy.array'' must have its header as the first row. | | + | | inputTable | list of lists; dict of lists; list of tuples; flat list; pandas.DataFrame; numpy.array | -- | The data to convert. A flat list produces a lookup table with sequential keys. A numpy.array must have its header as the first row. | |
| - | | ''numKeys'' | int | — | Number of key columns. Key column names will have ''*'' appended in the output. Ignored for a flat list, which always produces a lookup table with sequential keys. | | + | | numKeys | int | -- | Number of key columns. Key column names will have an asterisk appended in the output. Ignored for a flat list, which always produces a lookup table with sequential keys. | |
| ==== Examples ==== | ==== Examples ==== | ||
| - | The following examples use a consistent set of inputs: | + | The following examples use a consistent set of inputs: t1 is a table of land cover patches with columns PatchId*, Area, and CategoryId; t2 is a lookup table mapping category identifiers to category names; and v1 is a scalar minimum area threshold. |
| - | + | ||
| - | * ''t1'' — a table of land cover patches, with columns ''PatchId*'', ''Area'', and ''CategoryId'' | + | |
| - | * ''t2'' — a lookup table mapping category identifiers to category names | + | |
| - | * ''v1'' — a scalar minimum area threshold | + | |
| - | Install and import ''numpy'', then inspect all inputs passed in by Dinamica: | + | Install and import numpy, then inspect all inputs passed in by Dinamica: |
| <code python> | <code python> | ||
| Line 167: | Line 174: | ||
| </code> | </code> | ||
| - | > **Note:** The ''print()'' call is visible in Dinamica EGO's Message Log, shown as a Result-level message. Log levels are ordered Unconditional, Error, Warning, Result, Info, Info2, Debug, Debug2 — messages printed from Python are only shown when the Message Log level is set to Result or a more verbose level; at Unconditional, Error, or Warning they are suppressed. | + | The print() call is visible in Dinamica EGO's Message Log, shown as a Result-level message. Log levels are ordered Unconditional, Error, Warning, Result, Info, Info2, Debug, Debug2 -- messages printed from Python are only shown when the Message Log level is set to Result or a more verbose level; at Unconditional, Error, or Warning they are suppressed. |
| Print the rows of both connected tables to verify their contents: | Print the rows of both connected tables to verify their contents: | ||
| Line 221: | Line 228: | ||
| ==== Writing the expression in EGO Script ==== | ==== Writing the expression in EGO Script ==== | ||
| - | The ''expression'' input can be filled in directly as a text constant, using [[code_type|Code Type]]'s own raw string syntax — the same ''$"<delimiter>( raw_characters )<delimiter>"'' form used by String constants. This is also the form the Dinamica EGO GUI's dedicated code editor generates when it writes the ''expression'' port's value, so a hand-written script and one produced by the GUI take the same shape. See [[code_type|Code Type]] for the full grammar, including the base64 alternative form. | + | Expression can be filled in directly as a text constant, using [[Code Type]]'s own raw string syntax -- the same ''$"<delimiter>( raw_characters )<delimiter>"'' form used by String constants. This is also the form the Dinamica EGO GUI's dedicated code editor generates when it writes the Expression port's value, so a hand-written script and one produced by the GUI take the same shape. See [[Code Type]] for the full grammar, including the base64 alternative form. |
| The following counts the patches in a land cover areas table whose area meets a minimum threshold: | The following counts the patches in a land cover areas table whose area meets a minimum threshold: | ||
| Line 239: | Line 246: | ||
| </code> | </code> | ||
| - | ''CalculatePythonExpression'' returns a ''Struct'' containing all values assigned to ''dinamica.outputs''. The ''ExtractStructNumber'' functor then pulls the ''patchCount'' entry out of that struct by name. See [[#retrieving_outputs|Retrieving outputs]] for the full list of extraction functors. | + | Here landCoverAreas is bound to t1 and minimumArea to v1, each through a [[Number Table]] or [[Number Value]] hook. Calculate Python Expression returns a Struct containing all values assigned to dinamica.outputs; the ExtractStructNumber functor then pulls the patchCount entry out of that struct by name. See "Retrieving outputs" above for the full list of extraction functors. |
| - | A more involved example: installing ''numpy'' to compute area statistics for a land cover patches table and flag outlier patches, then retrieving both the scalar statistics and the resulting table: | + | A more involved example: installing numpy to compute area statistics for a land cover patches table and flag outlier patches, then retrieving both the scalar statistics and the resulting table: |
| <code> | <code> | ||
| Line 270: | Line 277: | ||
| </code> | </code> | ||
| - | Here ''landCoverPatches'' is bound to ''t1'' through a single [[Number Table]] hook. | + | Here landCoverPatches is bound to t1 through a single [[Number Table]] hook. |
| - | + | ||
| - | ===== Group ===== | + | |
| - | + | ||
| - | [[Functor List#Integration|Integration]] | + | |
| ===== Internal Name ===== | ===== Internal Name ===== | ||
| CalculatePythonExpression | CalculatePythonExpression | ||