Differences

This shows you the differences between two versions of the page.

--- manipulating_tables_and_lookup_tables [2026/08/07 03:02]
hermann
+++ manipulating_tables_and_lookup_tables [2026/08/18 03:09] (current)
hermann [Printing a table with any number of key columns]
@@ Line 497: / Line 497: @@
 One difference in shape: the loop accumulates into a Table, since ''emptyResults'' was declared as one, while ''%'' always returns a Lookup Table. Where the Table form is the one wanted, a ''Table'' carrier converts it — ''asTable := Table doubledResults'' — one Real key and one Real value fitting both types equally well.
+===== Table Manager =====
+The accumulator pattern in [[#storing_results_across_a_loop|Storing results across a loop]] uses a Mux to carry a growing table from one iteration to the next -- which, as that section's closing note explains, forces the loop to run sequentially, one iteration at a time. **Table Manager** solves the same class of problem -- building a table with an extra key column from results computed across a loop -- without that cost: each iteration registers its own sub-table independently, with no Mux and nothing threaded between iterations, so the loop producing those partial results stays eligible to run in parallel, per [[basic_data_flow|Basic Data Flow]]. A single [[#mergesubtables|MergeSubTables]] call combines everything afterward.
+A container and two functors do the core work; a fourth covers a narrower case:
+==== TableManager ====
+A container functor. It creates the manager instance and exposes it to whatever runs inside it.
+^ Port ^ Direction ^ Type ^ Description ^
+| ''tableManager'' | Internal Output | TableManager | The manager instance. Auto-binds to the ''tableManager'' input of [[#subtable|SubTable]] and [[#mergesubtables|MergeSubTables]] calls nested anywhere inside this container, including inside further nested containers such as a [[For]] loop or a [[Group]]. |
+The container itself takes no inputs and defines no regular outputs.
+==== SubTable ====
+Registers one sub-table under a name and a key.
+^ Port ^ Direction ^ Type ^ Required? ^ Description ^
+| ''tableName'' | Input | Name | Yes | Identifies which set of sub-tables this one joins. |
+| ''keys'' | Input | Tuple | Yes | The key identifying this sub-table within the set. |
+| ''subTable'' | Input | Table | Yes | The sub-table itself. |
+| ''emptySubTableIsAllowed'' | Input | Boolean | No | Whether an empty sub-table is accepted rather than raising an error. Default ''.yes''. |
+| ''tableManager'' | Input | TableManager | Yes | Auto-binds to the enclosing [[#tablemanager|TableManager]] container; normally left unconnected. |
+| ''object'' | Output | Table | -- | The same sub-table passed in, returned unchanged so it can still be referenced or connected onward. |
+Every ''SubTable'' call sharing a ''tableName'' is checked against the first one registered under that name: the ''keys'' tuple must have the same element types, and ''subTable'' must have the same columns. A mismatch, or registering the same ''keys'' twice under the same ''tableName'', raises an error.
+==== MergeSubTables ====
+Combines every sub-table registered under a name into a base table, then discards them from the manager.
+^ Port ^ Direction ^ Type ^ Required? ^ Description ^
+| ''baseTable'' | Input | Table | Yes | The table the combined sub-tables are inserted into. |
+| ''tableName'' | Input | Name | Yes | Which set of sub-tables to merge. |
+| ''allowNamesWithNoAssociatedSubTables'' | Input | Boolean | No | If ''.no'', merging a name nothing was ever registered under raises an error; if ''.yes'', it's silently skipped. Default ''.no''. |
+| ''tableManager'' | Input | TableManager | Yes | Auto-binds to the enclosing [[#tablemanager|TableManager]] container. |
+| ''table'' | Output | Table | -- | ''baseTable'' with every sub-table registered under ''tableName'' inserted, each keyed by the Tuple it was registered with. |
+Each sub-table's ''keys'' tuple can hold more than one element; every element becomes its own leading key column on ''baseTable''. A ''tableName'' can only be merged once -- the merge removes its sub-tables from the manager as it runs.
+> **Warning:** ''MergeSubTables'' has no data dependency on the ''SubTable'' calls that feed it -- its ''baseTable'' input, its literal ''tableName'', and its auto-bound manager are all available from the moment the container starts, and none of them depend on the loop that registered the sub-tables. Placing it after a loop in the script, even inside the same ''TableManager'' container, does **not** guarantee it runs after that loop finishes; the two have no connection, so the engine is free to run them in either order or at the same time. Force the ordering explicitly with a [[Group]] gated by the loop's ''sequenceOutput'', as in the example below -- see [[basic_data_flow#ordering_without_data_sequence_connections|Ordering without data: sequence connections]] and [[ego_script#sequence_ports|Sequence ports]].
+==== TableManagerValue ====
+An explicit pass-through for a TableManager instance, for the rare case where auto-bind doesn't reach -- for instance, threading a manager through a submodel boundary as a named port rather than relying on ambient container nesting. Within a single model, auto-bind on ''SubTable'' and ''MergeSubTables'' makes this unnecessary.
+^ Port ^ Direction ^ Type ^ Required? ^ Description ^
+| ''tableManager'' | Input | TableManager | No | Auto-binds to the enclosing [[#tablemanager|TableManager]] container. |
+| ''tableManager'' | Output | TableManager | -- | The same instance, under an explicit name. |
+==== Example ====
+Building a two-key table from per-year sub-tables computed inside a loop, with the merge correctly sequenced after the loop:
+<code>
+emptyPriceTable := Table [ "Year*#real", "City*#string", "Price#real" ];
+TableManager {{
+    // forDone carries no data of its own; it exists purely so the Group
+    // below can force itself to wait for every iteration of this loop.
+    forDone := For 2004 2007 {{
+        year := Step;
+        // Stand-in for a per-year computation; in practice this might come
+        // from a database query, a Calculate Table Values call, or another
+        // model entirely. A Table literal only accepts literal cell values,
+        // so a real per-year computation would build this row by row with
+        // AddTupleValue and AddTableRow instead, the same as any table
+        // whose contents aren't known until the model runs.
+        yearPrices := Table [
+            "City*", "Price",
+            "Boston", 4000,
+            "Chelsea", 200
+        ];
+        _ := SubTable "prices" year yearPrices;
+    }};
+    // Group's sequenceInput, fed from the loop's sequenceOutput, forces
+    // this Group -- and MergeSubTables inside it -- to wait for the whole
+    // loop to finish, without threading a Mux through its iterations.
+    _ := Group forDone {{
+        priceTable := MergeSubTables emptyPriceTable "prices";
+    }};
+}};
+</code>
+''priceTable'' is usable here, once the Group it was produced inside — and through its ''sequenceInput'', the loop that Group was forced to wait for — has finished.
+The loop's own iterations are still eligible to run in parallel with each other: nothing inside it is a Mux, nothing inside it is consumed from outside it, and it feeds no submodel output port -- see [[basic_data_flow|Basic Data Flow]]. Only the loop as a whole is sequenced against the merge, which is why this avoids the cost the Mux-accumulator pattern above pays on every single iteration.
+> **Note:** ''year'' is passed to ''SubTable'' directly, not wrapped in brackets as ''[year]''. Bracket syntax is for Tuple **literals**; a connected Real value converts to a one-element Tuple automatically on connection, the same pattern used by ''GetTableFromKey'' in [[#sub-tables|Sub-Tables]] above.
 ===== Printing a table with a generic format =====
@@ Line 949: / Line 1045: @@
 ];
-result := CalculatePythonExpression (String $"(
+result := CalculatePythonExpression $"(
 inputTable = dinamica.inputs['t1']
 header = inputTable[0]
@@ Line 962: / Line 1058: @@
 dinamica.outputs['reshapedTable'] = dinamica.prepareTable(newTable, 1)
-)") {{
+)" {{
     NumberTable myTable 1;
 }};