Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
working_with_subregions [2026/08/02 02:14]
hermann 		working_with_subregions [2026/08/02 21:25] (current)
hermann
Line 3: Line 3:
 ===== Overview ===== ===== Overview =====
  
-Some workflows shouldn'​t run against a whole map at once — a national land-use model might need different parameters per country, or a hydrology model might only make sense computed watershed by watershed. ​Region Manager ​is what makes that practical: define the regions once, from a categorical map whose class values are the boundaries, and everything downstream — reading metadata, narrowing to one region, splitting a map into pieces, and merging the pieces back — works against that single definition.+Region Manager ​exists for three distinct reasons, and it's worth being clear about which one applies before reaching for it:
  
-The running ​example through this page computes an accumulated-cost surface ​separately ​for each region, ​using [[Calc Cost Map]], then reassembles ​the per-region ​results into one global cost map. It's a genuinely representative case: split, do real per-region ​work, collect, merge.+  * **The computation itself requires region-specific inputs.** [[Patcher]] and [[Expander]] each take exactly one [[Transition Matrix Type|Transition Matrix]] and one [[Transition Function Parameter Matrix Type|Transition Function Parameter Matrix]] per call. If those rates or patch parameters genuinely differ by region, there'​s no single call that can express that — running ​once per region, with region-specific parameters, is the only way to get that behavior. 
 +  * **The map is too large to process as one raster.** Splitting into regions keeps each piece a manageable size to work with, independent of whether the calculation itself cares about regions at all. 
 +  * **A windowed or accumulating calculation would be distorted at a hard-clipped edge.** Anything that looks beyond the current cell — a cost surface, a distance calculation,​ a neighborhood statistic — needs *real* context past a region'​s boundary, not just reserved space for it. ''​borderCells''​ reserves that extra space around ​each region, ​but by default it's filled with null, not real data — getting real values into it also requires ''​keepNonRegionCells:​ Yes''​ on the [[Regionalize ​Map]] / [[Regionalize Categorical Map]] call that produces ​the regional map. 
 + 
 +The first case is structural: some functors simply can't vary their parameters by region ​any other way. The second and third are about how a computation is carried out, not what it computes — the same result could, in principle, come from one call over the whole map, if size or edge effects weren't a concern. 
 + 
 +Whichever reason applies, the mechanics are the same: define the regions once, from a categorical map whose class values are the boundaries, and everything downstream — reading metadata, narrowing to one region, ​splitting a map into pieces, and merging the pieces back — works against that single definition. The running example through this page is the first case: running [[Patcher]] with a different change matrix and transition function parameter matrix per region.
  
 ===== Defining regions ===== ===== Defining regions =====
Line 16: Line 22:
 | ''​regionManager''​ | Internal Output | Region Manager | — | The region manager, exposed to functors nested inside the container. | | ''​regionManager''​ | Internal Output | Region Manager | — | The region manager, exposed to functors nested inside the container. |
  
-A nonzero ''​borderCells''​ matters ​for anything that accumulates across neighboring ​cells — a cost surface, a neighborhood statistic — since without it, a region'​s edge cells would be computed as if the world stopped there.+A nonzero ''​borderCells''​ matters ​here too: [[Patcher]]'​s own neighborhood search window (3×3 cells by default) means a region'​s edge cells still need real context beyond ​the hard-clipped boundary. ''​borderCells''​ alone only reserves that space, though — making it real data instead of null also requires ''​keepNonRegionCells:​ Yes''​ when the region is split out, covered in [[#​splitting_a_global_map_into_regions|Splitting a global map into regions]] below.
  
 **Example** — the shell everything else in this page builds on: **Example** — the shell everything else in this page builds on:
  
 <​code>​ <​code>​
-sourcesMap ​:= LoadMap ​"​c:/​data/​sources.tif";​ +landscapeMap ​:= LoadCategoricalMap ​"​c:/​data/​landscape.tif" ​.none .default 0; 
-frictionsMap ​:= LoadMap "​c:/​data/​frictions.tif";+probabilitiesMap ​:= LoadMap "​c:/​data/​probabilities.tif";
 regionsMap := LoadCategoricalMap "​c:/​data/​regions.tif"​ .none .default 0; regionsMap := LoadCategoricalMap "​c:/​data/​regions.tif"​ .none .default 0;
  
Line 49: Line 55:
 | ''​regionTopYCoordinate'',​ ''​regionLeftXCoordinate'',​ ''​regionBottomYCoordinate'',​ ''​regionRightXCoordinate''​ | Output | Real Value | — | The region'​s four coordinate boundaries. | | ''​regionTopYCoordinate'',​ ''​regionLeftXCoordinate'',​ ''​regionBottomYCoordinate'',​ ''​regionRightXCoordinate''​ | Output | Real Value | — | The region'​s four coordinate boundaries. |
  
-Both are for reading region metadata itself — bounding boxes, a region'​s mask, its coordinate system. Neither is the cheapest way to just *visit* every region, which the next section covers separately.+Both are for reading region metadata itself — bounding boxes, a region'​s mask, its coordinate system: 
 + 
 +<​code>​ 
 +RegionManager regionsMap 5 {{ 
 +    allRegions := GetAllRegionsInfo;​ 
 +}}; 
 +</​code>​
  
 ===== Visiting one region, or every region ===== ===== Visiting one region, or every region =====
Line 63: Line 75:
 A region'​s ID is one of the regions map's own category values, so the cheapest way to visit every region is [[For Each Category]] against that categorization directly — no table needs to be built and then queried for its keys first, the way looping over [[Get All Regions Info]]'​s output would require. [[For Each Category]] takes a Categorization,​ but since [[Categorical Map Type]] converts to [[Categorization Type]] automatically on any connection, the regions map itself can be passed straight in — there'​s no need for an explicit extraction step like [[Get Map Categories]] first. A region'​s ID is one of the regions map's own category values, so the cheapest way to visit every region is [[For Each Category]] against that categorization directly — no table needs to be built and then queried for its keys first, the way looping over [[Get All Regions Info]]'​s output would require. [[For Each Category]] takes a Categorization,​ but since [[Categorical Map Type]] converts to [[Categorization Type]] automatically on any connection, the regions map itself can be passed straight in — there'​s no need for an explicit extraction step like [[Get Map Categories]] first.
  
-As with [[For Each]], [[For Each Category]]'​s current-step output isn't named ''​regionId'',​ so [[Region]] is still nested inside to re-expose it under the right name — [[Regionalize Map]] and the rest of this page's functors auto-bind to a port by that exact name.+As with [[For Each]], [[For Each Category]]'​s current-step output isn't named ''​regionId'',​ so [[Region]] is still nested inside to re-expose it under the right name — [[Regionalize Map]] and the rest of this page's functors auto-bind to a port by that exact name. ''​step''​ itself is an internal output, not an ordinary variable, so it can't be passed as a bare argument directly — it has to be aliased to a variable with ''​=''​ first, the same way [[Region Manager]]'​s own confirmed example aliases ''​regionManager''​ to ''​manager''​ before using it.
  
-It's tempting to reach for [[For Each Region]] ​instead, since it bundles "​define regions"​ and "loop over them" into one container. Resist that here: it has its own required ''​regions''​ input and creates its own region manager internally, rather than sharing one from an enclosing [[Region Manager]]. That'​s ​fine if the loop is the entire task, but this page's example needs the *same* region manager instance still available after the loop, for the merge — which [[For Each Region]] can't provide, since its manager stops existing once its own block ends.+[[For Each Region]] bundles "​define regions"​ and "loop over them" into one container, but it has its own required ''​regions''​ input and creates its own region manager internally, rather than sharing one from an enclosing [[Region Manager]]. That works fine when the loop is the entire task. It doesn'​t work here, because ​this page's example needs the *same* region manager instance still available after the loop, for the merge — and [[For Each Region]] can't provide ​that, since its manager stops existing once its own block ends.
  
 <​code>​ <​code>​
 RegionManager regionsMap 5 {{ RegionManager regionsMap 5 {{
     _ := ForEachCategory regionsMap {{     _ := ForEachCategory regionsMap {{
-        Region ​step {{ +        ​regionId = step; 
-            // regionId comes from step; regionManager auto-binds from the + 
-            // enclosing RegionManager.+        ​Region ​regionId ​{{ 
 +            // regionManager auto-binds from the enclosing RegionManager.
         }};         }};
     }};     }};
Line 90: Line 103:
 | ''​regionalMap''​ | Output | Map / Categorical Map | — | The map cut down to this region. | | ''​regionalMap''​ | Output | Map / Categorical Map | — | The map cut down to this region. |
  
-This is where the actual per-region computation happens — split the global inputs, run whatever the model needs against the region-sized ​pieces, in this case [[Calc Cost Map]]:+[[Patcher]] needs both the landscape and the spatial probability map at matching, region-sized ​extents. Its neighborhood search window means the border cells reserved by ''​borderCells''​ need to hold real data, not null — so ''​keepNonRegionCells''​ is set explicitly to ''​Yes''​ on both calls, rather than left at its default:
  
 <​code>​ <​code>​
-Region ​step {{ +Region ​regionId ​{{ 
-    ​regionalSources ​:= RegionalizeMap sourcesMap; +    ​regionalLandscape ​:= RegionalizeCategoricalMap { globalMap = landscapeMap,​ keepNonRegionCells = .yes }; 
-    ​regionalFrictions ​:= RegionalizeMap ​frictionsMap;​ +    ​regionalProbabilities ​:= RegionalizeMap { globalMap ​= probabilitiesMap, keepNonRegionCells ​= .yes };
- +
-    ​{ costs = regionalCost, directions ​= _ } := +
-        CalcCostMap regionalSources regionalFrictions .no;+
 }}; }};
 </​code>​ </​code>​
 +
 +===== Selecting each region'​s parameters =====
 +
 +[[Patcher]]'​s ''​changes''​ and ''​transitionParameters''​ inputs are exactly what makes this a case where regions are structurally necessary — each is a single value per call, with no way to vary it by region except by calling [[Patcher]] once per region. Storing each region'​s parameters as rows in a table, keyed by ''​RegionId'',​ and slicing out the current region'​s rows with [[Get Table From Key]] keeps that variation in one place instead of duplicating the whole model per region.
 +
 +^ Port ^ Direction ^ Type ^ Required? ^ Description ^
 +| ''​table''​ | Input | Table | Yes | The table to read from. |
 +| ''​keys''​ | Input | Tuple | Yes | The leading key(s) identifying which sub-table to retrieve. |
 +| ''​result''​ | Output | Table | — | The matching sub-table, with those keys stripped off. |
 +
 +<​code>​
 +changesByRegion := LoadTable "​c:/​data/​changes_by_region.csv";​
 +paramsByRegion := LoadTable "​c:/​data/​params_by_region.csv";​
 +
 +regionChanges := GetTableFromKey changesByRegion regionId;
 +regionParams := GetTableFromKey paramsByRegion regionId;
 +</​code>​
 +
 +''​changesByRegion''​ is shaped ''​RegionId*'',​ ''​From*'',​ ''​To*'',​ ''​Cells'';​ ''​paramsByRegion''​ is shaped ''​RegionId*'',​ ''​From*'',​ ''​To*'',​ ''​Mean_Patch_Size'',​ ''​Patch_Size_Variance'',​ ''​Patch_Isometry''​. Stripping ''​RegionId''​ leaves exactly the column layout [[Change Matrix Type]] and [[Transition Function Parameter Matrix Type]] expect, so both convert automatically once connected to [[Patcher]] — no explicit conversion step needed.
  
 ===== Collecting regional results ===== ===== Collecting regional results =====
Line 108: Line 137:
 ^ Port ^ Direction ^ Type ^ Required? ^ Description ^ ^ Port ^ Direction ^ Type ^ Required? ^ Description ^
 | ''​globalMapName''​ | Input | Name | Yes | The shared name this regional fragment will be collected under. | | ''​globalMapName''​ | Input | Name | Yes | The shared name this regional fragment will be collected under. |
-| ''​regionalMap''​ | Input | Map ([[Regional Map]]) / Categorical Map ([[Regional Categorical Map]]) | Yes | The per-region ​result to store. |+| ''​regionalMap''​ | Input | Map ([[Regional Map]]) / Categorical Map ([[Regional Categorical Map]]) | Yes | The result to store — any map, at any extent, not necessarily already clipped to the region. It's clipped down to the corresponding region automatically. |
 | ''​regionId''​ | Input | Integer Value | No, auto-bound | Which region this fragment belongs to. | | ''​regionId''​ | Input | Integer Value | No, auto-bound | Which region this fragment belongs to. |
 | ''​regionManager''​ | Input | Region Manager | No, auto-bound | The region manager this fragment belongs to. | | ''​regionManager''​ | Input | Region Manager | No, auto-bound | The region manager this fragment belongs to. |
 +
 +That last point is worth being explicit about: the map passed in doesn'​t have to already be region-sized — even the full global map would work, since [[Regional Map]]/​[[Regional Categorical Map]] clip it to the region internally rather than assuming it's already been through [[Regionalize Map]]/​[[Regionalize Categorical Map]]. The worked example below still runs its map through [[Regionalize Map]]/​[[Regionalize Categorical Map]] first, but for a different reason: [[Patcher]] itself needs region-sized inputs to compute against, not just to store the result — the clipping done here is redundant on an already region-sized map, not required by [[Regional Map]] itself.
  
 <​code>​ <​code>​
-Region ​step {{ +regionChanges := GetTableFromKey changesByRegion regionId; 
-    ​regionalSources ​:= RegionalizeMap sourcesMap; +regionParams := GetTableFromKey paramsByRegion regionId; 
-    ​regionalFrictions ​:= RegionalizeMap ​frictionsMap;+ 
 +Region ​regionId ​{{ 
 +    ​regionalLandscape ​:= RegionalizeCategoricalMap { globalMap = landscapeMap,​ keepNonRegionCells = .yes }; 
 +    ​regionalProbabilities ​:= RegionalizeMap ​{ globalMap = probabilitiesMap,​ keepNonRegionCells = .yes };
  
-    { costs = regionalCost, directions ​= _ } := +    { changedLandscape ​= regionalChanged, corrodedProbabilities = _, remainingChanges ​= _ } := 
-        ​CalcCostMap regionalSources regionalFrictions .no;+        ​Patcher regionalLandscape regionalProbabilities regionChanges regionParams;
  
-    RegionalMap "​cost_map" ​regionalCost;+    RegionalMap "​cost_map" ​regionalChanged;
 }}; }};
 </​code>​ </​code>​
 +
 +[[Patcher]] returns three outputs, and this example only stores one of them. ''​corrodedProbabilities''​ — the probability surface with used-up areas depleted — and ''​remainingChanges''​ — a Change Matrix of whatever couldn'​t be placed, given the current ''​neighborWindowLines''/''​neighborWindowColumns''/''​pruneFactor''​ — are both genuinely useful diagnostics in a real model, not values to ignore on principle. They'​re discarded here only because this page is illustrating region-manager mechanics rather than full Patcher diagnostics. All three outputs are still named explicitly, with the unwanted two bound to ''​_'',​ rather than left out of the destructuring entirely — whether EGO Script allows binding only some of a multi-output functor'​s outputs isn't confirmed, so naming all of them is the safer pattern used consistently throughout this page.
 +
 +A model that does care about ''​remainingChanges''​ would most likely want it accumulated across regions too, the same way [[#​computing_per-class_areas_within_each_region|the area-by-region example]] accumulates its table with [[Mux Table]] — checking, once the loop finishes, whether any region failed to place all of its requested changes.
  
 ===== Merging regions back into one map ===== ===== Merging regions back into one map =====
Line 138: Line 176:
 A few more things worth knowing before relying on the merge: A few more things worth knowing before relying on the merge:
  
 +  * **Splitting keeps border cells so Patcher'​s neighborhood window has real context; merging discards them anyway.** Each region'​s own border zone duplicates territory another region already owns and computes for itself. Merging with ''​mergeNonRegionCells:​ Yes''​ would pull that duplicated, independently-computed data back in instead of leaving each region'​s own interior result as the authority for its own cells — which is why the worked example uses ''​keepNonRegionCells:​ Yes''​ when splitting, but leaves ''​mergeNonRegionCells''​ at its default ''​No''​.
   * **The merged map's format is inherited from the first fragment stored, not negotiated across all of them.** Cell type, null value, and categorization all come from whichever [[Regional Map]] or [[Regional Categorical Map]] call registered first for that name — only the dimensions come from the regions map itself. If regions are processed in an order that isn't guaranteed, don't rely on which fragment "​wins."​   * **The merged map's format is inherited from the first fragment stored, not negotiated across all of them.** Cell type, null value, and categorization all come from whichever [[Regional Map]] or [[Regional Categorical Map]] call registered first for that name — only the dimensions come from the regions map itself. If regions are processed in an order that isn't guaranteed, don't rely on which fragment "​wins."​
   * **Mismatched null values fail the merge, not silently overwrite each other.** When ''​mergeNonRegionCells''​ brings overlapping border cells from two different regions together, they can only combine if they agree, or one of them is null — two regions disagreeing on what a cell's value should be raises an error rather than picking one.   * **Mismatched null values fail the merge, not silently overwrite each other.** When ''​mergeNonRegionCells''​ brings overlapping border cells from two different regions together, they can only combine if they agree, or one of them is null — two regions disagreeing on what a cell's value should be raises an error rather than picking one.
Line 150: Line 189:
  
 <​code>​ <​code>​
-sourcesMap ​:= LoadMap ​"​c:/​data/​sources.tif";​ +landscapeMap ​:= LoadCategoricalMap ​"​c:/​data/​landscape.tif" ​.none .default 0; 
-frictionsMap ​:= LoadMap "​c:/​data/​frictions.tif";+probabilitiesMap ​:= LoadMap "​c:/​data/​probabilities.tif";
 regionsMap := LoadCategoricalMap "​c:/​data/​regions.tif"​ .none .default 0; regionsMap := LoadCategoricalMap "​c:/​data/​regions.tif"​ .none .default 0;
 +
 +changesByRegion := LoadTable "​c:/​data/​changes_by_region.csv";​
 +paramsByRegion := LoadTable "​c:/​data/​params_by_region.csv";​
  
 RegionManager regionsMap 5 {{ RegionManager regionsMap 5 {{
     sequenceOut := ForEachCategory regionsMap {{     sequenceOut := ForEachCategory regionsMap {{
-        ​Region ​step {{ +        ​regionId = step; 
-            ​regionalSources ​:= RegionalizeMap sourcesMap; + 
-            ​regionalFrictions ​:= RegionalizeMap ​frictionsMap;+        regionChanges := GetTableFromKey changesByRegion regionId; 
 +        regionParams := GetTableFromKey paramsByRegion regionId; 
 + 
 +        Region regionId ​{{ 
 +            ​regionalLandscape ​:= RegionalizeCategoricalMap { globalMap = landscapeMap,​ keepNonRegionCells = .yes }; 
 +            ​regionalProbabilities ​:= RegionalizeMap ​{ globalMap = probabilitiesMap,​ keepNonRegionCells = .yes };
  
-            { costs = regionalCost, directions ​= _ } := +            { changedLandscape ​= regionalChanged, corrodedProbabilities = _, remainingChanges ​= _ } := 
-                ​CalcCostMap regionalSources regionalFrictions .no;+                ​Patcher regionalLandscape regionalProbabilities regionChanges regionParams;
  
-            ​RegionalMap ​"cost_map" ​regionalCost;+            ​RegionalCategoricalMap ​"changed_landscape" ​regionalChanged;
         }};         }};
     }};     }};
  
     _ := Group sequenceOut {{     _ := Group sequenceOut {{
-        ​costMap ​:= MergeRegionalMaps ​"cost_map" .no; +        ​finalLandscape ​:= MergeRegionalCategoricalMaps ​"changed_landscape" .no; 
-        ​SaveMap costMap ​"​c:/​data/​cost_map_merged.tif";+        ​SaveCategoricalMap finalLandscape ​"​c:/​data/​changed_landscape_merged.tif";
     }};     }};
 }}; }};
 </​code>​ </​code>​
  
-One region manager, defined once at the top, is what threads the whole thing together: [[For Each Category]] loops directly against the categories already attached to ''​regionsMap'',​ [[Region]] narrows against the manager once per loop iteration, [[Regional Map]] stores fragments into it, and [[Merge Regional Maps]] — deliberately delayed until the loop's ''​sequenceOutput''​ fires — reads every one of those fragments back out of the same instance.+One region manager, defined once at the top, is what threads the whole thing together: [[For Each Category]] loops directly against the categories already attached to ''​regionsMap''​, [[Get Table From Key]] slices each region'​s own change matrix and transition parameters out of the two shared tables, [[Region]] narrows against the manager once per loop iteration, [[Regional ​Categorical ​Map]] stores fragments into it, and [[Merge Regional ​Categorical ​Maps]] — deliberately delayed until the loop's ''​sequenceOutput''​ fires — reads every one of those fragments back out of the same instance.
  
 ===== Computing per-class areas within each region ===== ===== Computing per-class areas within each region =====
Line 191: Line 238:
 | ''​feedback''​ | Input | Table | Yes | The value produced by the current iteration, becoming ''​table''​ for the next one. | | ''​feedback''​ | Input | Table | Yes | The value produced by the current iteration, becoming ''​table''​ for the next one. |
 | ''​table''​ | Output | Table | — | The accumulated value entering the current iteration. | | ''​table''​ | Output | Table | — | The accumulated value entering the current iteration. |
 +
 +This section doesn'​t use [[Region]] at all — ''​regionId''​ is used directly, since [[Regionalize Categorical Map]] and [[Set Table By Key]] both auto-bind or accept it positionally without needing it re-exposed under a container of its own. There'​s also no windowed calculation here, so unlike the [[Patcher]] example, ''​keepNonRegionCells''​ is left at its default — [[Calc Areas]] only tallies cells within the region itself:
  
 <​code>​ <​code>​