Differences

This shows you the differences between two versions of the page.

--- working_with_subregions [2026/08/02 02:44]
hermann
+++ working_with_subregions [2026/08/02 21:25] (current)
hermann
@@ Line 3: / Line 3: @@
 ===== Overview =====
-Some workflows shouldn't run against a whole map at once — a national land-use model might need different parameters per country, or a hydrology model might only make sense computed watershed by watershed. Region Manager is what makes that practical: define the regions once, from a categorical map whose class values are the boundaries, and everything downstream — reading metadata, narrowing to one region, splitting a map into pieces, and merging the pieces back — works against that single definition.
+Region Manager exists for three distinct reasons, and it's worth being clear about which one applies before reaching for it:
-The running example through this page computes an accumulated-cost surface separately for each region, using [[Calc Cost Map]], then reassembles the per-region results into one global cost map.
+  * **The computation itself requires region-specific inputs.** [[Patcher]] and [[Expander]] each take exactly one [[Transition Matrix Type|Transition Matrix]] and one [[Transition Function Parameter Matrix Type|Transition Function Parameter Matrix]] per call. If those rates or patch parameters genuinely differ by region, there's no single call that can express that — running once per region, with region-specific parameters, is the only way to get that behavior.
+  * **The map is too large to process as one raster.** Splitting into regions keeps each piece a manageable size to work with, independent of whether the calculation itself cares about regions at all.
+  * **A windowed or accumulating calculation would be distorted at a hard-clipped edge.** Anything that looks beyond the current cell — a cost surface, a distance calculation, a neighborhood statistic — needs *real* context past a region's boundary, not just reserved space for it. ''borderCells'' reserves that extra space around each region, but by default it's filled with null, not real data — getting real values into it also requires ''keepNonRegionCells: Yes'' on the [[Regionalize Map]] / [[Regionalize Categorical Map]] call that produces the regional map.
+The first case is structural: some functors simply can't vary their parameters by region any other way. The second and third are about how a computation is carried out, not what it computes — the same result could, in principle, come from one call over the whole map, if size or edge effects weren't a concern.
+Whichever reason applies, the mechanics are the same: define the regions once, from a categorical map whose class values are the boundaries, and everything downstream — reading metadata, narrowing to one region, splitting a map into pieces, and merging the pieces back — works against that single definition. The running example through this page is the first case: running [[Patcher]] with a different change matrix and transition function parameter matrix per region.
 ===== Defining regions =====
@@ Line 16: / Line 22: @@
 | ''regionManager'' | Internal Output | Region Manager | — | The region manager, exposed to functors nested inside the container. |
-A nonzero ''borderCells'' matters for anything that accumulates across neighboring cells — a cost surface, a neighborhood statistic — since without it, a region's edge cells would be computed as if the world stopped there.
+A nonzero ''borderCells'' matters here too: [[Patcher]]'s own neighborhood search window (3×3 cells by default) means a region's edge cells still need real context beyond the hard-clipped boundary. ''borderCells'' alone only reserves that space, though — making it real data instead of null also requires ''keepNonRegionCells: Yes'' when the region is split out, covered in [[#splitting_a_global_map_into_regions|Splitting a global map into regions]] below.
 **Example** — the shell everything else in this page builds on:
 <code>
-sourcesMap := LoadMap "c:/data/sources.tif";
+landscapeMap := LoadCategoricalMap "c:/data/landscape.tif" .none .default 0;
-frictionsMap := LoadMap "c:/data/frictions.tif";
+probabilitiesMap := LoadMap "c:/data/probabilities.tif";
 regionsMap := LoadCategoricalMap "c:/data/regions.tif" .none .default 0;
@@ Line 49: / Line 55: @@
 | ''regionTopYCoordinate'', ''regionLeftXCoordinate'', ''regionBottomYCoordinate'', ''regionRightXCoordinate'' | Output | Real Value | — | The region's four coordinate boundaries. |
-Both are for reading region metadata itself — bounding boxes, a region's mask, its coordinate system — and neither one feeds into the running example that continues through the rest of this page. As a standalone check, either can simply be dropped inside the same shell:
+Both are for reading region metadata itself — bounding boxes, a region's mask, its coordinate system:
 <code>
@@ Line 56: / Line 62: @@
 }};
 </code>
-The cheapest way to actually *visit* every region — which the running example does need — is covered separately, next.
 ===== Visiting one region, or every region =====
@@ Line 99: / Line 103: @@
 | ''regionalMap'' | Output | Map / Categorical Map | — | The map cut down to this region. |
-This is where the actual per-region computation happens — split the global inputs, run whatever the model needs against the region-sized pieces, in this case [[Calc Cost Map]]. ''regionId'' and ''regionManager'' both auto-bind here from the enclosing [[Region]], so neither needs to be passed explicitly:
+[[Patcher]] needs both the landscape and the spatial probability map at matching, region-sized extents. Its neighborhood search window means the border cells reserved by ''borderCells'' need to hold real data, not null — so ''keepNonRegionCells'' is set explicitly to ''Yes'' on both calls, rather than left at its default:
 <code>
 Region regionId {{
-    regionalSources := RegionalizeMap sourcesMap;
+    regionalLandscape := RegionalizeCategoricalMap { globalMap = landscapeMap, keepNonRegionCells = .yes };
-    regionalFrictions := RegionalizeMap frictionsMap;
+    regionalProbabilities := RegionalizeMap { globalMap = probabilitiesMap, keepNonRegionCells = .yes };
-    { costs = regionalCost, directions = _ } :=
-        CalcCostMap regionalSources regionalFrictions .no;
 }};
 </code>
+===== Selecting each region's parameters =====
+[[Patcher]]'s ''changes'' and ''transitionParameters'' inputs are exactly what makes this a case where regions are structurally necessary — each is a single value per call, with no way to vary it by region except by calling [[Patcher]] once per region. Storing each region's parameters as rows in a table, keyed by ''RegionId'', and slicing out the current region's rows with [[Get Table From Key]] keeps that variation in one place instead of duplicating the whole model per region.
+^ Port ^ Direction ^ Type ^ Required? ^ Description ^
+| ''table'' | Input | Table | Yes | The table to read from. |
+| ''keys'' | Input | Tuple | Yes | The leading key(s) identifying which sub-table to retrieve. |
+| ''result'' | Output | Table | — | The matching sub-table, with those keys stripped off. |
+<code>
+changesByRegion := LoadTable "c:/data/changes_by_region.csv";
+paramsByRegion := LoadTable "c:/data/params_by_region.csv";
+regionChanges := GetTableFromKey changesByRegion regionId;
+regionParams := GetTableFromKey paramsByRegion regionId;
+</code>
+''changesByRegion'' is shaped ''RegionId*'', ''From*'', ''To*'', ''Cells''; ''paramsByRegion'' is shaped ''RegionId*'', ''From*'', ''To*'', ''Mean_Patch_Size'', ''Patch_Size_Variance'', ''Patch_Isometry''. Stripping ''RegionId'' leaves exactly the column layout [[Change Matrix Type]] and [[Transition Function Parameter Matrix Type]] expect, so both convert automatically once connected to [[Patcher]] — no explicit conversion step needed.
 ===== Collecting regional results =====
@@ Line 117: / Line 137: @@
 ^ Port ^ Direction ^ Type ^ Required? ^ Description ^
 | ''globalMapName'' | Input | Name | Yes | The shared name this regional fragment will be collected under. |
-| ''regionalMap'' | Input | Map ([[Regional Map]]) / Categorical Map ([[Regional Categorical Map]]) | Yes | The per-region result to store. |
+| ''regionalMap'' | Input | Map ([[Regional Map]]) / Categorical Map ([[Regional Categorical Map]]) | Yes | The result to store — any map, at any extent, not necessarily already clipped to the region. It's clipped down to the corresponding region automatically. |
 | ''regionId'' | Input | Integer Value | No, auto-bound | Which region this fragment belongs to. |
 | ''regionManager'' | Input | Region Manager | No, auto-bound | The region manager this fragment belongs to. |
+That last point is worth being explicit about: the map passed in doesn't have to already be region-sized — even the full global map would work, since [[Regional Map]]/[[Regional Categorical Map]] clip it to the region internally rather than assuming it's already been through [[Regionalize Map]]/[[Regionalize Categorical Map]]. The worked example below still runs its map through [[Regionalize Map]]/[[Regionalize Categorical Map]] first, but for a different reason: [[Patcher]] itself needs region-sized inputs to compute against, not just to store the result — the clipping done here is redundant on an already region-sized map, not required by [[Regional Map]] itself.
 <code>
+regionChanges := GetTableFromKey changesByRegion regionId;
+regionParams := GetTableFromKey paramsByRegion regionId;
 Region regionId {{
-    regionalSources := RegionalizeMap sourcesMap;
+    regionalLandscape := RegionalizeCategoricalMap { globalMap = landscapeMap, keepNonRegionCells = .yes };
-    regionalFrictions := RegionalizeMap frictionsMap;
+    regionalProbabilities := RegionalizeMap { globalMap = probabilitiesMap, keepNonRegionCells = .yes };
-    { costs = regionalCost, directions = _ } :=
+    { changedLandscape = regionalChanged, corrodedProbabilities = _, remainingChanges = _ } :=
-        CalcCostMap regionalSources regionalFrictions .no;
+        Patcher regionalLandscape regionalProbabilities regionChanges regionParams;
-    RegionalMap "cost_map" regionalCost;
+    RegionalMap "cost_map" regionalChanged;
 }};
 </code>
+[[Patcher]] returns three outputs, and this example only stores one of them. ''corrodedProbabilities'' — the probability surface with used-up areas depleted — and ''remainingChanges'' — a Change Matrix of whatever couldn't be placed, given the current ''neighborWindowLines''/''neighborWindowColumns''/''pruneFactor'' — are both genuinely useful diagnostics in a real model, not values to ignore on principle. They're discarded here only because this page is illustrating region-manager mechanics rather than full Patcher diagnostics. All three outputs are still named explicitly, with the unwanted two bound to ''_'', rather than left out of the destructuring entirely — whether EGO Script allows binding only some of a multi-output functor's outputs isn't confirmed, so naming all of them is the safer pattern used consistently throughout this page.
+A model that does care about ''remainingChanges'' would most likely want it accumulated across regions too, the same way [[#computing_per-class_areas_within_each_region|the area-by-region example]] accumulates its table with [[Mux Table]] — checking, once the loop finishes, whether any region failed to place all of its requested changes.
 ===== Merging regions back into one map =====
@@ Line 147: / Line 176: @@
 A few more things worth knowing before relying on the merge:
+  * **Splitting keeps border cells so Patcher's neighborhood window has real context; merging discards them anyway.** Each region's own border zone duplicates territory another region already owns and computes for itself. Merging with ''mergeNonRegionCells: Yes'' would pull that duplicated, independently-computed data back in instead of leaving each region's own interior result as the authority for its own cells — which is why the worked example uses ''keepNonRegionCells: Yes'' when splitting, but leaves ''mergeNonRegionCells'' at its default ''No''.
   * **The merged map's format is inherited from the first fragment stored, not negotiated across all of them.** Cell type, null value, and categorization all come from whichever [[Regional Map]] or [[Regional Categorical Map]] call registered first for that name — only the dimensions come from the regions map itself. If regions are processed in an order that isn't guaranteed, don't rely on which fragment "wins."
   * **Mismatched null values fail the merge, not silently overwrite each other.** When ''mergeNonRegionCells'' brings overlapping border cells from two different regions together, they can only combine if they agree, or one of them is null — two regions disagreeing on what a cell's value should be raises an error rather than picking one.
@@ Line 159: / Line 189: @@
 <code>
-sourcesMap := LoadMap "c:/data/sources.tif";
+landscapeMap := LoadCategoricalMap "c:/data/landscape.tif" .none .default 0;
-frictionsMap := LoadMap "c:/data/frictions.tif";
+probabilitiesMap := LoadMap "c:/data/probabilities.tif";
 regionsMap := LoadCategoricalMap "c:/data/regions.tif" .none .default 0;
+changesByRegion := LoadTable "c:/data/changes_by_region.csv";
+paramsByRegion := LoadTable "c:/data/params_by_region.csv";
 RegionManager regionsMap 5 {{
     sequenceOut := ForEachCategory regionsMap {{
         regionId = step;
+        regionChanges := GetTableFromKey changesByRegion regionId;
+        regionParams := GetTableFromKey paramsByRegion regionId;
         Region regionId {{
-            regionalSources := RegionalizeMap sourcesMap;
+            regionalLandscape := RegionalizeCategoricalMap { globalMap = landscapeMap, keepNonRegionCells = .yes };
-            regionalFrictions := RegionalizeMap frictionsMap;
+            regionalProbabilities := RegionalizeMap { globalMap = probabilitiesMap, keepNonRegionCells = .yes };
-            { costs = regionalCost, directions = _ } :=
+            { changedLandscape = regionalChanged, corrodedProbabilities = _, remainingChanges = _ } :=
-                CalcCostMap regionalSources regionalFrictions .no;
+                Patcher regionalLandscape regionalProbabilities regionChanges regionParams;
-            RegionalMap "cost_map" regionalCost;
+            RegionalCategoricalMap "changed_landscape" regionalChanged;
         }};
     }};
     _ := Group sequenceOut {{
-        costMap := MergeRegionalMaps "cost_map" .no;
+        finalLandscape := MergeRegionalCategoricalMaps "changed_landscape" .no;
-        SaveMap costMap "c:/data/cost_map_merged.tif";
+        SaveCategoricalMap finalLandscape "c:/data/changed_landscape_merged.tif";
     }};
 }};
 </code>
-One region manager, defined once at the top, is what threads the whole thing together: [[For Each Category]] loops directly against the categories already attached to ''regionsMap'', [[Region]] narrows against the manager once per loop iteration, [[Regional Map]] stores fragments into it, and [[Merge Regional Maps]] — deliberately delayed until the loop's ''sequenceOutput'' fires — reads every one of those fragments back out of the same instance.
+One region manager, defined once at the top, is what threads the whole thing together: [[For Each Category]] loops directly against the categories already attached to ''regionsMap'', [[Get Table From Key]] slices each region's own change matrix and transition parameters out of the two shared tables, [[Region]] narrows against the manager once per loop iteration, [[Regional Categorical Map]] stores fragments into it, and [[Merge Regional Categorical Maps]] — deliberately delayed until the loop's ''sequenceOutput'' fires — reads every one of those fragments back out of the same instance.
 ===== Computing per-class areas within each region =====
@@ Line 203: / Line 239: @@
 | ''table'' | Output | Table | — | The accumulated value entering the current iteration. |
-This section doesn't use [[Region]] at all — ''regionId'' is used directly, since [[Regionalize Categorical Map]] and [[Set Table By Key]] both auto-bind or accept it positionally without needing it re-exposed under a container of its own:
+This section doesn't use [[Region]] at all — ''regionId'' is used directly, since [[Regionalize Categorical Map]] and [[Set Table By Key]] both auto-bind or accept it positionally without needing it re-exposed under a container of its own. There's also no windowed calculation here, so unlike the [[Patcher]] example, ''keepNonRegionCells'' is left at its default — [[Calc Areas]] only tallies cells within the region itself:
 <code>