Differences

This shows you the differences between two versions of the page.

--- weights_of_evidence_calibration [2026/08/10 14:12]
hermann
+++ weights_of_evidence_calibration [2026/08/12 01:22] (current)
hermann
@@ Line 9: / Line 9: @@
 Weights of Evidence is a Bayesian method, originally used in geology to map favorability for events such as mineralization, adapted here to relate a land-use transition to a set of predictive spatial variables.
-For a transition from category i to category j, each class of a predictive variable (a category, or a range of a continuous gray-tone variable) is assigned two numbers: a positive weight (W+), reflecting how the transition's likelihood changes for cells that fall inside that class, and a negative weight (W-), reflecting the same for cells that fall outside it.
+For a transition from category i to category j, each class of a predictive variable (a category, or a range of a continuous gray-tone variable) is assigned two numbers: a positive weight (W<sup>+</sup>), reflecting how the transition's likelihood changes for cells that fall inside that class, and a negative weight (W<sup>-</sup>), reflecting the same for cells that fall outside it.
 [[calc_w._of_e._probability_map|Calc W. Of E. Probability Map]] combines these weights with a transition's baseline probability to produce a probability for each cell, based on which range or category the cell falls into for each variable. This is why a weight's sign has a consistent meaning:
@@ Line 17: / Line 17: @@
   * **Negative** -- lowers the probability below baseline. Cells in that range are less likely than average to make the transition -- the range repels it.
-**Contrast** (C = W+ - W-) measures a range's overall association with the transition: it is what determines whether the range meaningfully favors or repels the transition, or has close to no effect. This is the number the significance test applies to (see [[#determining_coefficients|Determining Coefficients]] below). A range with a Weight_Coefficient near zero and a non-significant Contrast are two ways of saying the same thing: the range isn't doing meaningful explanatory work for the transition.
+**Contrast** (C = W<sup>+</sup> - W<sup>-</sup>) measures a range's overall association with the transition: it is what determines whether the range meaningfully favors or repels the transition, or has close to no effect. This is the number the significance test applies to (see [[#determining_coefficients|Determining Coefficients]] below). A range with a Weight_Coefficient near zero and a non-significant Contrast are two ways of saying the same thing: the range isn't doing meaningful explanatory work for the transition.
 Every variable's weights combine together to produce a transition's overall probability at each cell, which is why the method requires its input variables to be spatially independent of one another (see [[#checking_variable_independence|Checking Variable Independence]] below).
@@ Line 54: / Line 54: @@
 Determine Weights Of Evidence Ranges finds range boundaries by watching how the transition's weight changes as it sweeps across a continuous variable's values, rather than requiring pre-set cut points.
-Starting at the variable's minimum value, it grows a sequence of nested buffers outward in steps equal to the Increment: the first buffer covers up to one Increment past the minimum, the second covers up to two Increments past it, and so on, each one containing everything the previous one did plus a bit more. For each buffer, it counts how many cells fall inside it and how many of those cells actually made the transition, and from that pair derives the weight (W+) that buffer's edge would produce if used as a single cutoff. Sweeping through the whole sequence of buffers produces a curve: weight as a function of how far the buffer has grown.
+Starting at the variable's minimum value, it grows a sequence of nested buffers outward in steps equal to the Increment: the first buffer covers up to one Increment past the minimum, the second covers up to two Increments past it, and so on, each one containing everything the previous one did plus a bit more. For each buffer, it counts how many cells fall inside it and how many of those cells actually made the transition, and from that pair derives the weight (W<sup>+</sup>) that buffer's edge would produce if used as a single cutoff. Sweeping through the whole sequence of buffers produces a curve: weight as a function of how far the buffer has grown.
 Where this curve stays flat, the variable isn't discriminating anything at that point -- going a bit further out doesn't change the odds. Where the curve bends, that's where crossing that value of the variable starts to matter. So the range boundaries are wherever the curve changes direction, which is a shape-finding problem, not something that can be set analytically in advance.
@@ Line 80: / Line 80: @@
   - For any row where ''Significant = No'', read that row's ''Possible_Transitions'' value and set the minimum delta slightly above it, then repeat from step 1.
   - If several rows are not significant at once, use the largest ''Possible_Transitions'' value among them as the basis for the next minimum delta.
-This loop can also be driven automatically within a single model run -- see [[#automating_the_range_tuning_loop|Automating the Range Tuning Loop]] below.
 Each continuous gray-tone variable goes through this tuning individually -- the ranges for one variable do not carry over to another. Category variables skip this section entirely, since none of the four Skeleton parameters apply to them.
@@ Line 108: / Line 106: @@
   * **Possible_Transitions** -- the number of cells that could have made this transition, given they started in the "from" category and fell inside this variable's range. What counts as "possible" is controlled by ''potentialTransitionCriteria'' (see below).
   * **Executed_Transitions** -- of those possible cells, how many actually made the transition during the observed period.
-  * **Weight_Coefficient** -- the resulting weight (W+) for that range or category: its contribution to the transition's probability wherever it applies.
+  * **Weight_Coefficient** -- the resulting weight (W<sup>+</sup>) for that range or category: its contribution to the transition's probability wherever it applies.
-  * **Contrast** -- W+ minus W- for that range or category. Positive values favor the transition, negative values repel it, and values near zero mean the range has little effect. See [[#what_is_weights_of_evidence|What Is Weights of Evidence]] above for how this is derived.
+  * **Contrast** -- W<sup>+</sup> minus W<sup>-</sup> for that range or category. Positive values favor the transition, negative values repel it, and values near zero mean the range has little effect. See [[#what_is_weights_of_evidence|What Is Weights of Evidence]] above for how this is derived.
   * **Significant** -- whether the Contrast passes the 95% confidence threshold (|C| > 1.96 S(C)). Rows marked "No" are exactly the ones the range-tuning loop above targets when adjusting the minimum delta.
@@ Line 160: / Line 158: @@
 This example ties the pieces above together: one continuous variable, one categorical variable, a range calculation, a coefficient calculation, and saving both results.
-Each variable here is loaded as its own separate map, connected to its own [[name_map|Name Map]]. Every map, cube or not, is organized into named [[map_type|layers]] -- a single, non-cube map simply has one, defaulting to ''layer_0'' unless it defines its own name; [[extract_map_layer|Extract Map Layer]] pulls a specific layer out on its own, if one is needed outside a Skeleton reference. It's also possible to assemble every predictive variable into a single multi-layer cube via [[create_cube_map|Create Cube Map]], the way Lesson 18's own example does -- but for a handful of variables that adds a layer of indirection without much benefit. A cube's usefulness shows up with many static variables: one [[name_map|Name Map]] alias then covers the whole set, and the model loads and connects one file instead of one per variable. Inside a cube, each layer's name comes from [[number_and_name_map|Number And Name Map]] rather than plain Name Map, since it's assigning both an index and a name to a single layer as it's assembled.
+Each variable here is loaded as its own separate map, connected to its own [[name_map|Name Map]]. Every map, cube or not, is organized into named [[map_type|layers]] -- a single, non-cube map defaults to one layer named ''layer_0'', but the functor that produced the map can assign its own name instead: [[calc_distance_map|Calc Distance Map]], for instance, names its output ''distance_to_<category>'' rather than using the default. [[extract_map_layer|Extract Map Layer]]'s own editor lists a map's actual layer names, which is the place to check when in doubt, and it also pulls a specific layer out on its own if one is needed outside a Skeleton reference. It's also possible to assemble every predictive variable into a single multi-layer cube via [[create_cube_map|Create Cube Map]], the way Lesson 18's own example does -- but for a handful of variables that adds a layer of indirection without much benefit. A cube's usefulness shows up with many static variables: one [[name_map|Name Map]] alias then covers the whole set, and the model loads and connects one file instead of one per variable. Inside a cube, each layer's name comes from [[number_and_name_map|Number And Name Map]] rather than plain Name Map, since it's assigning both an index and a name to a single layer as it's assembled.
 <code>
@@ Line 178: / Line 176: @@
 // alias is whatever name is given to that map's Name Map hook below --
 // this is how each container matches a Skeleton entry back to the actual
-// map connected to it. A map that does not explicitly define its own
+// map connected to it. "layer_0" is only the default layer name for a
-// layer names uses the default "layer_0" for its first (here, only)
+// map that doesn't define its own -- check with Extract Map Layer's
-// layer. A continuous gray-tone variable is followed by four numbers, in
+// editor if a map's actual layer name is in doubt. A continuous
-// this order: Increment, Maximum Delta, Minimum Delta, Tolerance Angle
+// gray-tone variable is followed by four numbers, in this order:
-// -- see "The Weights of Evidence Skeleton" above. A category variable
+// Increment, Maximum Delta, Minimum Delta, Tolerance Angle -- see "The
-// takes no numbers.
+// Weights of Evidence Skeleton" above. A category variable takes no
+// numbers.
 skeleton := WeightsOfEvidenceSkeleton [
 ->1 [
@@ Line 222: / Line 221: @@
 The saved ''coefficients.dcf'' is the finished Weights object -- reload it with [[load_weights|Load Weights]] wherever [[calc_w._of_e._probability_map|Calc W. Of E. Probability Map]] needs it (see [[#using_the_weights_downstream|Using the Weights Downstream]]). The saved ''report.csv'' can be opened directly in a spreadsheet to review ''Contrast'' and ''Significant'' per range, without needing to scroll the message log.
-===== Automating the Range Tuning Loop =====
-The manual procedure in [[#determining_ranges_for_a_continuous_variable|Determining Ranges for a Continuous Variable]] above adjusts Minimum Delta by hand, rerunning the model and reading the ''Significant'' column each time. This example automates that same procedure in a single model run.
-The key that makes this possible: a [[weights_of_evidence_skeleton_type|Weights Of Evidence Skeleton]] can be expressed as a Table instead of a fixed literal, using the column format ''From*'', ''To*'', ''Variable_Name*'', ''Categorical'', ''Increment'', ''Minimum_Delta'', ''Maximum_Delta'', ''Tolerance_Angle'' -- ''Categorical'' is 0 for a continuous gray-tone variable, 1 for a category variable. A Table [[type_system#table_format_of_table-shaped_types|converts to a Skeleton automatically]] wherever one is expected, and unlike a literal, a Table's cells can be rewritten by ordinary functors such as [[set_table_cell_value|Set Table Cell Value]]. That is what lets Minimum Delta change from one iteration to the next.
-The loop's *feedback* value has to be a Table, since it's rewritten by [[set_table_cell_value|Set Table Cell Value]] on every pass -- but the *starting* value doesn't have to be. Because the conversion works in both directions, the starting Skeleton can instead be built through its own graphical editor, the same familiar interface used everywhere else on this page, and it will convert to a Table automatically the first time it feeds into the loop. The example below does this rather than hand-writing the starting Table.
-The model wraps the range-and-coefficient calculation in a [[do_while|Do While]] loop. Each pass: runs both calibration functors using the current Skeleton table, drills into the resulting ''report'' to find every range for the variable and transition in question, and scans them for any row where ''Significant'' is 0. If it finds one, it takes the largest ''Possible_Transitions'' among the failing rows, moves Minimum Delta just past it, writes that back into the Skeleton table, and loops again. Once every row comes back significant, the loop stops and the results are saved -- the same two outputs as the basic example above.
-This assumes exactly one variable and one transition, so each ''Range_Lower_Limit'' maps to exactly one row; a model with more variables or transitions would need one additional nested [[for_each|For Each]] per extra key column -- see "Sub-Tables" in [[manipulating_tables_and_lookup_tables|Manipulating Tables and Lookup Tables]].
-<code>
-// ---- Fixed inputs ----
-Group {{
-    initialLandscape := LoadCategoricalMap "landscape_1997.tif";
-    finalLandscape   := LoadCategoricalMap "landscape_2000.tif";
-}};
-// continuous gray-tone
-distanceToRoads := LoadMap "d_all_roads.tif";
-// Starting Skeleton, configured through its own graphical editor -- see
-// "The Weights of Evidence Skeleton" above -- rather than hand-written
-// as a Table. Converts to a Table automatically the first time it feeds
-// into the loop below, which is what lets Minimum_Delta be rewritten
-// each iteration from that point on.
-startingSkeleton := WeightsOfEvidenceSkeleton [
-->1 [
-        "roads/layer_0" 100 500000 10 5,
-        "protected/layer_0"
-    ]
-];
-// Tune Minimum Delta until every range in the report comes back
-// Significant.
-_ := DoWhile {{
-    skeletonTable := MuxTable startingSkeleton nextSkeletonTable;
-    ranges := DetermineWeightsOfEvidenceRanges {
-        initialLandscape = initialLandscape,
-        finalLandscape = finalLandscape,
-        skeleton = skeletonTable
-    } {{
-        NameMap distanceToRoads "roads";
-    }};
-    { weights = weights, report = report } := DetermineWeightsOfEvidenceCoefficients {
-        initialLandscape = initialLandscape,
-        finalLandscape = finalLandscape,
-        ranges = ranges
-    } {{
-        NameMap distanceToRoads "roads";
-    }};
-    // Drop the three fixed keys (Transition_From, Transition_To,
-    // Variable) in one call, leaving a table keyed by Range_Lower_Limit
-    // and Range_Upper_Limit -- one row per determined range.
-    rangesForVariable := GetTableFromKey report [2, 1, "roads/layer_0"];
-    lowerLimits := GetTableKeys rangesForVariable;
-    _ := ForEach lowerLimits {{
-        anySoFar   := MuxValue 0 nextAny;
-        worstSoFar := MuxValue 0 nextWorst;
-        lowerLimit := Step;
-        // Peel off Range_Lower_Limit too, landing on the single row for
-        // this range, still formally keyed by Range_Upper_Limit.
-        thisRange := GetTableFromKey rangesForVariable lowerLimit;
-        upperLimitKeys := GetTableKeys thisRange;
-        upperLimit := GetTableValue upperLimitKeys [1] 2;
-        significant := GetTableValue thisRange upperLimit "Significant";
-        possible    := GetTableValue thisRange upperLimit "Possible_Transitions";
-        isNonSignificant := $ [ not $significant ];
-        nextAny := $ [ $isNonSignificant or $anySoFar ];
-        nextWorst := $ [
-            if $isNonSignificant and $possible > $worstSoFar
-            then $possible
-            else $worstSoFar
-        ];
-    }};
-    // Move Minimum Delta just past the range that failed significance,
-    // and loop again if anything still needs it.
-    newMinimumDelta := $ [ $nextWorst + 1 ];
-    nextSkeletonTable := SetTableCellValue skeletonTable "Minimum_Delta" [2, 1, "roads/layer_0"] newMinimumDelta;
-    keepGoing := $ [ $nextAny ];
-    _ := SetWhileCondition keepGoing;
-}};
-// Step 3: save the results
-Group {{
-    SaveWeights { weights = weights, filename = "coefficients.dcf" };
-    SaveTable   { table = report, filename = "report.csv" };
-}};
-</code>
 ===== Tips =====
@@ Line 332: / Line 227: @@
   * The [[name_map|Name Map]] aliases used inside [[determine_weights_of_evidence_ranges|Determine Weights Of Evidence Ranges]] and [[determine_weights_of_evidence_coefficients|Determine Weights Of Evidence Coefficients]] must match exactly between the two containers -- this is how each container knows which named map corresponds to which variable in the Skeleton.
   * To inspect or chart a report outside the message log, connect its ''report'' output to [[save_table|Save Table]] and export to CSV.