Model, loss, thresholds, metrics¶
Numbers are from DAM 1 (research/stable/ordinal_regression_mtl_mar_2024/, config.yaml).
DAM 2/3 reuse the same head, loss and metrics (Experiments).
Architecture (DAM 1)¶
flowchart TB
A["raw audio<br/>[ΣW, ≤480000]"] -->|"pad to 480,000"| B["log-mel on GPU<br/>[ΣW, 80, 3000]"]
B --> C["Whisper small.en encoder<br/>LoRA r=32 α=64, conv1/conv2 full<br/>[ΣW, 1500, 768]"]
C -->|"mean over time"| D["[ΣW, 768]"]
D --> E["Shared: Linear 768→256 + Mish<br/>Linear 256→64 (no act.)"]
E --> F1["depression head<br/>64→128 Mish Dropout 0.4 →1 (no bias)"]
E --> F2["anxiety head<br/>64→128 Mish Dropout 0.4 →1 (no bias)"]
F1 --> G["score per window<br/>[ΣW, 1] per task"]
F2 --> G
G -->|"mean per recording (length)"| H["[B] per task"]| Part | Setting | Where |
|---|---|---|
| Log-mel | 80 bins, n_fft 400, hop 160, per-window z-norm, log10 clip max−8, (x+4)/4, then per-bin shift to IDEAL_LOGMEL_ENERGIES |
WhisperTorchFeatureExtractor |
| Encoder | openai/whisper-small.en, decoder dropped, 1500 positions, dropouts 0 |
WhisperBackbone |
| LoRA | r 32, α 64, dropout 0.4, all-linear, bias all; conv1, conv2 trained fully |
KintsugiBackboneBase.apply_lora |
| Shared layers | [256, 64]; last layer is the shared factor B in Wt = Ht·B |
MultitaskHead.SharedLayers |
| Task head | proj 128, dropout 0.4, final Linear without bias (CORAL holds the biases) | MultitaskHead.TaskHead |
| Optimizer | AdamW: backbone lr 1.6e-4, head + CORAL biases lr 8e-4, wd 1e-3; MultiStepLR γ 0.5 at epochs 20, 26 | DepAnxClassifier.configure_optimizers |
| Precision | fp16 on GPU, DDP find_unused_parameters=True |
launcher |
3 · Model (DAM 1)¶
Raw audio windows become log-mel images on the GPU, the Whisper encoder (with LoRA) turns each window into a 768-d vector, and a shared layer feeds one small head per task that outputs a single score.
Main path: steps 25–31 of a train run (walkthrough). Gold boxes open the step; grey boxes are side paths and open the code.
flowchart LR
subgraph research_stable_ordinal_regression_mtl_mar_2024_model_py___box["mtl_mar_2024/model.py"]
direction TB
research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier["<b>DepAnxClassifier()</b><br/>DAM 1: log-mel -› Whisper small.en<br/>encoder (LoRA) -› mean pool -›<br/>multitask CORAL head."]
class research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier opt
click research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/research/stable/ordinal_regression_mtl_mar_2024/model.py#L25" _blank
research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier_forward["<b>27 · DepAnxClassifier.forward()</b><br/>Backbone then MultitaskHead -›<br/>({task: [ΣW, 1] scores}, lengths)."]
class research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier_forward main
click research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier_forward "../walkthrough/#step-27"
research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier_forward_backbone["<b>28 · DepAnxClassifier.forward_backbone()</b><br/>Pad raw audio to 480,000 samples,<br/>log-mel on GPU, Whisper encoder -›<br/>[ΣW, 768]."]
class research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier_forward_backbone main
click research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier_forward_backbone "../walkthrough/#step-28"
research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier_configure_optimizers["<b>DepAnxClassifier.configure_optimizers()</b><br/>AdamW with two groups (backbone,<br/>head + CORAL biases) and<br/>MultiStepLR."]
class research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier_configure_optimizers opt
click research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier_configure_optimizers "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/research/stable/ordinal_regression_mtl_mar_2024/model.py#L109" _blank
end
subgraph kirad_feature_extraction_whisper_py___box["feature_extraction_whisper.py"]
direction TB
kirad_feature_extraction_whisper_py__WhisperTorchFeatureExtractor_forward["<b>29 · WhisperTorchFeatureExtractor.forward()</b><br/>Normalize each window, compute<br/>log-mel, shift bins to<br/>IDEAL_LOGMEL_ENERGIES."]
class kirad_feature_extraction_whisper_py__WhisperTorchFeatureExtractor_forward main
click kirad_feature_extraction_whisper_py__WhisperTorchFeatureExtractor_forward "../walkthrough/#step-29"
kirad_feature_extraction_whisper_py__WhisperTorchFeatureExtractor__torch_extract_fbank_features["<b>WhisperTorchFeatureExtractor._torch_extract_fbank_features()</b><br/>Log10 mel spectrogram, clipped to<br/>8 below the max, scaled (x + 4) /<br/>4 as Whisper."]
class kirad_feature_extraction_whisper_py__WhisperTorchFeatureExtractor__torch_extract_fbank_features opt
click kirad_feature_extraction_whisper_py__WhisperTorchFeatureExtractor__torch_extract_fbank_features "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/feature_extraction_whisper.py#L97" _blank
end
subgraph kirad_base_models_py___box["base_models.py"]
direction TB
kirad_base_models_py__WhisperBackbone_forward["<b>30 · WhisperBackbone.forward()</b><br/>Whisper encoder on log-mel<br/>windows, mean over time -› [ΣW, D]<br/>(768 for small.en)."]
class kirad_base_models_py__WhisperBackbone_forward main
click kirad_base_models_py__WhisperBackbone_forward "../walkthrough/#step-30"
kirad_base_models_py__KintsugiBackboneBase_apply_lora["<b>KintsugiBackboneBase.apply_lora()</b><br/>Wrap self.backbone with PEFT LoRA<br/>from lora_params; no-op (full<br/>fine-tune) if empty."]
class kirad_base_models_py__KintsugiBackboneBase_apply_lora opt
click kirad_base_models_py__KintsugiBackboneBase_apply_lora "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/base_models.py#L818" _blank
kirad_base_models_py__MultitaskHead_forward["<b>31 · MultitaskHead.forward()</b><br/>Shared layers, then one TaskHead<br/>per task -› {task: [ΣW, 1] score}."]
class kirad_base_models_py__MultitaskHead_forward main
click kirad_base_models_py__MultitaskHead_forward "../walkthrough/#step-31"
kirad_base_models_py__MultitaskHead_SharedLayers["<b>MultitaskHead.SharedLayers()</b><br/>Shared MLP: Linear+Mish per<br/>proj_dim, last Linear without<br/>activation (e.g. D-›256-›64)."]
class kirad_base_models_py__MultitaskHead_SharedLayers opt
click kirad_base_models_py__MultitaskHead_SharedLayers "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/base_models.py#L1002" _blank
kirad_base_models_py__MultitaskHead_TaskHead["<b>MultitaskHead.TaskHead()</b><br/>Per-task head: Linear(in,<br/>proj_dim) -› Mish -› Dropout -›<br/>Linear(proj_dim, 1, no bias)."]
class kirad_base_models_py__MultitaskHead_TaskHead opt
click kirad_base_models_py__MultitaskHead_TaskHead "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/base_models.py#L1034" _blank
kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_training_step["<b>25 · MultiTaskOrdinalRegressionPLModule.training_step()</b><br/>Lightning train step: returns the<br/>step dict with total_loss."]
class kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_training_step main
click kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_training_step "../walkthrough/#step-25"
kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_step["<b>26 · MultiTaskOrdinalRegressionPLModule.step()</b><br/>Run model once, sum each task's<br/>loss (task.step) into total_loss."]
class kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_step main
click kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_step "../walkthrough/#step-26"
end
from_prev(["◀ from step 24: broadcast_collate()"])
class from_prev edge
from_prev ==> kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_training_step
click from_prev "#stage-data"
kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_training_step ==> kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_step
kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_step ==> research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier_forward
research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier_forward ==> research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier_forward_backbone
research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier_forward_backbone ==> kirad_feature_extraction_whisper_py__WhisperTorchFeatureExtractor_forward
kirad_feature_extraction_whisper_py__WhisperTorchFeatureExtractor_forward ==> kirad_base_models_py__WhisperBackbone_forward
kirad_base_models_py__WhisperBackbone_forward ==> kirad_base_models_py__MultitaskHead_forward
to_next(["▶ next, step 32: OrdinalRegressionPLModule.step()"])
class to_next edge
kirad_base_models_py__MultitaskHead_forward ==> to_next
click to_next "#stage-loss"
kirad_feature_extraction_whisper_py__WhisperTorchFeatureExtractor_forward -.-> kirad_feature_extraction_whisper_py__WhisperTorchFeatureExtractor__torch_extract_fbank_features
research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier_forward_backbone -.->|"log-mel"| kirad_base_models_py__WhisperBackbone_forward
research_stable_ordinal_regression_mtl_mar_2024_model_py__DepAnxClassifier_forward -.->|"768-d"| kirad_base_models_py__MultitaskHead_forward
kirad_base_models_py__MultitaskHead_forward -.-> kirad_base_models_py__MultitaskHead_SharedLayers
kirad_base_models_py__MultitaskHead_SharedLayers -.->|"64-d"| kirad_base_models_py__MultitaskHead_TaskHead
classDef main fill:#FFF4DC,stroke:#A67C1B,stroke-width:2px,color:#1A1A1A
classDef opt fill:#F3F4F6,stroke:#B8BEC7,color:#555,stroke-dasharray:4 3
classDef edge fill:#FFFFFF,stroke:#1F7A5C,color:#1F7A5C,stroke-width:1.5px
| Step | Function | File | What it does |
|---|---|---|---|
| — | DepAnxClassifier |
research/stable/ordinal_regression_mtl_mar_2024/model.py:25 |
DAM 1: log-mel -> Whisper small.en encoder (LoRA) -> mean pool -> multitask CORAL head. |
| 27 | DepAnxClassifier.forward |
research/stable/ordinal_regression_mtl_mar_2024/model.py:84 |
Backbone then MultitaskHead -> ({task: [ΣW, 1] scores}, lengths). |
| 28 | DepAnxClassifier.forward_backbone |
research/stable/ordinal_regression_mtl_mar_2024/model.py:89 |
Pad raw audio to 480,000 samples, log-mel on GPU, Whisper encoder -> [ΣW, 768]. |
| — | DepAnxClassifier.configure_optimizers |
research/stable/ordinal_regression_mtl_mar_2024/model.py:109 |
AdamW with two groups (backbone, head + CORAL biases) and MultiStepLR. |
| 29 | WhisperTorchFeatureExtractor.forward |
kirad/feature_extraction_whisper.py:138 |
Normalize each window, compute log-mel, shift bins to IDEAL_LOGMEL_ENERGIES. |
| — | WhisperTorchFeatureExtractor._torch_extract_fbank_features |
kirad/feature_extraction_whisper.py:97 |
Log10 mel spectrogram, clipped to 8 below the max, scaled (x + 4) / 4 as Whisper. |
| 30 | WhisperBackbone.forward |
kirad/base_models.py:877 |
Whisper encoder on log-mel windows, mean over time -> [ΣW, D] (768 for small.en). |
| — | KintsugiBackboneBase.apply_lora |
kirad/base_models.py:818 |
Wrap self.backbone with PEFT LoRA from lora_params; no-op (full fine-tune) if empty. |
| 31 | MultitaskHead.forward |
kirad/base_models.py:1075 |
Shared layers, then one TaskHead per task -> {task: [ΣW, 1] score}. |
| — | MultitaskHead.SharedLayers |
kirad/base_models.py:1002 |
Shared MLP: Linear+Mish per proj_dim, last Linear without activation (e.g. D->256->64). |
| — | MultitaskHead.TaskHead |
kirad/base_models.py:1034 |
Per-task head: Linear(in, proj_dim) -> Mish -> Dropout -> Linear(proj_dim, 1, no bias). |
| 25 | MultiTaskOrdinalRegressionPLModule.training_step |
kirad/base_models.py:722 |
Lightning train step: returns the step dict with total_loss. |
| 26 | MultiTaskOrdinalRegressionPLModule.step |
kirad/base_models.py:734 |
Run model once, sum each task's loss (task.step) into total_loss. |
CORAL loss¶
One score s per window and task; K = 27 (PHQ-9, num_classes 28) or 21 (GAD-7, 22) learned
biases bk. Logits s + bk; target is the thermometer code [y > k]
(torch.gt(y, arange(K))); BCE summed over cutoffs, each weighted 0/1 by
ordinal_loss_target_type (exact28 / exact22 = all ones).
| Term | Weight (config key, default) | DAM 1 current |
|---|---|---|
| CORAL | coral_loss_weight, 1.0 |
1.0 |
| Variance of window scores within a recording | score_variance_loss_weight, 0.0 |
40.0 |
| MSE to teacher score (KD) | kd_loss_weight, 0.0; needs score_targets_csv |
0 |
Train: per-window loss, averaged within each recording, then over recordings. Val/test: loss of the per-recording mean score
(mean_before_loss). Total loss = sum over tasks.
4 · Loss¶
Each task turns its score into 27 (or 21) ordered yes/no questions with CORAL. Train: CORAL loss per window, averaged per recording; val/test: loss of the per-recording mean score. A variance penalty makes windows of the same recording agree.
Main path: steps 32–35 of a train run (walkthrough). Gold boxes open the step; grey boxes are side paths and open the code.
flowchart LR
subgraph kirad_base_models_py___box["base_models.py"]
direction TB
kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_step["<b>MultiTaskOrdinalRegressionPLModule.step()</b><br/>Run model once, sum each task's<br/>loss (task.step) into total_loss."]
class kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_step opt
click kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_step "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/base_models.py#L734" _blank
kirad_base_models_py__OrdinalRegressionPLModule_step["<b>32 · OrdinalRegressionPLModule.step()</b><br/>Accumulate per-recording mean/var<br/>scores for val/test, then return<br/>the loss."]
class kirad_base_models_py__OrdinalRegressionPLModule_step main
click kirad_base_models_py__OrdinalRegressionPLModule_step "../walkthrough/#step-32"
kirad_base_models_py__OrdinalRegressionPLModule_compute_and_log_loss["<b>33 · OrdinalRegressionPLModule.compute_and_log_loss()</b><br/>Loss = coral_w·CORAL +<br/>var_w·Var(window scores) +<br/>kd_w·MSE(teacher), logged."]
class kirad_base_models_py__OrdinalRegressionPLModule_compute_and_log_loss main
click kirad_base_models_py__OrdinalRegressionPLModule_compute_and_log_loss "../walkthrough/#step-33"
end
subgraph kirad_losses_py___box["losses.py"]
direction TB
kirad_losses_py__CORALLoss_forward["<b>35 · CORALLoss.forward()</b><br/>Weighted sum of BCE over cutoffs:<br/>sigmoid(s + b_k) vs [y › k] for<br/>each cutoff k."]
class kirad_losses_py__CORALLoss_forward main
click kirad_losses_py__CORALLoss_forward "../walkthrough/#step-35"
end
subgraph kirad_utils_py___box["utils.py"]
direction TB
kirad_utils_py__average_and_variance_tensor_in_segments["<b>34 · average_and_variance_tensor_in_segments()</b><br/>Per-recording mean and (biased)<br/>variance of window scores: [ΣW,<br/>...] -› two [B, ...]."]
class kirad_utils_py__average_and_variance_tensor_in_segments main
click kirad_utils_py__average_and_variance_tensor_in_segments "../walkthrough/#step-34"
kirad_utils_py__average_tensor_in_segments["<b>average_tensor_in_segments()</b><br/>Mean over each recording's<br/>windows: [ΣW, ...] -› [B, ...]<br/>using `length`."]
class kirad_utils_py__average_tensor_in_segments opt
click kirad_utils_py__average_tensor_in_segments "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/utils.py#L90" _blank
end
from_prev(["◀ from step 31: MultitaskHead.forward()"])
class from_prev edge
from_prev ==> kirad_base_models_py__OrdinalRegressionPLModule_step
click from_prev "#stage-model"
kirad_base_models_py__OrdinalRegressionPLModule_step ==> kirad_base_models_py__OrdinalRegressionPLModule_compute_and_log_loss
kirad_base_models_py__OrdinalRegressionPLModule_compute_and_log_loss ==> kirad_utils_py__average_and_variance_tensor_in_segments
kirad_utils_py__average_and_variance_tensor_in_segments ==> kirad_losses_py__CORALLoss_forward
to_next(["▶ next, step 36: MultiTaskOrdinalRegressionPLModule.on_validation_test_epoch_end()"])
class to_next edge
kirad_losses_py__CORALLoss_forward ==> to_next
click to_next "#stage-metrics"
kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_step -.->|"per task"| kirad_base_models_py__OrdinalRegressionPLModule_step
kirad_utils_py__average_and_variance_tensor_in_segments -.-> kirad_utils_py__average_tensor_in_segments
kirad_base_models_py__OrdinalRegressionPLModule_compute_and_log_loss -.-> kirad_losses_py__CORALLoss_forward
classDef main fill:#FFF4DC,stroke:#A67C1B,stroke-width:2px,color:#1A1A1A
classDef opt fill:#F3F4F6,stroke:#B8BEC7,color:#555,stroke-dasharray:4 3
classDef edge fill:#FFFFFF,stroke:#1F7A5C,color:#1F7A5C,stroke-width:1.5px
| Step | Function | File | What it does |
|---|---|---|---|
| — | MultiTaskOrdinalRegressionPLModule.step |
kirad/base_models.py:734 |
Run model once, sum each task's loss (task.step) into total_loss. |
| 32 | OrdinalRegressionPLModule.step |
kirad/base_models.py:572 |
Accumulate per-recording mean/var scores for val/test, then return the loss. |
| 33 | OrdinalRegressionPLModule.compute_and_log_loss |
kirad/base_models.py:504 |
Loss = coral_w·CORAL + var_w·Var(window scores) + kd_w·MSE(teacher), logged. |
| 35 | CORALLoss.forward |
kirad/losses.py:43 |
Weighted sum of BCE over cutoffs: sigmoid(s + b_k) vs [y > k] for each cutoff k. |
| 34 | average_and_variance_tensor_in_segments |
kirad/utils.py:122 |
Per-recording mean and (biased) variance of window scores: [ΣW, ...] -> two [B, ...]. |
| — | average_tensor_in_segments |
kirad/utils.py:90 |
Mean over each recording's windows: [ΣW, ...] -> [B, ...] using length. |
Thresholds¶
Tuned after every validation on the per-recording scores, stored as buffers in the checkpoint, reused unchanged on test.
| Method | Optimizes | Algorithm |
|---|---|---|
accuracy, absolute_error, macro_recall |
per-sample cost | DP, O(n) |
macro_precision, macro_f1 |
per-class cost | DP over cost matrix, O(t²) |
coral |
none | −bk at the metric cutoffs (learned in training) |
Candidates: midpoints between unique validation scores, plus ±∞. Classes come from
METRIC_TARGET_CUTOFFS: depression (9, 14) → 3 classes, anxiety (4, 9, 14) → 4 classes; a label
counts the cutoffs it exceeds (strict >). Report cards default to macro_f1.
Metrics¶
For each binary cut (e.g. PHQ > 9, PHQ > 14), IndetSnSpArray evaluates every (lower, upper)
threshold pair; scores in between are indeterminate. At budgets of 0 % and 40 %
indeterminate, it reports AUROC and the best min(Sn, Sp) (sn_eq_sp). Cuts are averaged per
task, tasks are combined by harmonic mean.
| Metric | Use |
|---|---|
metrics/val/sn_eq_sp@0%i |
selects the checkpoint and drives early stopping (patience 10 validations) |
metrics/val/auroc@0%i, @40%i |
reported |
metrics/test/sn_at_val_eq_thresh@…, sp_…, indet_frac_… |
test at thresholds chosen on val |
A validation runs every training_examples_per_eval recordings (16,000 in DAM 1), else once per epoch.
5 · Thresholds and metrics¶
After every validation, scores of all recordings are gathered, five threshold sets are tuned by dynamic programming, a sixth is read from the CORAL biases, and Sn/Sp with an indeterminate band are computed. Test reuses the validation thresholds.
Main path: steps 36–41 of a train run (walkthrough). Gold boxes open the step; grey boxes are side paths and open the code.
flowchart LR
subgraph kirad_base_models_py___box["base_models.py"]
direction TB
kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_on_validation_test_epoch_end["<b>36 · MultiTaskOrdinalRegressionPLModule.on_validation_test_epoch_end()</b><br/>Run each task's epoch end, then<br/>log harmonic mean of shared<br/>metrics across tasks."]
class kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_on_validation_test_epoch_end main
click kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_on_validation_test_epoch_end "../walkthrough/#step-36"
kirad_base_models_py__OrdinalRegressionPLModule_on_validation_test_epoch_end["<b>37 · OrdinalRegressionPLModule.on_validation_test_epoch_end()</b><br/>Gather scores, log W&B table, tune<br/>thresholds on val, compute Sn/Sp<br/>metrics."]
class kirad_base_models_py__OrdinalRegressionPLModule_on_validation_test_epoch_end main
click kirad_base_models_py__OrdinalRegressionPLModule_on_validation_test_epoch_end "../walkthrough/#step-37"
kirad_base_models_py__OrdinalRegressionPLModule_quantize_labels["<b>38 · OrdinalRegressionPLModule.quantize_labels()</b><br/>Map input labels (e.g. PHQ-9 sum)<br/>to output labels (e.g. 0/1/2 for<br/>L/M/H)."]
class kirad_base_models_py__OrdinalRegressionPLModule_quantize_labels main
click kirad_base_models_py__OrdinalRegressionPLModule_quantize_labels "../walkthrough/#step-38"
kirad_base_models_py__OrdinalRegressionPLModule_log_ordinal_metrics["<b>41 · OrdinalRegressionPLModule.log_ordinal_metrics()</b><br/>Log AUROC and min(Sn, Sp) per<br/>binary cut and indet budget, plus<br/>their mean."]
class kirad_base_models_py__OrdinalRegressionPLModule_log_ordinal_metrics main
click kirad_base_models_py__OrdinalRegressionPLModule_log_ordinal_metrics "../walkthrough/#step-41"
kirad_base_models_py__OrdinalRegressionPLModule_log_thresholding_metrics["<b>OrdinalRegressionPLModule.log_thresholding_metrics()</b><br/>Log each tuned method's cost when<br/>scored with every method's<br/>thresholds."]
class kirad_base_models_py__OrdinalRegressionPLModule_log_thresholding_metrics opt
click kirad_base_models_py__OrdinalRegressionPLModule_log_thresholding_metrics "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/base_models.py#L315" _blank
end
subgraph kirad_ordinal_thresholding_py___box["ordinal_thresholding.py"]
direction TB
kirad_ordinal_thresholding_py__OptimalOrdinalThresholdingViaDynamicProgramming_tune_thresholds["<b>40 · OptimalOrdinalThresholdingViaDynamicProgramming.tune_thresholds()</b><br/>Set thresholds by DP over<br/>midpoints of unique scores<br/>(+/-inf); return the best metric."]
class kirad_ordinal_thresholding_py__OptimalOrdinalThresholdingViaDynamicProgramming_tune_thresholds main
click kirad_ordinal_thresholding_py__OptimalOrdinalThresholdingViaDynamicProgramming_tune_thresholds "../walkthrough/#step-40"
kirad_ordinal_thresholding_py__OptimalCostPerSampleOrdinalThresholding_dp_step["<b>OptimalCostPerSampleOrdinalThresholding.dp_step()</b><br/>O(n) DP step: histogram per-sample<br/>costs over candidates, cumsum,<br/>running max."]
class kirad_ordinal_thresholding_py__OptimalCostPerSampleOrdinalThresholding_dp_step opt
click kirad_ordinal_thresholding_py__OptimalCostPerSampleOrdinalThresholding_dp_step "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/ordinal_thresholding.py#L291" _blank
kirad_ordinal_thresholding_py__OptimalCostPerClassOrdinalThresholding_dp_step["<b>OptimalCostPerClassOrdinalThresholding.dp_step()</b><br/>O(T^2) DP step: add prev_cost to<br/>each lower-threshold row and max<br/>over it."]
class kirad_ordinal_thresholding_py__OptimalCostPerClassOrdinalThresholding_dp_step opt
click kirad_ordinal_thresholding_py__OptimalCostPerClassOrdinalThresholding_dp_step "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/ordinal_thresholding.py#L450" _blank
kirad_ordinal_thresholding_py__CORALThresholding_tune_thresholds["<b>CORALThresholding.tune_thresholds()</b><br/>Copy -biases[threshold_indices]<br/>from the CORAL loss; scores and<br/>labels are ignored."]
class kirad_ordinal_thresholding_py__CORALThresholding_tune_thresholds opt
click kirad_ordinal_thresholding_py__CORALThresholding_tune_thresholds "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/ordinal_thresholding.py#L85" _blank
end
subgraph kirad_metrics_py___box["metrics.py"]
direction TB
kirad_metrics_py__MultiCatMetric["<b>MultiCatMetric()</b><br/>Accumulate parallel tensors over<br/>batches and ranks (uuids, scores,<br/>variances, labels)."]
class kirad_metrics_py__MultiCatMetric opt
click kirad_metrics_py__MultiCatMetric "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/metrics.py#L41" _blank
kirad_metrics_py__IndetSnSpArray_build["<b>39 · IndetSnSpArray.build()</b><br/>Sn/Sp/indet_frac for all threshold<br/>pairs (or the given ones) via<br/>histogram cumsums."]
class kirad_metrics_py__IndetSnSpArray_build main
click kirad_metrics_py__IndetSnSpArray_build "../walkthrough/#step-39"
kirad_metrics_py__IndetSnSpArray_roc_curve["<b>IndetSnSpArray.roc_curve()</b><br/>Pareto ROC curve of (Sn, Sp) using<br/>only pairs with indet_frac ‹=<br/>indet_budget."]
class kirad_metrics_py__IndetSnSpArray_roc_curve opt
click kirad_metrics_py__IndetSnSpArray_roc_curve "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/metrics.py#L291" _blank
kirad_metrics_py__IndetSnSpArray_sn_eq_sp_graph["<b>IndetSnSpArray.sn_eq_sp_graph()</b><br/>Best min(Sn, Sp) per indeterminate<br/>fraction; these thresholds are<br/>saved in the ckpt."]
class kirad_metrics_py__IndetSnSpArray_sn_eq_sp_graph opt
click kirad_metrics_py__IndetSnSpArray_sn_eq_sp_graph "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/metrics.py#L305" _blank
end
from_prev(["◀ from step 35: CORALLoss.forward()"])
class from_prev edge
from_prev ==> kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_on_validation_test_epoch_end
click from_prev "#stage-loss"
kirad_base_models_py__MultiTaskOrdinalRegressionPLModule_on_validation_test_epoch_end ==> kirad_base_models_py__OrdinalRegressionPLModule_on_validation_test_epoch_end
kirad_base_models_py__OrdinalRegressionPLModule_on_validation_test_epoch_end ==> kirad_base_models_py__OrdinalRegressionPLModule_quantize_labels
kirad_base_models_py__OrdinalRegressionPLModule_quantize_labels ==> kirad_metrics_py__IndetSnSpArray_build
kirad_metrics_py__IndetSnSpArray_build ==> kirad_ordinal_thresholding_py__OptimalOrdinalThresholdingViaDynamicProgramming_tune_thresholds
kirad_ordinal_thresholding_py__OptimalOrdinalThresholdingViaDynamicProgramming_tune_thresholds ==> kirad_base_models_py__OrdinalRegressionPLModule_log_ordinal_metrics
to_next(["▶ next, step 42: DistributedModelCheckpoint.on_validation_end()"])
class to_next edge
kirad_base_models_py__OrdinalRegressionPLModule_log_ordinal_metrics ==> to_next
click to_next "#stage-outputs"
kirad_metrics_py__MultiCatMetric -.->|"gathered scores"| kirad_base_models_py__OrdinalRegressionPLModule_on_validation_test_epoch_end
kirad_base_models_py__OrdinalRegressionPLModule_on_validation_test_epoch_end -.->|"val only"| kirad_ordinal_thresholding_py__OptimalOrdinalThresholdingViaDynamicProgramming_tune_thresholds
kirad_ordinal_thresholding_py__OptimalOrdinalThresholdingViaDynamicProgramming_tune_thresholds -.-> kirad_ordinal_thresholding_py__OptimalCostPerSampleOrdinalThresholding_dp_step
kirad_ordinal_thresholding_py__OptimalOrdinalThresholdingViaDynamicProgramming_tune_thresholds -.-> kirad_ordinal_thresholding_py__OptimalCostPerClassOrdinalThresholding_dp_step
kirad_base_models_py__OrdinalRegressionPLModule_on_validation_test_epoch_end -.-> kirad_ordinal_thresholding_py__CORALThresholding_tune_thresholds
kirad_base_models_py__OrdinalRegressionPLModule_on_validation_test_epoch_end -.-> kirad_metrics_py__IndetSnSpArray_build
kirad_base_models_py__OrdinalRegressionPLModule_on_validation_test_epoch_end -.->|"Sn=Sp ops (val)"| kirad_metrics_py__IndetSnSpArray_sn_eq_sp_graph
kirad_base_models_py__OrdinalRegressionPLModule_on_validation_test_epoch_end -.-> kirad_base_models_py__OrdinalRegressionPLModule_log_ordinal_metrics
kirad_base_models_py__OrdinalRegressionPLModule_log_ordinal_metrics -.-> kirad_metrics_py__IndetSnSpArray_roc_curve
kirad_base_models_py__OrdinalRegressionPLModule_on_validation_test_epoch_end -.-> kirad_base_models_py__OrdinalRegressionPLModule_log_thresholding_metrics
classDef main fill:#FFF4DC,stroke:#A67C1B,stroke-width:2px,color:#1A1A1A
classDef opt fill:#F3F4F6,stroke:#B8BEC7,color:#555,stroke-dasharray:4 3
classDef edge fill:#FFFFFF,stroke:#1F7A5C,color:#1F7A5C,stroke-width:1.5px
| Step | Function | File | What it does |
|---|---|---|---|
| 36 | MultiTaskOrdinalRegressionPLModule.on_validation_test_epoch_end |
kirad/base_models.py:680 |
Run each task's epoch end, then log harmonic mean of shared metrics across tasks. |
| 37 | OrdinalRegressionPLModule.on_validation_test_epoch_end |
kirad/base_models.py:423 |
Gather scores, log W&B table, tune thresholds on val, compute Sn/Sp metrics. |
| 38 | OrdinalRegressionPLModule.quantize_labels |
kirad/base_models.py:222 |
Map input labels (e.g. PHQ-9 sum) to output labels (e.g. 0/1/2 for L/M/H). |
| 41 | OrdinalRegressionPLModule.log_ordinal_metrics |
kirad/base_models.py:228 |
Log AUROC and min(Sn, Sp) per binary cut and indet budget, plus their mean. |
| — | OrdinalRegressionPLModule.log_thresholding_metrics |
kirad/base_models.py:315 |
Log each tuned method's cost when scored with every method's thresholds. |
| 40 | OptimalOrdinalThresholdingViaDynamicProgramming.tune_thresholds |
kirad/ordinal_thresholding.py:179 |
Set thresholds by DP over midpoints of unique scores (+/-inf); return the best metric. |
| — | OptimalCostPerSampleOrdinalThresholding.dp_step |
kirad/ordinal_thresholding.py:291 |
O(n) DP step: histogram per-sample costs over candidates, cumsum, running max. |
| — | OptimalCostPerClassOrdinalThresholding.dp_step |
kirad/ordinal_thresholding.py:450 |
O(T^2) DP step: add prev_cost to each lower-threshold row and max over it. |
| — | CORALThresholding.tune_thresholds |
kirad/ordinal_thresholding.py:85 |
Copy -biases[threshold_indices] from the CORAL loss; scores and labels are ignored. |
| — | MultiCatMetric |
kirad/metrics.py:41 |
Accumulate parallel tensors over batches and ranks (uuids, scores, variances, labels). |
| 39 | IndetSnSpArray.build |
kirad/metrics.py:176 |
Sn/Sp/indet_frac for all threshold pairs (or the given ones) via histogram cumsums. |
| — | IndetSnSpArray.roc_curve |
kirad/metrics.py:291 |
Pareto ROC curve of (Sn, Sp) using only pairs with indet_frac <= indet_budget. |
| — | IndetSnSpArray.sn_eq_sp_graph |
kirad/metrics.py:305 |
Best min(Sn, Sp) per indeterminate fraction; these thresholds are saved in the ckpt. |