Data contract

What the code expects on disk and what reaches the model. All defaults below are read from kirad/datasets_and_dataloaders.py and kirad/constants.py.

Metadata CSV columns

One CSV per split: with a CSV path, dataset_splits, data_sources and dataset_fold are ignored (load_dataset), so set df_or_dataset_version in the train: and val_test: (or val:/test:) sections of the config.

Column	Needed for	Used by	Notes
uuid	always	row index, sampler, W&B score tables	unique per recording
filename	always	load_relative_dataset	→ <audio_root>/<filename>.wav, <text_root>/<filename>.json
phq	depression task	label_column.depression	PHQ-9 total, 0–27 (raw score, not a class)
gad	anxiety task	label_column.anxiety	GAD-7 total, 0–21
duration column	always	num_windows, min/max filter	seconds after VAD; name set by duration_column (configs: prompt_vad_trimmed_duration train, prompt_vad_cut_duration val/test)
asr_output_id	text runs: DAM 2 (DAM 3 only indirectly, via the cached DAM 2 embeddings; config_llma_dam3.0.yaml reads text directly)	KintsugiTextDataset.__getitem__	must exist; value is not used for the path
user (+ demographics)	merge_rows_by_subject only	merge_df_rows_by_subject	rows merge when all present Kintsugi columns match
scores_<task>	knowledge distillation only	KintsugiMetadata.__getitem__ → label.<task>.kd	from score_targets_csv (uuid, scores_depression, scores_anxiety); teacher-model scores, not human ratings

File layout

<audio_root>/<filename>.wav    16 kHz (not resampled: only a warning), channel 0 used, VAD already applied
<text_root>/<filename>.json    whisper_timestamped output; only its "text" field is read
train.csv  val.csv  test.csv   metadata above (paths relative to the config folder or cwd)

audio_root / text_root are KintsugiMetadata params (default: current directory). VAD is not in this repo; recordings shorter than min_duration (30 s) are dropped.

Windows (30 s)

Quantity	Value	Where
Window length	30 s = 480,000 samples at 16 kHz	INFERENCE_WINDOW_SIZE, EXPECTED_SAMPLE_RATE
Encoder positions per window	1500 × 320 samples	mar_2024/model.py (self.padding)
Log-mel per window	80 bins × 3000 frames (hop 160, n_fft 400)	WhisperTorchFeatureExtractor
Windows per recording (all, overlap 0)	⌊duration / 30⌋, spread from start to end (leftover audio falls between windows)	get_window_starts
… with pad_last_chunk_to_full: true	⌈duration / 30⌉ (last one zero-padded)	get_preprocessor_with_audio_normalization

window_method: all (every window; val/test), single_random (1 random), single_start (first 30 s), random_adjacent_pair (2 consecutive; DAM 1 train, feeds the variance loss).

Batch

broadcast_collate concatenates the windows of all B recordings; length[i] = windows of recording i, so ΣW = length.sum().

Key	Shape	Content
uuid	[B]	recording ids
features.audio	[ΣW, 480000]	raw waveform (preprocessor: false, DAM 1 current config)
features.audio	[ΣW, 80, 3000]	HF log-mel (preprocessor: openai/whisper-small.en, DAM 2/3)
features.text.input_ids, .attention_mask	[ΣW, L]	tokens, tiled to every window; L = 512 for BERT
features.backbone_cache	[ΣW, D]	cached embeddings, tiled like the other modalities (DAM 3 phase 1 target)
label.<task>.primary	[B]	raw PHQ / GAD score
label.<task>.kd	[B]	teacher score (only with score_targets_csv)
length	[B]	windows per recording

batch_size in the config counts recordings, not windows.

Data parameters and defaults

KintsugiMetadata (config key metadata=kirad.datasets_and_dataloaders.KintsugiMetadata):

Param	Default	Notes
df_or_dataset_version	required	CSV path, DataFrame, {metadata, audio_root, text_root} or a private Kintsugi version name
label_column	{depression: phq, anxiety: gad}	task → CSV column
duration_column	None	effectively required (used for num_windows)
min_duration / max_duration	30 / None s	inclusive filters
window_duration	30 s	keep equal to the audio dataset's
max_overlap_frac	0.0	keep equal to the audio dataset's
fraction	1.0	subsample, random_state=1
merge_rows_by_subject	False	many_to_one / many_to_many concatenate a subject's audio
score_targets_csv	None	KD targets
audio_root / text_root	None (cwd)

KintsugiAudioDataset (audio=...):

Param	Default	Current configs
preprocessor	openai/whisper-small.en	DAM 1: false (log-mel on GPU in the model)
normalize_audio	False	true (DC removal + peak scale to [-1, 1])
normalize_features	False	true in DAM 2/3
window_method	all	train random_adjacent_pair (DAM 1) / single_random (DAM 2)
window_duration, max_overlap_frac	30, 0.0
ideal_logmel_energies	None	DAM 2/3: logmel_energies.npy (not in repo)
pad_last_chunk_to_full	False	true in DAM 3
augmentations	()	[{class_name, params, aug_prob}], e.g. RawBoost

KintsugiTextDataset (text=...): preprocessor google-bert/bert-base-uncased, text_output_per_sample merge, prompt None (LLaMA only), max_tokens None (tokenizer max). KintsugiCacheDataset (backbone_cache=...): backbone_cache required, feat_start_idx 0, feat_end_idx None.

Per-split merge order: default < val_test (val and test) < train/val/test (LauncherConfig.get_params_by_split).

2 · Data

The metadata CSV says which WAV files to open and what their labels are. Audio is cut into 30 s windows and turned into features; windows of all recordings in a batch are concatenated and length remembers where each recording ends.

Main path: steps 15–24 of a train run (walkthrough). Gold boxes open the step; grey boxes are side paths and open the code.

flowchart LR subgraph kirad_datasets_and_dataloaders_py___box["datasets_and_dataloaders.py"] direction TB kirad_datasets_and_dataloaders_py__KintsugiMetadata["<b>15 · KintsugiMetadata()</b><br/>Metadata table indexed by UUID:<br/>loads CSV, merges KD scores,<br/>filters by duration."] class kirad_datasets_and_dataloaders_py__KintsugiMetadata main click kirad_datasets_and_dataloaders_py__KintsugiMetadata "../walkthrough/#step-15" kirad_datasets_and_dataloaders_py__KintsugiMetadata_preprocess_metadata["<b>17 · KintsugiMetadata.preprocess_metadata()</b><br/>Join KD score targets, optionally<br/>merge rows per subject, count<br/>windows per recording."] class kirad_datasets_and_dataloaders_py__KintsugiMetadata_preprocess_metadata main click kirad_datasets_and_dataloaders_py__KintsugiMetadata_preprocess_metadata "../walkthrough/#step-17" kirad_datasets_and_dataloaders_py__KintsugiMetadata_filter_metadata["<b>18 · KintsugiMetadata.filter_metadata()</b><br/>Subsample `fraction` of rows, then<br/>drop recordings outside min/max<br/>duration (s)."] class kirad_datasets_and_dataloaders_py__KintsugiMetadata_filter_metadata main click kirad_datasets_and_dataloaders_py__KintsugiMetadata_filter_metadata "../walkthrough/#step-18" kirad_datasets_and_dataloaders_py__KintsugiMetadata___getitem__["<b>21 · KintsugiMetadata.__getitem__()</b><br/>Return {uuid, label: {task:<br/>{primary: raw score, kd: teacher<br/>score}}} for one UUID."] class kirad_datasets_and_dataloaders_py__KintsugiMetadata___getitem__ main click kirad_datasets_and_dataloaders_py__KintsugiMetadata___getitem__ "../walkthrough/#step-21" kirad_datasets_and_dataloaders_py__KintsugiAudioDataset___getitem__["<b>22 · KintsugiAudioDataset.__getitem__()</b><br/>Load, concatenate (channel 0),<br/>augment and window one recording's<br/>audio."] class kirad_datasets_and_dataloaders_py__KintsugiAudioDataset___getitem__ main click kirad_datasets_and_dataloaders_py__KintsugiAudioDataset___getitem__ "../walkthrough/#step-22" kirad_datasets_and_dataloaders_py__get_preprocessor_with_audio_normalization["<b>19 · get_preprocessor_with_audio_normalization()</b><br/>Build the function audio [T] -›<br/>windows -› features used by<br/>KintsugiAudioDataset."] class kirad_datasets_and_dataloaders_py__get_preprocessor_with_audio_normalization main click kirad_datasets_and_dataloaders_py__get_preprocessor_with_audio_normalization "../walkthrough/#step-19" kirad_datasets_and_dataloaders_py__get_window_starts["<b>23 · get_window_starts()</b><br/>Start sample(s) of the windows cut<br/>from one recording, by<br/>window_method."] class kirad_datasets_and_dataloaders_py__get_window_starts main click kirad_datasets_and_dataloaders_py__get_window_starts "../walkthrough/#step-23" kirad_datasets_and_dataloaders_py__KintsugiTextDataset___getitem__["<b>KintsugiTextDataset.__getitem__()</b><br/>Read the transcript JSON(s) of one<br/>recording and tokenize them."] class kirad_datasets_and_dataloaders_py__KintsugiTextDataset___getitem__ opt click kirad_datasets_and_dataloaders_py__KintsugiTextDataset___getitem__ "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/datasets_and_dataloaders.py#L572" _blank kirad_datasets_and_dataloaders_py__KintsugiCacheDataset["<b>KintsugiCacheDataset()</b><br/>Precomputed backbone features by<br/>UUID (DAM 3 teacher cache),<br/>optional dim slice."] class kirad_datasets_and_dataloaders_py__KintsugiCacheDataset opt click kirad_datasets_and_dataloaders_py__KintsugiCacheDataset "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/datasets_and_dataloaders.py#L628" _blank kirad_datasets_and_dataloaders_py__UuidSampler["<b>20 · UuidSampler()</b><br/>Yields metadata UUIDs (shuffled<br/>for train, sequential for<br/>val/test) for the loader."] class kirad_datasets_and_dataloaders_py__UuidSampler main click kirad_datasets_and_dataloaders_py__UuidSampler "../walkthrough/#step-20" end subgraph kirad_dataset_utils_py___box["dataset_utils.py"] direction TB kirad_dataset_utils_py__load_relative_dataset["<b>16 · load_relative_dataset()</b><br/>Load metadata and add<br/>audio_filename / text_filename<br/>paths from the filename column."] class kirad_dataset_utils_py__load_relative_dataset main click kirad_dataset_utils_py__load_relative_dataset "../walkthrough/#step-16" kirad_dataset_utils_py__load_dataset["<b>load_dataset()</b><br/>Return metadata as a DataFrame:<br/>pass-through, read a CSV path, or<br/>a Kintsugi version."] class kirad_dataset_utils_py__load_dataset opt click kirad_dataset_utils_py__load_dataset "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/dataset_utils.py#L49" _blank kirad_dataset_utils_py__merge_df_rows_by_subject["<b>merge_df_rows_by_subject()</b><br/>Merge rows sharing user + labels +<br/>demographics into one row per<br/>subject."] class kirad_dataset_utils_py__merge_df_rows_by_subject opt click kirad_dataset_utils_py__merge_df_rows_by_subject "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/dataset_utils.py#L651" _blank kirad_dataset_utils_py__broadcast_collate["<b>24 · broadcast_collate()</b><br/>DataLoader collate: concat windows<br/>of all recordings; length =<br/>windows per recording."] class kirad_dataset_utils_py__broadcast_collate main click kirad_dataset_utils_py__broadcast_collate "../walkthrough/#step-24" end subgraph kirad_augmentations_py___box["augmentations.py"] direction TB kirad_augmentations_py__RawBoost["<b>RawBoost()</b><br/>RawBoost waveform augmentation<br/>(LnL/ISD/SSI noise); default algo<br/>4 = all three."] class kirad_augmentations_py__RawBoost opt click kirad_augmentations_py__RawBoost "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/augmentations.py#L166" _blank end subgraph kirad_constants_py___box["constants.py"] direction TB kirad_constants_py__DatasetFields["<b>DatasetFields()</b><br/>Key names of a batch dict: uuid,<br/>features, label.‹task›.primary/kd,<br/>length, loss."] class kirad_constants_py__DatasetFields opt click kirad_constants_py__DatasetFields "https://github.com/Pipotheavenger/kintsugi-rad-docs/blob/main/kirad/constants.py#L154" _blank end from_prev(["◀ from step 14: LauncherConfig.get_dataset()"]) class from_prev edge from_prev ==> kirad_datasets_and_dataloaders_py__KintsugiMetadata click from_prev "#stage-launch" kirad_datasets_and_dataloaders_py__KintsugiMetadata ==> kirad_dataset_utils_py__load_relative_dataset kirad_dataset_utils_py__load_relative_dataset ==> kirad_datasets_and_dataloaders_py__KintsugiMetadata_preprocess_metadata kirad_datasets_and_dataloaders_py__KintsugiMetadata_preprocess_metadata ==> kirad_datasets_and_dataloaders_py__KintsugiMetadata_filter_metadata kirad_datasets_and_dataloaders_py__KintsugiMetadata_filter_metadata ==> kirad_datasets_and_dataloaders_py__get_preprocessor_with_audio_normalization kirad_datasets_and_dataloaders_py__get_preprocessor_with_audio_normalization ==> kirad_datasets_and_dataloaders_py__UuidSampler kirad_datasets_and_dataloaders_py__UuidSampler ==> kirad_datasets_and_dataloaders_py__KintsugiMetadata___getitem__ kirad_datasets_and_dataloaders_py__KintsugiMetadata___getitem__ ==> kirad_datasets_and_dataloaders_py__KintsugiAudioDataset___getitem__ kirad_datasets_and_dataloaders_py__KintsugiAudioDataset___getitem__ ==> kirad_datasets_and_dataloaders_py__get_window_starts kirad_datasets_and_dataloaders_py__get_window_starts ==> kirad_dataset_utils_py__broadcast_collate to_next(["▶ next, step 25: MultiTaskOrdinalRegressionPLModule.training_step()"]) class to_next edge kirad_dataset_utils_py__broadcast_collate ==> to_next click to_next "#stage-model" kirad_dataset_utils_py__load_relative_dataset -.-> kirad_dataset_utils_py__load_dataset kirad_datasets_and_dataloaders_py__KintsugiMetadata -.-> kirad_datasets_and_dataloaders_py__KintsugiMetadata_preprocess_metadata kirad_datasets_and_dataloaders_py__KintsugiMetadata_preprocess_metadata -.->|"train only"| kirad_dataset_utils_py__merge_df_rows_by_subject kirad_datasets_and_dataloaders_py__KintsugiMetadata_preprocess_metadata -.->|"count windows"| kirad_datasets_and_dataloaders_py__get_window_starts kirad_datasets_and_dataloaders_py__KintsugiMetadata -.-> kirad_datasets_and_dataloaders_py__KintsugiMetadata_filter_metadata kirad_datasets_and_dataloaders_py__UuidSampler -.->|"uuid"| kirad_datasets_and_dataloaders_py__KintsugiAudioDataset___getitem__ kirad_datasets_and_dataloaders_py__KintsugiAudioDataset___getitem__ -.->|"optional"| kirad_augmentations_py__RawBoost kirad_datasets_and_dataloaders_py__KintsugiAudioDataset___getitem__ -.-> kirad_datasets_and_dataloaders_py__get_preprocessor_with_audio_normalization kirad_datasets_and_dataloaders_py__get_preprocessor_with_audio_normalization -.->|"cut"| kirad_datasets_and_dataloaders_py__get_window_starts kirad_datasets_and_dataloaders_py__KintsugiAudioDataset___getitem__ -.-> kirad_dataset_utils_py__broadcast_collate kirad_datasets_and_dataloaders_py__KintsugiTextDataset___getitem__ -.-> kirad_dataset_utils_py__broadcast_collate kirad_datasets_and_dataloaders_py__KintsugiMetadata___getitem__ -.->|"labels"| kirad_dataset_utils_py__broadcast_collate kirad_dataset_utils_py__broadcast_collate -.->|"batch keys"| kirad_constants_py__DatasetFields classDef main fill:#FFF4DC,stroke:#A67C1B,stroke-width:2px,color:#1A1A1A classDef opt fill:#F3F4F6,stroke:#B8BEC7,color:#555,stroke-dasharray:4 3 classDef edge fill:#FFFFFF,stroke:#1F7A5C,color:#1F7A5C,stroke-width:1.5px

Step	Function	File	What it does
15	KintsugiMetadata	kirad/datasets_and_dataloaders.py:71	Metadata table indexed by UUID: loads CSV, merges KD scores, filters by duration.
17	KintsugiMetadata.preprocess_metadata	kirad/datasets_and_dataloaders.py:210	Join KD score targets, optionally merge rows per subject, count windows per recording.
18	KintsugiMetadata.filter_metadata	kirad/datasets_and_dataloaders.py:257	Subsample fraction of rows, then drop recordings outside min/max duration (s).
21	KintsugiMetadata.__getitem__	kirad/datasets_and_dataloaders.py:293	Return {uuid, label: {task: {primary: raw score, kd: teacher score}}} for one UUID.
22	KintsugiAudioDataset.__getitem__	kirad/datasets_and_dataloaders.py:439	Load, concatenate (channel 0), augment and window one recording's audio.
19	get_preprocessor_with_audio_normalization	kirad/datasets_and_dataloaders.py:790	Build the function audio [T] -> windows -> features used by KintsugiAudioDataset.
23	get_window_starts	kirad/datasets_and_dataloaders.py:738	Start sample(s) of the windows cut from one recording, by window_method.
—	KintsugiTextDataset.__getitem__	kirad/datasets_and_dataloaders.py:572	Read the transcript JSON(s) of one recording and tokenize them.
—	KintsugiCacheDataset	kirad/datasets_and_dataloaders.py:628	Precomputed backbone features by UUID (DAM 3 teacher cache), optional dim slice.
20	UuidSampler	kirad/datasets_and_dataloaders.py:703	Yields metadata UUIDs (shuffled for train, sequential for val/test) for the loader.
16	load_relative_dataset	kirad/dataset_utils.py:97	Load metadata and add audio_filename / text_filename paths from the filename column.
—	load_dataset	kirad/dataset_utils.py:49	Return metadata as a DataFrame: pass-through, read a CSV path, or a Kintsugi version.
—	merge_df_rows_by_subject	kirad/dataset_utils.py:651	Merge rows sharing user + labels + demographics into one row per subject.
24	broadcast_collate	kirad/dataset_utils.py:830	DataLoader collate: concat windows of all recordings; length = windows per recording.
—	RawBoost	kirad/augmentations.py:166	RawBoost waveform augmentation (LnL/ISD/SSI noise); default algo 4 = all three.
—	DatasetFields	kirad/constants.py:154	Key names of a batch dict: uuid, features, label.<task>.primary/kd, length, loss.