2 Commits

Author	SHA1	Message	Date
William Thurston	8a6d34daf7	Add kv_head_mult parameter for training and evaluation scripts ... - Introduced `kv_head_mult` to control the number of query heads sharing a key/value head in `base_train.py`, `mid_train.py`, and `runmps.sh`. - Updated logging to include global token per second metrics during training. - Added assertions to ensure `kv_head_mult` is valid and properly integrated into model calculations.	2025-11-09 14:23:45 -08:00
William Thurston	b1d49aade5	Add scripts for running evaluations and training with W&B integration ... - Added `dev/runmps_evals.sh` for evaluating checkpoints and logging results to W&B. - Introduced `dev/runmps.sh` for orchestrating training stages with W&B support. - Updated `.gitignore` to include `wandb/` and `.runmps_wandb_ids`. - Changed permissions for `dev/runcpu.sh` and added executable flag. - Enhanced existing scripts to log metrics to W&B during training and evaluation processes.	2025-11-05 11:49:50 -08:00