The single download path in nanochat/common.py uses urllib.request,
which on macOS python.org installs (and some other Python distributions)
relies on Python's bundled CA cert store. That store is empty until the
user runs the post-install "Install Certificates.command" script, so
HTTPS downloads fail with `SSL: CERTIFICATE_VERIFY_FAILED` against
karpathy-public.s3.us-west-2.amazonaws.com (and any HTTPS endpoint).
This breaks runs/runcpu.sh for fresh macOS Python installs:
- scripts.base_eval can't fetch eval_bundle.zip
- scripts.chat_sft can't fetch identity_conversations.jsonl
requests uses certifi by default, which ships its own CA bundle, so
this works on every supported platform without any per-machine cert
configuration. requests is already a transitive dep used in
nanochat/dataset.py, so this introduces no new dependency.
raise_for_status() is added so HTTP errors fail loudly instead of
silently writing the response body of an error page. A 60-second
timeout is added; the previous urlopen call had no timeout.
When swapping Float8Linear to Linear in disable_fp8 context manager,
using device=fp8_module.weight.device directly allocates new tensors
on GPU, causing unnecessary VRAM spike (~1GB for large models).
This fix uses device='meta' to avoid physical memory allocation,
then swaps in the weight tensor reference. This eliminates the
unnecessary VRAM spike during evaluation phase.
Fixes issue #592
Co-authored-by: RoomWithOutRoof <roomwithoutroof@sparklab.ai>
The bf16 cast is intentional for speed on Hopper+ GPUs, but should be
skipped on other platforms rather than blindly applied. fp16 is unstable
here due to its limited exponent range, and fp32 platforms don't benefit
from the cast. Now: bf16 when COMPUTE_DTYPE is bf16, no cast otherwise.
Inspired by PR #667.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New architectural features:
- Smear: mix previous token embedding into current position via learned
gate, providing cheap bigram-like info (works in training + KV cache)
- Backout: subtract learned fraction of mid-layer residual before logit
projection to remove low-level features
Hyperparameter tuning:
- Muon momentum warmdown 0.97→0.90 during LR warmdown phase
- Non-uniform per-layer init: resid_lambdas 1.15→1.05, x0_lambdas 0.20→0.05
- c_fc init scale 0.4x, QK norm scale 1.2, sliding window seq_len/4
- Speedrun data:params ratio reduced to 8
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* printing steps count
* adding reply only loss for chat
* using the mask by render_conversation function of tokeniser
* undoing some changes
* putting back the comment which got removed accidently, no functionality change
