tacit/nanochat
1
0
Fork 0
mirror of https://github.com/karpathy/nanochat.git synced 2025-12-06 04:12:13 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

fix: return inf instead of crashing when evaluate_bpb has zero total_bytes

Browse Source 
Edge case: all tokens are special tokens or ignored

Return infinity to indicate no meaningful bytes were processed
This commit is contained in:
Bhaskar 2025-10-14 17:21:11 +05:30
parent dd6ff9a1cc
commit 02440f670d
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
nanochat/loss_eval.py
View File

@ -59,5 +59,7 @@ def evaluate_bpb(model, batches, steps, token_bytes):
# move both to cpu, calculate bpb and return # move both to cpu, calculate bpb and return
total_nats = total_nats.item() total_nats = total_nats.item()
total_bytes = total_bytes.item() total_bytes = total_bytes.item()
if total_bytes == 0:
return float('inf')
bpb = total_nats / (math.log(2) * total_bytes) bpb = total_nats / (math.log(2) * total_bytes)
return bpb return bpb