tacit/nanochat
1
0
Fork 0
mirror of https://github.com/karpathy/nanochat.git synced 2026-01-27 05:44:14 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

remove batch_num_tokens definition which was only used for experiment logging

Browse Source
This commit is contained in:
svlandeg 2025-12-30 11:32:47 +01:00
parent 1ca9280328
commit 32ce342c88
1 changed files with 0 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
scripts/chat_sft.py
View File

@ -219,7 +219,6 @@ for step in range(num_iterations):
dist.all_reduce(num_tokens, op=dist.ReduceOp.SUM) # sum over ranks
for micro_step, (train_inputs, train_targets) in enumerate(steps):
batch_num_tokens = (train_targets >= 0).sum()
with autocast_ctx:
loss = model(train_inputs, train_targets, loss_reduction='sum')
loss = loss / num_tokens # normalize loss here