tacit/nanochat
1
0
Fork 0
mirror of https://github.com/karpathy/nanochat.git synced 2026-07-06 04:49:14 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

fix typo in scripts/chat_rl.py

Browse Source 
typo in comments: change "GAPO" to "DAPO"
This commit is contained in:
Haoyu Wang 2026-01-04 17:56:01 -05:00 committed by GitHub
parent 962b6bfba3
commit db5e62fc2a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
scripts/chat_rl.py
View File

@ -6,7 +6,7 @@ simpler and more similar to just REINFORCE:
1) Delete trust region, so there is no KL regularization to a reference model 1) Delete trust region, so there is no KL regularization to a reference model
2) We are on policy, so there's no need for PPO ratio+clip. 2) We are on policy, so there's no need for PPO ratio+clip.
3) We use GAPO style normalization that is token-level, not sequence-level. 3) We use DAPO style normalization that is token-level, not sequence-level.
4) Instead of z-score normalization (r - mu)/sigma, only use (r - mu) as the advantage. 4) Instead of z-score normalization (r - mu)/sigma, only use (r - mu) as the advantage.
1 GPU: 1 GPU: