tacit

tacit synced commits to refs/pull/399/merge at tacit/nanochat from mirror 2026-01-09 05:02:39 +00:00

8b8249b11d Merge de50af6283 into f5a0ea4d3f

f5a0ea4d3f take out these gitignore dirs

4ddc803797 fix adamw slight bug. this chunk was copy pasted originally from modded-nanogpt, which still seems to have the bug

a1ccb3dc0b remove rust compilation as rustbpe is now installed from separate package (#416)

061f83c152 delete grad_clip. appears to not be necessary at all. not only was it buggy because the clipping happened per gpu before grad synchronization, but it costs ~2% MFU, and it also doesn't even help. I tried deleting it a while ago and back then it did help. So I'm guessing that some hyperparameter tuning obviated the reason for it since then

Compare 10 commits »

tacit synced commits to refs/pull/405/merge at tacit/nanochat from mirror 2026-01-09 05:02:39 +00:00

1a0c2f1a3c Merge 3c4be194e9 into f5a0ea4d3f

f5a0ea4d3f take out these gitignore dirs

4ddc803797 fix adamw slight bug. this chunk was copy pasted originally from modded-nanogpt, which still seems to have the bug

Compare 3 commits »

tacit synced commits to refs/pull/407/merge at tacit/nanochat from mirror 2026-01-09 05:02:39 +00:00

b7c83c8bba Merge 47885e743b into f5a0ea4d3f

f5a0ea4d3f take out these gitignore dirs

4ddc803797 fix adamw slight bug. this chunk was copy pasted originally from modded-nanogpt, which still seems to have the bug

a1ccb3dc0b remove rust compilation as rustbpe is now installed from separate package (#416)

Compare 4 commits »

tacit synced commits to refs/pull/414/merge at tacit/nanochat from mirror 2026-01-08 20:52:41 +00:00

ba7c2ef26c Merge 57bcf6786e into f5a0ea4d3f

f5a0ea4d3f take out these gitignore dirs

4ddc803797 fix adamw slight bug. this chunk was copy pasted originally from modded-nanogpt, which still seems to have the bug

a1ccb3dc0b remove rust compilation as rustbpe is now installed from separate package (#416)

061f83c152 delete grad_clip. appears to not be necessary at all. not only was it buggy because the clipping happened per gpu before grad synchronization, but it costs ~2% MFU, and it also doesn't even help. I tried deleting it a while ago and back then it did help. So I'm guessing that some hyperparameter tuning obviated the reason for it since then

Compare 5 commits »

tacit synced commits to refs/pull/85/merge at tacit/nanochat from mirror 2026-01-08 20:52:41 +00:00

e29067e3c7 Merge 04862cbfea into a1ccb3dc0b

a1ccb3dc0b remove rust compilation as rustbpe is now installed from separate package (#416)

Compare 2 commits »

tacit synced commits to refs/pull/412/merge at tacit/nanochat from mirror 2026-01-08 20:52:40 +00:00

846d281a4a Merge db5e62fc2a into f5a0ea4d3f

f5a0ea4d3f take out these gitignore dirs

4ddc803797 fix adamw slight bug. this chunk was copy pasted originally from modded-nanogpt, which still seems to have the bug

a1ccb3dc0b remove rust compilation as rustbpe is now installed from separate package (#416)

Compare 4 commits »

tacit synced commits to refs/pull/409/merge at tacit/nanochat from mirror 2026-01-08 20:52:40 +00:00

87fc4d47d9 Merge 489075bdbd into f5a0ea4d3f

f5a0ea4d3f take out these gitignore dirs

4ddc803797 fix adamw slight bug. this chunk was copy pasted originally from modded-nanogpt, which still seems to have the bug

a1ccb3dc0b remove rust compilation as rustbpe is now installed from separate package (#416)

Compare 4 commits »

tacit synced commits to refs/pull/400/merge at tacit/nanochat from mirror 2026-01-08 20:52:39 +00:00

fa2dafef9d Merge 32ce342c88 into 061f83c152

061f83c152 delete grad_clip. appears to not be necessary at all. not only was it buggy because the clipping happened per gpu before grad synchronization, but it costs ~2% MFU, and it also doesn't even help. I tried deleting it a while ago and back then it did help. So I'm guessing that some hyperparameter tuning obviated the reason for it since then

e8c30c3b19 add notebook used for scaling laws analysis

3af4dcf6ee also add scaling_laws.sh script if it's a useful reference

4cc605b940 quick pointer to miniseries post in readme for now

Compare 6 commits »

tacit synced commits to refs/pull/405/merge at tacit/nanochat from mirror 2026-01-08 20:52:39 +00:00

b5d5797c4e Merge 3c4be194e9 into a1ccb3dc0b

a1ccb3dc0b remove rust compilation as rustbpe is now installed from separate package (#416)

Compare 2 commits »

tacit synced commits to refs/pull/204/merge at tacit/nanochat from mirror 2026-01-08 20:52:38 +00:00

24e4615836 Merge 69b7cc9ac5 into a1ccb3dc0b

a1ccb3dc0b remove rust compilation as rustbpe is now installed from separate package (#416)

061f83c152 delete grad_clip. appears to not be necessary at all. not only was it buggy because the clipping happened per gpu before grad synchronization, but it costs ~2% MFU, and it also doesn't even help. I tried deleting it a while ago and back then it did help. So I'm guessing that some hyperparameter tuning obviated the reason for it since then

e8c30c3b19 add notebook used for scaling laws analysis

3af4dcf6ee also add scaling_laws.sh script if it's a useful reference

Compare 25 commits »

tacit synced commits to refs/pull/324/merge at tacit/nanochat from mirror 2026-01-08 20:52:38 +00:00

1fc84d4647 Merge e00c73322c into f5a0ea4d3f

f5a0ea4d3f take out these gitignore dirs

4ddc803797 fix adamw slight bug. this chunk was copy pasted originally from modded-nanogpt, which still seems to have the bug

a1ccb3dc0b remove rust compilation as rustbpe is now installed from separate package (#416)

Compare 4 commits »

tacit synced commits to refs/pull/370/merge at tacit/nanochat from mirror 2026-01-08 20:52:38 +00:00

61bf5525dc Merge 9a9b12b1be into f5a0ea4d3f

f5a0ea4d3f take out these gitignore dirs

4ddc803797 fix adamw slight bug. this chunk was copy pasted originally from modded-nanogpt, which still seems to have the bug

a1ccb3dc0b remove rust compilation as rustbpe is now installed from separate package (#416)

Compare 4 commits »

tacit synced commits to master at tacit/nanochat from mirror 2026-01-08 20:52:37 +00:00

f5a0ea4d3f take out these gitignore dirs

4ddc803797 fix adamw slight bug. this chunk was copy pasted originally from modded-nanogpt, which still seems to have the bug

a1ccb3dc0b remove rust compilation as rustbpe is now installed from separate package (#416)

Compare 3 commits »

tacit synced and deleted reference refs/tags/refs/pull/416/merge at tacit/nanochat from mirror 2026-01-08 20:52:37 +00:00

tacit synced commits to refs/pull/85/merge at tacit/nanochat from mirror 2026-01-08 12:42:51 +00:00

0fc5ac7742 Merge 04862cbfea into 061f83c152

061f83c152 delete grad_clip. appears to not be necessary at all. not only was it buggy because the clipping happened per gpu before grad synchronization, but it costs ~2% MFU, and it also doesn't even help. I tried deleting it a while ago and back then it did help. So I'm guessing that some hyperparameter tuning obviated the reason for it since then

e8c30c3b19 add notebook used for scaling laws analysis

3af4dcf6ee also add scaling_laws.sh script if it's a useful reference

4cc605b940 quick pointer to miniseries post in readme for now

Compare 7 commits »

tacit synced commits to refs/pull/409/head at tacit/nanochat from mirror 2026-01-08 12:42:50 +00:00

489075bdbd Add support for ROCm backend in speedrun script

061f83c152 delete grad_clip. appears to not be necessary at all. not only was it buggy because the clipping happened per gpu before grad synchronization, but it costs ~2% MFU, and it also doesn't even help. I tried deleting it a while ago and back then it did help. So I'm guessing that some hyperparameter tuning obviated the reason for it since then

e8c30c3b19 add notebook used for scaling laws analysis

3af4dcf6ee also add scaling_laws.sh script if it's a useful reference

4cc605b940 quick pointer to miniseries post in readme for now

Compare 20 commits »

tacit synced commits to refs/pull/409/merge at tacit/nanochat from mirror 2026-01-08 12:42:50 +00:00

cd18a1dcb3 Merge 489075bdbd into 061f83c152

489075bdbd Add support for ROCm backend in speedrun script

061f83c152 delete grad_clip. appears to not be necessary at all. not only was it buggy because the clipping happened per gpu before grad synchronization, but it costs ~2% MFU, and it also doesn't even help. I tried deleting it a while ago and back then it did help. So I'm guessing that some hyperparameter tuning obviated the reason for it since then

e8c30c3b19 add notebook used for scaling laws analysis

3af4dcf6ee also add scaling_laws.sh script if it's a useful reference

Compare 17 commits »

tacit synced commits to refs/pull/412/merge at tacit/nanochat from mirror 2026-01-08 12:42:50 +00:00

54d5d1f401 Merge db5e62fc2a into 061f83c152

061f83c152 delete grad_clip. appears to not be necessary at all. not only was it buggy because the clipping happened per gpu before grad synchronization, but it costs ~2% MFU, and it also doesn't even help. I tried deleting it a while ago and back then it did help. So I'm guessing that some hyperparameter tuning obviated the reason for it since then

e8c30c3b19 add notebook used for scaling laws analysis

3af4dcf6ee also add scaling_laws.sh script if it's a useful reference

4cc605b940 quick pointer to miniseries post in readme for now

Compare 6 commits »

tacit synced commits to refs/pull/405/merge at tacit/nanochat from mirror 2026-01-08 12:42:49 +00:00

8a4e83130e Merge 3c4be194e9 into 061f83c152

061f83c152 delete grad_clip. appears to not be necessary at all. not only was it buggy because the clipping happened per gpu before grad synchronization, but it costs ~2% MFU, and it also doesn't even help. I tried deleting it a while ago and back then it did help. So I'm guessing that some hyperparameter tuning obviated the reason for it since then

e8c30c3b19 add notebook used for scaling laws analysis

3af4dcf6ee also add scaling_laws.sh script if it's a useful reference

4cc605b940 quick pointer to miniseries post in readme for now

Compare 7 commits »

tacit synced commits to refs/pull/407/head at tacit/nanochat from mirror 2026-01-08 12:42:49 +00:00

47885e743b Fix WANDB_RUN parameter passing in runcpu.sh

061f83c152 delete grad_clip. appears to not be necessary at all. not only was it buggy because the clipping happened per gpu before grad synchronization, but it costs ~2% MFU, and it also doesn't even help. I tried deleting it a while ago and back then it did help. So I'm guessing that some hyperparameter tuning obviated the reason for it since then

e8c30c3b19 add notebook used for scaling laws analysis

3af4dcf6ee also add scaling_laws.sh script if it's a useful reference

4cc605b940 quick pointer to miniseries post in readme for now

Compare 20 commits »