Commit Graph

3 Commits

Author SHA1 Message Date
Barış Özmen
790f3be65c add rust batch encode as a faster option over encode 2025-12-18 19:17:59 +03:00
Andrej Karpathy
9467d83cf2 fix memory leak bug in rust tokenizer ty @mitsuhiko 2025-10-19 23:54:31 +00:00
karpathy
3a5e0bc50b initial commit 2025-10-13 06:49:24 -07:00