Commit Graph

5 Commits

Author SHA1 Message Date
Barış Özmen
bdd2e4e470 rustbpe: Remove redundant closure in reduce 2025-12-29 04:14:03 +03:00
Barış Özmen
015ce712d7 rustbpe: Add Default impl for Tokenizer 2025-12-29 04:13:47 +03:00
Barış Özmen
790f3be65c add rust batch encode as a faster option over encode 2025-12-18 19:17:59 +03:00
Andrej Karpathy
9467d83cf2 fix memory leak bug in rust tokenizer ty @mitsuhiko 2025-10-19 23:54:31 +00:00
karpathy
3a5e0bc50b initial commit 2025-10-13 06:49:24 -07:00