4 Commits

Author	SHA1	Message	Date
Kartik Vashishta	e0fc57fd45	Merge 181e7f1c15 into 230d6cf6c6	2026-02-01 22:31:03 -06:00
Kartik Vashishta	181e7f1c15	Fix SDPA KV-cache for per-row cache_seqlens	2026-02-01 18:36:07 +07:00
Andrej Karpathy	3ba42e8135	Fix SDPA KV-cache decode to respect sliding window (#456) ... SDPA fallback now respects sliding window during single-token KV-cache decode by slicing K/V to the last (window + 1) tokens. Also simplifies the mask building for chunk inference to properly apply sliding window in that path as well. Fixes #452 Co-Authored-By: Kartik Vashishta <kartikv776@gmail.com> Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>	2026-01-30 17:32:12 +00:00
Andrej Karpathy	8203efa919	implement flash attention 3 fallback to pytorch sdpa by touching as few lines of code as possible in main files and keeping all implementation to a single file. add tests. add helpful warning messages for the user.	2026-01-16 17:37:51 +00:00