tacit/nanochat
1
0
Fork 0
mirror of https://github.com/karpathy/nanochat.git synced 2026-05-25 17:18:01 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Ada is also supported

Browse Source
This commit is contained in:
Sofie Van Landeghem 2026-02-18 22:57:15 +01:00 committed by GitHub
parent 345935c5f3
commit 0c4ce3697e
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
nanochat/flash_attention.py
View File

@ -26,8 +26,8 @@ def _load_flash_attention_3():
if not torch.cuda.is_available():
return None
try:
# FA3 kernels are currently compiled for Hopper (sm90) and Ampere (sm80/sm86)
# Ada (sm89), Blackwell (sm100) need SDPA fallback until FA3 is recompiled
# FA3 kernels are currently compiled for Hopper (sm90), Ada (sm89) and Ampere (sm80/sm86)
# Blackwell (sm100) needs SDPA fallback until FA3 is recompiled or FA4 is released
from kernels import get_kernel, has_kernel
supported = has_kernel(hf_kernel)
if not supported: