tacit/nanochat
1
0
Fork 0
mirror of https://github.com/karpathy/nanochat.git synced 2026-03-19 11:23:19 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

fix(tokenizer): add bounds check for system-only conversation to avoid IndexError

Browse Source
This commit is contained in:
dhunganapramod9 2026-02-28 01:08:20 -05:00
parent 542beb0c8c
commit e009166646
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
nanochat/tokenizer.py
View File

@ -284,6 +284,8 @@ class RustBPETokenizer:
# some conversation surgery is necessary here for now...
conversation = copy.deepcopy(conversation) # avoid mutating the original
messages = conversation["messages"]
if len(messages) < 2:
raise ValueError("System message must be followed by a user message")
assert messages[1]["role"] == "user", "System message must be followed by a user message"
messages[1]["content"] = messages[0]["content"] + "\n\n" + messages[1]["content"]
messages = messages[1:]