Two bugs: (1) force-web-search toggle bypassed identity veto — 'who are u' with Search on hit Tavily and got personality-quiz garbage. Now we always check _is_identity_or_meta() which covers identity, creator, samosaChaat references AND greetings (hi/hello/hey/what's up) before honoring the force toggle. (2) Model ignored injected Tavily result and answered from training priors (e.g. generic VP bio instead of specific Armenia/Iran facts). Added a grounding suffix after <|output_end|> ('Based on the search results above, ' for web_search, 'The result is ' for calculator) so the model's next tokens condition on the fresh tool output instead of spinning up memory.
needs_calculator now extracts the actual expression from: bare arithmetic (900+100), verbal math (900 plus 100), percentage (17% tip on 45), with comma-stripping and whitespace normalization. serve.py wires it into the force-prefix path parallel to web_search — if no web-search trigger, check calculator, pre-seed real tool call + result so the model sees the grounded answer in context.
Adds needs_web_search_contextual(messages) that picks the subject from the most recent user turn and replaces him/her/it in the current query. Vetoes when prior turns were about identity. Also adds TRAINING_ROADMAP.md — six-phase plan (tokens redacted).
ChatInput: textarea on top, inline tool pills (Think, Search) on the left and send button on the right — single rounded pod, no more bolted-on feel. Smaller pill buttons with subtle ring instead of heavy borders. MessageBubble: add sanitizeModelOutput() that strips training-artifact leaks: <b>/<i>/<strong>/<em> HTML tags, stray standalone '<' markers, leading 'Answer:/Response:' labels, placeholder image markdown. Applied before tool-marker parsing so cleaned text also feeds the <think> card renderer.
There's only one deployed model (samosaChaat). Drop the 'nanochat · base' select dropdown from the Sidebar and replace the header model badge with a static 'samosaChaat' label. Removes unused MODEL_OPTIONS / setModel / ChevronDown imports.
Added patterns for: tell me about yourself / you / about you, what do/can you do, what are your capabilities / skills, how do you work, what are you good at, what's your purpose / story / mission, where did you come from, how were you built, are you an AI / chatbot / language model, model meta (model/version/context/training cutoff), creator socials (github/linkedin/twitter), and more writing tasks (song, joke). All 27 identity cases now short-circuit without hitting Tavily.
The heuristic classifier was triggering web_search on 'who is your creator', 'who is manmohan sharma', 'who created you' etc — which returned irrelevant Tavily results (Tyler the Creator, Waaree CFO) when the model's SFT training already has the correct grounded identity answer. Added _IDENTITY_VETO_PATTERNS covering: self-referential questions, creator/maker/developer queries, competitor/provenance attacks (are you chatgpt/made by openai), samosaChaat/Manmohan name references, meta-questions (parameters/architecture/training/open source), greetings (hi/hello/hey), small talk, and writing/reasoning tasks that the model answers from memory. Veto runs before all positive classification.
New Globe/'Search' toggle next to the Brain/'Think' button. When ON, every message sent pushes force_web_search=true through: frontend -> chat-api -> Modal. Modal bypasses the heuristic classifier and always pre-seeds the assistant turn with a real Tavily-grounded tool call + result. Toggle is independent of Think — use either or both. Classifier still runs when toggle is OFF, so auto-detection of 'current president' / 'latest weather' / etc still works without any user action.
Bug: after runtime tool injection, the post-injection break scanned gen_ids[pre_injection_len:] which included our own injected <|output_start|>…<|output_end|> — so the loop-break fired IMMEDIATELY and stopped the turn before the model could write its final answer. Visible on multi-turn queries like a follow-up 'tell me more about him' where the model naturally issued a tool call, got real Tavily output, and then got cut off. Fix: track post_injection_start (the index AFTER injected tokens) and only scan from there for stray markers.
Adds modal/_query_classifier.py with regex patterns covering time-sensitive queries (current/present/latest/today/weather/CEO/president/stock/news/sports/etc). Modal serve.py classifies each user message and, when it matches, pre-seeds the assistant turn with a real Tavily-backed tool call + result — so 'whos the present president' now triggers web_search the same as 'current president'. Also tightens the post-injection break to fire on any leaked tool marker. UI: MessageBubble.tsx now strips orphan close-tags (<|output_end|> without an open), dedupes consecutive identical tool-result blocks, and removes fragment markers from text segments so they don't leak into the message body.
Token-id sequence match failed because BPE has multiple valid tokenizations of the same text, so the greedy encoder output didn't match the model's sampled path. Instead decode gen_ids directly and search for the marker text. Batch-decoding produces complete text even if single-token decodes return empty strings.
Training data taught the model to echo another <|output_start|>…<|output_end|> after our injected real tool result. Detect that second sequence and break the turn; the grounded answer has already streamed to the client.
Previous text-stream approach lost markers because BPE partial-byte tokens decode to empty strings, so assistant_text never accumulated the full marker. Switch to matching the ordinary-token id sequence directly (python_start = [60,124,25145,95,17104,124,62]).
The SFT loader tokenizes assistant content with .encode() (ordinary), not .encode_special(), so the model was trained to emit <|python_start|> / <|python_end|> as the 7-token ordinary sequence [60, 124, 25145, 95, 17104, 124, 62] rather than as special token id 32764. My prior state-machine matched token_id == python_start_id, which never fired — so tool calls were never executed and the model just hallucinated fake tool results (Official leadership page etc). Fix: detect markers in the decoded text stream, parse the payload between <|python_start|> and <|python_end|>, execute the tool, inject the real <|output_start|>…<|output_end|> tokens into both the SSE stream and the model's input_ids. Next-token prediction is now grounded on real Tavily output.
Model R6 (97% pass rate on 33-probe eval, val_bpb 0.2635):
- modal/serve.py + modal/_tools.py: tool-aware streaming with
TavilySearchBackend auto-detect, python_start/end state machine,
output_start/end forcing; mount tavily secret
- modal/serve.py: MODEL_TAG=d24-sft-r6, model path points at new SFT r6
- services/chat-api/routes/messages.py: accept thinking_mode flag,
inject samosaChaat system prompt (direct or <think> variant) into
first user message before streaming to Modal
- services/frontend/components/chat/ChatInput.tsx: Brain toggle
'Think' button next to send; when active, model uses think mode
- services/frontend/components/chat/ChatWindow.tsx: track
thinkingMode state, pass through to API body as thinking_mode
- services/frontend/components/chat/MessageBubble.tsx: parse and
render <think>...</think> as collapsible italic blocks;
<|python_start|>...<|python_end|> as tool-call cards with icons
per tool name; <|output_start|>...<|output_end|> as result cards
with expandable JSON
- nanochat/tools.py: TavilySearchBackend class + auto-detect
- nanochat/ui.html: legacy UI reasoning toggle (kept for parity)
Tool execution verified live: query -> web_search via Tavily ->
Macron returned with grounded answer.
When docker compose recreates a service, it gets a new internal IP.
nginx was resolving upstream hostnames once at startup and serving 502
until someone manually restarted it — which is what broke /api/auth
after the last deploy.
Uses Docker Compose's embedded DNS (127.0.0.11) and moves each
proxy_pass onto a variable so nginx re-resolves every request.
Rewrites replace the path-stripping behavior that variable-form
proxy_pass doesn't provide out of the box.
Also adds a `nginx -t && nginx -s reload` step in the deploy workflow
so future nginx.conf edits land without manual ssh.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Assistant responses were capped at max-w-[75%] of the column, so long
replies broke into a narrow block with dead space on the right. Cap
only applies to user bubbles now; assistant messages use w-full of the
max-w-3xl content column, matching how ChatGPT/Claude render replies.
Also bumps message vertical spacing from mb-3 to mb-5.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
LandingNav was max-w-3xl which forced "How it works" and "Try
samosaChaat" to wrap on two lines. Bumps the pill to 1100px,
tightens the link padding, demotes the @ handle to lg+, and adds
whitespace-nowrap to every chip so nothing wraps again. Default
theme is now dark — the no-flash init script adds .dark unless the
user has explicitly stored 'light', and the useTheme hook seeds
from the same logic.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
aws-actions/configure-aws-credentials needs id-token: write to mint the
OIDC JWT and assume AWS_ROLE_ARN. Without it the deploy-ec2 workflow
fails at the credentials step. Add the permission at workflow scope.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(inference): deploy d24 SFT weights to Modal
Repoint Modal inference app from the broken d20 checkpoint to our own
ManmohanSharma/nanochat-d24 SFT step 484. Rewrites the standalone model
as an inference-only port of nanochat/gpt.py so the modern architecture
(smear gate, per-layer value embeddings, ve_gate, backout, sliding
window attention via SDPA, rotary base 100000, padded vocab, logit
softcap) loads cleanly from the checkpoint. Tokenizer loads the pickled
tiktoken encoding directly so special tokens end up at their true IDs
(32759-32767), and the stop check uses that set instead of hardcoded
0-8. GPU bumped to L4 for headroom. HF token sourced from the
'huggingface' Modal secret.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(frontend): polished redesign with serif display + dark mode
Lifts the craft level of the landing and chat UI without changing the
desi identity. Adds Fraunces for display headlines, a floating pill
LandingNav, a saffron-glow hero with a large serif headline and black
pill CTAs, and three gradient-tiled feature cards with inline SVG
glyphs replacing the emoji cards. The chat empty state is now a serif
greeting with pill-chip prompt starters, and ChatInput is a single
rounded pod so the send button sits inside the input (fixes the
misaligned floating button). Adds a class-based dark mode across the
chat surfaces with a sun/moon toggle in the sidebar footer, powered by
a small useTheme hook and a no-flash init script in the root layout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(frontend): add ESLint config so CI lint step passes
next lint was failing with an interactive prompt because the repo had
no ESLint config. Adds a minimal next/core-web-vitals extends and
drops the now-unloadable @typescript-eslint/no-explicit-any disable
directive in the stream proxy by narrowing the body type to unknown.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaced the double-proxy (browser→Next.js→chat-api→Modal) with
direct streaming (browser→nginx→chat-api→Modal). Added nginx route
for /api/conversations → chat-api. Inlined SSE parsing in ChatWindow
instead of useSSE hook going through /api/chat/stream.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
