* feat(inference): deploy d24 SFT weights to Modal
Repoint Modal inference app from the broken d20 checkpoint to our own
ManmohanSharma/nanochat-d24 SFT step 484. Rewrites the standalone model
as an inference-only port of nanochat/gpt.py so the modern architecture
(smear gate, per-layer value embeddings, ve_gate, backout, sliding
window attention via SDPA, rotary base 100000, padded vocab, logit
softcap) loads cleanly from the checkpoint. Tokenizer loads the pickled
tiktoken encoding directly so special tokens end up at their true IDs
(32759-32767), and the stop check uses that set instead of hardcoded
0-8. GPU bumped to L4 for headroom. HF token sourced from the
'huggingface' Modal secret.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(frontend): polished redesign with serif display + dark mode
Lifts the craft level of the landing and chat UI without changing the
desi identity. Adds Fraunces for display headlines, a floating pill
LandingNav, a saffron-glow hero with a large serif headline and black
pill CTAs, and three gradient-tiled feature cards with inline SVG
glyphs replacing the emoji cards. The chat empty state is now a serif
greeting with pill-chip prompt starters, and ChatInput is a single
rounded pod so the send button sits inside the input (fixes the
misaligned floating button). Adds a class-based dark mode across the
chat surfaces with a sun/moon toggle in the sidebar footer, powered by
a small useTheme hook and a no-flash init script in the root layout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(frontend): add ESLint config so CI lint step passes
next lint was failing with an interactive prompt because the repo had
no ESLint config. Adds a minimal next/core-web-vitals extends and
drops the now-unloadable @typescript-eslint/no-explicit-any disable
directive in the stream proxy by narrowing the body type to unknown.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaced the double-proxy (browser→Next.js→chat-api→Modal) with
direct streaming (browser→nginx→chat-api→Modal). Added nginx route
for /api/conversations → chat-api. Inlined SSE parsing in ChatWindow
instead of useSSE hook going through /api/chat/stream.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Chat-api expects {content, temperature, max_tokens, top_k} but frontend
was sending {messages: [...]}. Now extracts last user message as content
when proxying to /api/conversations/:id/messages.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Nginx was catching /api/chat/stream and /api/conversations and sending
them to chat-api:8002, bypassing the frontend's Next.js API routes.
Now only /api/auth/* goes directly to auth service. Everything else
goes to frontend, which proxies internally to backend services.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Chat-api doesn't define the users model (owned by auth service), so
SQLAlchemy can't resolve the FK. use_alter=True defers the constraint
to ALTER TABLE, avoiding the NoReferencedTableError at startup.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The inference client now auto-detects if the URL already ends with
/generate (Modal's endpoint URL pattern) and skips appending the path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Landing page: warm gradient background, illustrations flanking hero text
(180-220px), new tagline, features section with 3 cards, footer updated
to "Built by Manmohan", gold CTA and nav buttons, toran moved to hero.
Chat page: removed "Chat Completions" header, added samosa logo and
bigger suggestion cards to empty state, sidebar empty state message,
input area top border/shadow, more prominent new chat button.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
AUTH_BASE_URL, FRONTEND_URL, INTERNAL_API_KEY, SESSION_SECRET,
COOKIE_SECURE, COOKIE_DOMAIN, REFRESH_COOKIE_NAME were in .env
but not passed to auth container. OAuth callbacks were using
localhost:8001 instead of the public URL.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root .gitignore had `models/` which matched both ML weights AND
SQLAlchemy model dirs. Changed to `/models/` (root only).
Added auth/src/models/ (User) and chat-api/src/models/ (Conversation, Message).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Auth service was crash-looping with ModuleNotFoundError for
prometheus_fastapi_instrumentator. Chat-api was also missing it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove NextAuth and replace with token-based auth against the backend
auth service (OAuth + JWT). The frontend now redirects login to
/api/auth/google and /api/auth/github (proxied by nginx to the auth
service), captures the JWT from the redirect query param, and uses it
for all API calls.
Key changes:
- Remove next-auth dependency and all NextAuth config/routes
- Add lib/auth-client.ts (JWT token storage + auth headers)
- Add hooks/useAuth.ts (client-side auth state + token capture)
- Rewrite middleware.ts to pass-through (client-side auth only)
- Login page uses plain <a> links to /api/auth/{provider}
- Chat page captures access_token from OAuth redirect
- Zustand store fetches conversations from real chat-api via JWT
- API routes proxy /api/conversations/* to chat-api with auth
- chat/stream route supports conversationId + auth header forwarding
- useSSE hook accepts auth headers for authenticated streaming
- Sidebar loads conversations from API, supports delete
- Landing page (Hero, LandingNav) uses useAuth instead of useSession
- Add .env.production.example and scripts/generate-jwt-keys.sh
Mock echo fallback preserved when CHAT_API_URL is not set.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- deploy.sh: single script to switch between EC2 and EKS modes
- ec2: docker-compose with ECR images + nginx SSL reverse proxy
- eks: terraform apply + helm install (for demos/grading)
- eks-down: terraform destroy (stop costs)
- docker-compose.prod.yml: ECR image overrides + nginx service
- nginx/nginx.conf: reverse proxy with SSL, SSE streaming support
- deploy-ec2.yml: auto-deploy to EC2 after images are built
- Remove old single-server deploy.yml
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The observability PR added structlog and prometheus-fastapi-instrumentator
to inference pyproject.toml but did not regenerate uv.lock, causing
Docker build to fail with --locked flag.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root pyproject.toml uses uv features (extra in sources, conflicts)
that caused uv sync to fail in CI. Fix by:
1. Replace pip install uv==0.4.30 with astral-sh/setup-uv@v4 (latest)
2. Add --no-workspace flag so services don't inherit root config
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaces the helm/observability scaffold with a real monitoring stack
wired into the samosaChaat platform.
Helm chart (helm/observability/)
- Chart.yaml declares kube-prometheus-stack (~62.0) and loki-stack
(~2.10) as subchart dependencies.
- values.yaml configures Prometheus (15d retention, 50Gi PVC,
ServiceMonitor + rule selector on app.kubernetes.io/part-of:
samosachaat), Alertmanager (10Gi PVC), Grafana (OAuth-only via
GitHub + Google, local login disabled, Prometheus + Loki datasources,
dashboards auto-provisioned from a ConfigMap, email + Slack contact
points with a critical route to Slack), Loki (50Gi, 30d retention,
tsdb schema), and Promtail (JSON pipeline that lifts level / service
/ trace_id / user_id into labels, scrape config with pod labels).
- Alert rules: HighCPU, HighMemory, DiskSpaceLow, High5xxRate,
InferenceServiceDown, HighP99Latency.
- templates/grafana-dashboards-configmap.yaml renders every file under
dashboards/ into a single grafana_dashboard=1 ConfigMap.
- dashboards/node-health.json, app-performance.json, inference.json -
fully-formed Grafana dashboards with Prometheus datasource variable,
templated app selector, thresholded gauges, and LogQL-ready labels.
Scraping (helm/samosachaat/templates/servicemonitor.yaml)
- ServiceMonitor CRs for auth / chat-api / inference that Prometheus
picks up via the part-of=samosachaat selector; scrapes /metrics
every 15s and replaces the app label so dashboards line up.
Application instrumentation
- services/{auth,chat-api,inference} each depend on
prometheus-fastapi-instrumentator and expose /metrics (request count,
latency histograms, in-progress gauges).
- services/auth/src/logging_setup.py and
services/inference/src/logging_setup.py mirror the canonical
chat-api implementation - structlog JSON with service, trace_id,
user_id context injection.
- configure_logging() is called at create_app() in auth and inference;
inference's main.py now uses structlog via get_logger() instead of
logging.getLogger.
- log_level setting added to auth + inference config (LOG_LEVEL env).
Docs
- contracts/logging-standard.md defines the required JSON fields,
Python (structlog) + Node.js (pino) implementations, LogQL examples
for cross-service queries, and the x-trace-id propagation contract.
Closes#9
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds tooling and documentation for Day 2 cluster operations:
- scripts/rotate-nodes.sh: interactive node-rotation driver that applies
terraform to pick up the latest SSM-resolved EKS AMI and watches the
rolling replacement.
- scripts/demo-schema-change.sh: end-to-end demo of the zero-downtime
is_favorited column migration via helm upgrade + migration hook.
- scripts/verify-deployment.sh: post-deploy health check across pods,
per-service HTTP health endpoints, rollout status, and PDBs.
- docs/chaos-runbook.md: failure-mode playbook with simulate / Grafana /
Loki / recovery steps for six scenarios (pod kill, node failure, DB
pool exhaustion, inference OOM, high latency, SSL issues) plus a
Loki quick-reference.
- terraform/modules/eks: expose current_node_ami_id output, add
update_config.max_unavailable_percentage (configurable, default 33)
so node-group rolls are controlled.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds GitHub Actions workflows for per-service CI (paths-filter gated),
dev image builds to ECR via OIDC, RC*-tag UAT promotion with image
re-tagging and Helm deploy, v*-tag blue/green prod release with smoke
test + ingress swap, and a nightly docker-compose integration suite.
Ships a Helm umbrella chart (dev/uat/prod values) with Deployments,
ClusterIP Services, ALB Ingress (samosachaat.art + grafana host), HPAs
for chat-api/inference in prod, PDBs, ConfigMap/Secret wiring, and an
alembic db-migrate Helm hook job.
Wires commitlint + husky for Conventional Commits at the repo root.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- FastAPI service that manages conversations and messages in PostgreSQL
(SQLAlchemy 2.0 async + asyncpg) and streams assistant responses back
to the client via sse-starlette, forwarding the inference service SSE
contract unchanged.
- Auth guard validates every request against the auth service
/auth/validate endpoint (X-Internal-API-Key) and caches results in an
in-process TTL cache (5 min, 1024 entries) to absorb request bursts.
- Every query filters by authenticated user_id; cross-user access
returns 404. Message send flow auto-titles the first message,
persists the streamed assistant response after the client disconnects,
and records token_count + inference_time_ms.
- /api/models{,/swap} proxies the inference admin surface; swap
requires is_admin on the validated user.
- Structured JSON logging via structlog with trace_id + user_id
ContextVars attached to every log line.
- Test suite (pytest + aiosqlite + respx) covers CRUD, user scoping,
streaming SSE persistence, regenerate, model proxy admin gate,
and the stream proxy error path. 16/16 passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
