Omri Jukin

Case study

Confidential AI Operator Console

Shows practical AI systems engineering beyond prompt wrappers: provider abstraction, local-first runtime design, multimodal media processing, scoped memory, redacted observability, guardrails, queues, database modeling, and a usable operator console.

in-progressSole full-stack AI systems engineer responsible for architecture, implementation, local provider integration, UI workflows, persistence, guardrails, tracing, and test coverage.Outcome

Problem

A confidential AI/media systems POC built as a local-first full-stack application. Public-facing details focus on the technical architecture, runtime boundaries, and implementation work rather than the confidential product concept. The system is implemented as an Nx/pnpm monorepo with a Next.js operator console, shared TypeScript packages, Drizzle/PostgreSQL persistence, pgvector-backed memory, Redis/BullMQ scheduling, Dockerized local AI/media services, and tested utility modules. The engineering work covers avatar interaction, browser media capture, selected keyframes, audio and visual signal summaries, provider health checks, local ASR/TTS integration, lip-sync support, scoped agent memory, redacted model-run logging, guardrail checks, and traceable cue-processing flows. The main challenge was making an experimental AI workflow inspectable and production-minded without exposing sensitive product mechanics. The implementation separates private planning from public response composition, validates structured model output, reports degraded provider states honestly, and links traces across media, analysis, memory, model runs, guardrails, conversation messages, and scheduler jobs.

Constraints

  • Keep the product concept private while making the engineering value visible
  • Avoid public details that reveal confidential mechanics or market positioning
  • Report unavailable or degraded local providers honestly instead of faking behavior
  • Prevent private runtime context from leaking into user-facing AI output
  • Make multimodal AI flows traceable across media, analysis, memory, model runs, and guardrails

What I built

Local-first AI systems proof of concept focused on multimodal cue processing, avatar interaction, scoped memory, model-provider orchestration, guardrails, redacted traces, and operator-grade debugging.

Architecture

A focused full-stack system with explicit boundaries between public UI, data access, server-side orchestration, and operational workflows.

01

Public UI

02

Typed API boundary

03

Database-backed content

Decisions and trade-offs

  • Framed the public case study around the AI systems architecture instead of the protected product idea
  • Separated private planning from public response composition to protect sensitive runtime context
  • Used scoped persistence so memory and traces stay tied to explicit runtime boundaries
  • Added guardrail checks before final user-facing output is stored or returned
  • Kept ASR, TTS, lip-sync, database, queue, and web services as explicit local runtime boundaries
  • Built debug traces around real system events instead of only showing generated text

Outcome

Built a working confidential AI systems POC that demonstrates local model orchestration, multimodal cue handling, avatar interaction, scoped memory, provider health reporting, redacted traces, guardrail checks, and inspectable operator workflows while keeping the product concept private.

Repository context

The repository and full product concept are private. Public-facing details focus on technical architecture, local provider integration, multimodal processing, runtime boundaries, guardrails, observability, and implementation ownership.