120 added, 52 removed. Audit A to A.
---
name: ai-llm-integration-expert
description: "Expert guide for integrating Large Language Models (LLMs), Model Context Protocol (MCP), RAG architecture, vector databases, and AI agents / Panduan ahli untuk integrasi LLM, Model Context Protocol (MCP), arsitektur RAG, vector database, dan agen AI."
author: "Roedy Rustam"
---
- # AI & LLM Integration Expert
+ # AI & LLM Integration Expert (2026 Edition)
[English](#english) | [Bahasa Indonesia](#bahasa-indonesia)
---
<a name="english"></a>
## English
### Description
- Production-grade guidelines for integrating Artificial Intelligence (AI), Model Context Protocol (MCP), and Large Language Models (LLMs) into modern applications. Covers Retrieval-Augmented Generation (RAG), vector embeddings, token streaming for real-time UI, agentic tool execution, and MCP architecture.
+ Production-grade guidelines for integrating AI, Model Context Protocol (MCP), and Large Language Models (LLMs) into modern applications. Covers RAG pipelines, vector embeddings, real-time token streaming, agentic tool execution, memory architectures, prompt caching, and multi-model orchestration.
### Trigger Conditions
- - Integrating OpenAI (GPT-4o/o3), Anthropic (Claude 3.7/Sonnet), Gemini (3.5/3.6 Pro/Flash), or open-source reasoning models (DeepSeek-R1/V3, Llama 3.3).
- - Implementing Model Context Protocol (MCP) server or client integrations.
- - Building AI chatbots, copilots, or autonomous AI agent workflows (LangGraph, Vercel AI SDK 4.x/5.x).
- - Implementing RAG with vector databases (Supabase `pgvector` with HNSW indexes, Qdrant, Pinecone).
- - Handling real-time AI token generation via Server-Sent Events (SSE), Web Streams, or WebSockets.
- - Designing AI agents with tool-calling capabilities and strict Zod schema validations.
+ - Integrating frontier models: **OpenAI GPT-4o / GPT-4.1 / GPT-5**, **Anthropic Claude 3.7 / Claude 4 Sonnet/Opus**, **Google Gemini 3.1 Pro/Flash/Ultra**, or open-source models (DeepSeek-V3/R2, Llama 4, Qwen 3).
+ - Implementing **Model Context Protocol (MCP)** server or client integrations (MCP 1.9+).
+ - Building AI chatbots, copilots, or **autonomous AI agent workflows** (LangGraph, OpenAI Agents SDK, Google ADK, Mastra.ai, Vercel AI SDK 4.x/5.x).
+ - Building **agentic memory** systems (short-term, long-term, episodic) using Mem0, MemGPT, or Supabase-backed memory stores.
+ - Implementing RAG with vector databases (Supabase `pgvector` HNSW, Qdrant, Pinecone, Weaviate).
+ - Handling real-time AI token streaming via **SSE**, **Web Streams**, or **WebSockets**.
+ - Designing AI agents with **tool-calling**, **computer use** (browser/OS automation), or **code interpreter** capabilities.
+ ### Model Capability Matrix (2026)
+
+ | Provider | Model | Context | Strengths |
+ |---|---|---|---|
+ | OpenAI | GPT-4o / GPT-4.1 | 128K | Multimodal, tool-calling, speed |
+ | OpenAI | GPT-5 | 1M+ | Reasoning, agentic tasks |
+ | Anthropic | Claude 4 Sonnet | 200K | Coding, long documents, Computer Use |
+ | Anthropic | Claude 4 Opus | 200K | Complex reasoning, deep analysis |
+ | Google | Gemini 3.1 Flash | 2M | Speed, cost efficiency, multimodal |
+ | Google | Gemini 3.1 Pro | 2M | Code, reasoning, deep research |
+ | Meta | Llama 4 Scout/Maverick | 512K | Open-source, self-hosted |
+ | DeepSeek | V3 / R2 | 128K | Code, math, cost-efficient |
+
### Core Architecture Guidelines
- #### 1. Model Context Protocol (MCP) Integration
- Standardize agent-tool interactions using the Model Context Protocol (MCP):
- - **MCP Servers**: Expose tools, resources, and prompt templates over JSON-RPC 2.0 (stdio or SSE transports).
+ #### 1. Model Context Protocol (MCP) Integration (v1.9+)
+ Standardize agent-tool interactions using MCP. The 2026 standard adds **Streamable HTTP transport** alongside stdio/SSE:
+ - **MCP Servers**: Expose tools, resources, and prompt templates over JSON-RPC 2.0.
+ - **Streamable HTTP**: New default transport for cloud-hosted MCP servers — supports bidirectional streaming without long-polling.
- **Security & Scope**: Validate all incoming parameters with Zod schemas. Enforce strict authorization boundaries before executing tool calls.
+ - **Tool namespacing**: Use `domain/action` naming (`files/read`, `db/query`) to avoid collisions in multi-server environments.
- #### 2. Advanced RAG (Retrieval-Augmented Generation) Pipeline
- Do not rely solely on internal model weights for domain knowledge. Build a production-grade RAG pipeline:
- 1. **Ingestion**: Chunk long documents (e.g., 500-1000 tokens with 10% overlap, respecting semantic boundary headings).
- 2. **Embedding**: Convert chunks to vectors using `text-embedding-3-small` or `nomic-embed-text`.
- 3. **Storage & Hybrid Search**: Store vectors in PostgreSQL using `pgvector` (with HNSW vector index) or Qdrant. Combine vector cosine similarity with full-text keyword search (BM25 / tsvector) for hybrid search retrieval.
- 4. **Reranking**: Use a cross-encoder reranker (e.g., Cohere Rerank) on top-K results to increase precision.
- 5. **Generation**: Inject top contextual snippets into the system prompt.
+ #### 2. AI Agents SDK Landscape (2026)
+ Choose the right agentic framework based on your needs:
- #### 3. Streaming Responses & Vercel AI SDK
- - **Backend (Next.js/Bun/FastAPI)**: Pipe stream chunks directly via Web Streams API or SSE.
- - **Frontend (React)**: Use Vercel AI SDK (`useChat`, `useCompletion`) or native ReadableStream readers for zero perceived latency.
+ | Framework | Best For | Key Feature |
+ |---|---|---|
+ | **OpenAI Agents SDK** | GPT-5 native agents | Handoffs, guardrails, tracing built-in |
+ | **Google ADK** | Gemini-powered agents | Multi-agent, streaming, Vertex AI integration |
+ | **LangGraph** | Complex stateful workflows | Graph-based, human-in-the-loop, any LLM |
+ | **Mastra.ai** | TypeScript-first agents | Built-in memory, evals, RAG |
+ | **Vercel AI SDK 5.x** | Streaming UI + agents | RSC streaming, multi-provider, tool-calling |
- #### 4. Structured Output & Tool Calling
+ #### 3. Agentic Memory Architecture
+ Production AI agents require persistent memory across sessions:
+ - **Short-term (In-context)**: Pass recent conversation turns in the prompt window.
+ - **Long-term (Vector Store)**: Store user preferences and past interactions as embeddings in `pgvector` or Qdrant; retrieve with semantic similarity.
+ - **Episodic (Episodic Memory)**: Use **Mem0** or **MemGPT** to give agents human-like recall — automatically summarizes and indexes past interactions.
+ - **Structured (Knowledge Graph)**: Use a graph DB (Neo4j, Kuzu) for entity relationships and factual memory.
+
+ #### 4. Advanced RAG (Retrieval-Augmented Generation) Pipeline
+ Build a production-grade RAG pipeline with hybrid search:
+ 1. **Ingestion**: Chunk documents (500-1000 tokens, 10% overlap, respect semantic headings).
+ 2. **Embedding**: Use `text-embedding-3-large` (OpenAI), `gemini-embedding-004` (Google), or `nomic-embed-text` (open-source).
+ 3. **Storage & Hybrid Search**: PostgreSQL `pgvector` (HNSW index) or Qdrant. Combine vector cosine similarity with BM25 full-text search for hybrid retrieval.
+ 4. **Reranking**: Apply cross-encoder reranker (Cohere Rerank 3, FlashRank) on top-K results.
+ 5. **Generation**: Inject top contextual snippets into the system prompt with source citations.
+
+ #### 5. Prompt Caching (Cost Optimization)
+ Reduce costs by 80-90% on repeated long prompts using provider-native caching:
+ - **Anthropic**: Automatic prompt caching for prompts > 1024 tokens (marked with `cache_control: {"type": "ephemeral"}`).
+ - **OpenAI**: Automatic prefix caching for prompts > 1024 tokens in GPT-4o and later models.
+ - **Google**: Context caching in Gemini API via `cachedContent` for large system prompts or documents.
+
+ #### 6. Streaming Responses & Vercel AI SDK 5.x
+ - **Backend (Next.js/Bun/FastAPI)**: Pipe stream chunks via Web Streams API or SSE.
+ - **Frontend (React)**: Use Vercel AI SDK `useChat` / `useCompletion` for zero perceived latency.
+ - **RSC Streaming**: Use AI SDK 5.x `streamUI` to stream React components from the server — agent-rendered UI.
+
+ #### 7. Computer Use & Browser Agents
+ Modern agents can control browsers and operating systems:
+ - **Anthropic Computer Use**: Claude 4 can take screenshots and control mouse/keyboard (ideal for browser automation).
+ - **Browser-Use**: Open-source library for browser-controlling agents with any LLM.
+ - **Playwright MCP Server**: Expose a full browser to AI agents via MCP tools (`navigate`, `click`, `screenshot`).
+
+ #### 8. Structured Output & Tool Calling
- Use native Tool/Function Calling APIs with strict JSON schemas via Zod (`zodResponseFormat`).
- - Support reasoning/thinking models (e.g., DeepSeek-R1, Gemini 3.5 Thought) by separating internal reasoning tokens `<think>` from client-facing output.
+ - Prefer **structured output mode** over prompt engineering for JSON — guaranteed schema conformance.
+ - Support reasoning models (DeepSeek-R2, Gemini 3.1 Pro Thinking, Claude 4 extended thinking) by separating internal `<think>` tokens from client-facing output.
- #### 5. Cost & Rate Limit Management
- - Monitor token consumption per session and store metrics in telemetry/analytics tables.
- - Implement semantic caching using Redis + vector similarity to skip redundant model queries.
+ #### 9. Cost & Rate Limit Management
+ - Monitor token consumption per session; store metrics in telemetry tables.
+ - Implement **semantic caching** using Redis + vector similarity to skip redundant model queries.
+ - Use **batch APIs** (OpenAI Batch, Anthropic Message Batches) for async, high-volume workloads at 50% cost reduction.
---
<a name="bahasa-indonesia"></a>
## Bahasa Indonesia
### Deskripsi
- Panduan tingkat produksi untuk mengintegrasikan Kecerdasan Buatan (AI), Model Context Protocol (MCP), dan Large Language Models (LLMs) ke dalam aplikasi modern. Panduan ini mencakup RAG (*Retrieval-Augmented Generation*), *vector embeddings*, *streaming* token UI *real-time*, eksekusi *tool* agen AI, dan arsitektur MCP.
+ Panduan tingkat produksi untuk mengintegrasikan AI, Model Context Protocol (MCP), dan LLM ke dalam aplikasi modern. Mencakup pipeline RAG, streaming token real-time, eksekusi tool agen AI, arsitektur memori, prompt caching, dan orkestrasi multi-model.
### Kondisi Pemicu
- - Mengintegrasikan OpenAI (GPT-4o/o3), Anthropic (Claude 3.7), Gemini (3.5/3.6), atau model penalaran open-source (DeepSeek-R1/V3, Llama 3.3).
- - Mengimplementasikan integrasi server atau klien Model Context Protocol (MCP).
- - Membangun chatbot AI, *copilot*, atau alur kerja agen AI otonom (LangGraph, Vercel AI SDK 4.x/5.x).
- - Mengimplementasikan RAG dengan *vector database* (Supabase `pgvector` indeks HNSW, Qdrant, Pinecone).
- - Menangani generasi token AI *real-time* via Server-Sent Events (SSE), Web Streams, atau WebSocket.
- - Merancang agen AI yang memiliki kemampuan *tool-calling* dengan validasi skema Zod yang ketat.
+ - Mengintegrasikan model frontier: GPT-4.1 / GPT-5, Claude 4 Sonnet/Opus, Gemini 3.1 Pro/Flash, atau model open-source (Llama 4, DeepSeek-V3/R2).
+ - Mengimplementasikan server/klien MCP (v1.9+ dengan Streamable HTTP transport).
+ - Membangun chatbot AI, copilot, atau alur kerja agen otonom (OpenAI Agents SDK, Google ADK, LangGraph, Mastra.ai).
+ - Membangun sistem **memori agentik** (jangka pendek, jangka panjang, episodik) menggunakan Mem0 atau pgvector.
+ - Mengimplementasikan RAG dengan vector database.
+ - Merancang agen AI dengan tool-calling, Computer Use, atau code interpreter.
### Panduan Arsitektur Inti
- #### 1. Integrasi Model Context Protocol (MCP)
- Strukturkan interaksi antara agen dan alat menggunakan standar MCP:
- - **Server MCP**: Sediakan alat (*tools*), sumber daya (*resources*), dan *prompt templates* melalui transport JSON-RPC 2.0 (stdio atau SSE).
- - **Keamanan**: Validasi semua parameter masuk menggunakan skema Zod dan berlakukan otorisasi sebelum mengeksekusi fungsi.
+ #### 1. Integrasi MCP (v1.9+)
+ Standarisasi interaksi agen-tool menggunakan MCP. Standar 2026 menambahkan **Streamable HTTP transport** — transport default baru untuk MCP server yang di-host di cloud, mendukung streaming dua arah tanpa long-polling.
- #### 2. Pipa RAG Lanjutan (*Retrieval-Augmented Generation*)
- 1. **Ingesti**: Pecah dokumen (500-1000 token dengan *overlap* 10% berbasis struktur judul semantik).
- 2. **Embedding**: Ubah teks menjadi vektor (`text-embedding-3-small`).
- 3. **Penyimpanan & Pencarian Hibrida**: Gunakan Supabase `pgvector` (indeks HNSW) atau Qdrant. Gabungkan pencarian vektor (*cosine similarity*) dengan pencarian teks (*full-text search*) untuk hasil yang lebih akurat.
- 4. **Reranking**: Terapkan model reranker (misalnya Cohere Rerank) pada hasil pencarian teratas.
- 5. **Generasi**: Suntikkan potongan konteks terbaik ke dalam *system prompt* LLM.
+ #### 2. Ekosistem Agents SDK (2026)
+ - **OpenAI Agents SDK**: Handoffs, guardrails, dan tracing bawaan untuk agen berbasis GPT-5.
+ - **Google ADK**: Multi-agen, streaming, integrasi Vertex AI untuk agen Gemini.
+ - **LangGraph**: Alur kerja stateful berbasis graph dengan dukungan human-in-the-loop.
+ - **Mastra.ai**: Framework TypeScript-first dengan memori, evaluasi, dan RAG bawaan.
+ - **Vercel AI SDK 5.x**: Streaming UI + agen dengan dukungan multi-provider dan RSC.
- #### 3. Streaming Respons & Vercel AI SDK
- - **Backend**: Alirkan *stream* token menggunakan Web Streams API atau SSE.
- - **Frontend**: Gunakan Vercel AI SDK (`useChat`) untuk memberikan pengalaman pengguna yang responsif tanpa jeda.
+ #### 3. Arsitektur Memori Agentik
+ - **Jangka Pendek**: Riwayat percakapan recent dalam context window.
+ - **Jangka Panjang (Vector Store)**: Preferensi dan interaksi masa lalu sebagai embedding di pgvector atau Qdrant.
+ - **Episodik**: Gunakan Mem0 atau MemGPT untuk recall mirip manusia — otomatis merangkum dan mengindeks interaksi lalu.
+ - **Terstruktur (Knowledge Graph)**: Gunakan graph DB (Neo4j, Kuzu) untuk relasi entitas dan memori faktual.
- #### 4. Output Terstruktur & Tool Calling
- - Gunakan API Tool Calling bawaan penyedia LLM dengan validasi Zod yang ketat (`zodResponseFormat`).
- - Untuk model penalaran (*reasoning models* seperti DeepSeek-R1 atau Gemini 3.5 Thought), pisahkan blok `<think>` internal dari keluaran teks publik.
+ #### 4. Pipeline RAG Lanjutan
+ 1. Chunking dokumen (500-1000 token, 10% overlap).
+ 2. Embedding dengan `text-embedding-3-large` atau `gemini-embedding-004`.
+ 3. Penyimpanan & pencarian hibrida: pgvector (HNSW) + BM25.
+ 4. Reranking dengan Cohere Rerank 3 atau FlashRank.
+ 5. Injeksi konteks terbaik ke system prompt.
- #### 5. Manajemen Biaya & Rate Limit
- - Pantau konsumsi token per sesi dan simpan di tabel telemetri.
- - Terapkan *semantic caching* menggunakan Redis + perbandingan vektor untuk menghemat biaya panggilan API.
+ #### 5. Prompt Caching (Optimasi Biaya)
+ Kurangi biaya 80-90% dengan caching prompt panjang:
+ - **Anthropic**: Caching otomatis untuk prompt > 1024 token.
+ - **OpenAI**: Prefix caching otomatis di GPT-4o ke atas.
+ - **Google**: Context caching di Gemini API via `cachedContent`.
+
+ #### 6. Streaming & Vercel AI SDK 5.x
+ Gunakan `useChat` / `useCompletion` untuk zero perceived latency di sisi klien. Gunakan `streamUI` di AI SDK 5.x untuk stream komponen React dari server (agent-rendered UI).
+
+ #### 7. Computer Use & Browser Agents
+ Claude 4 dapat mengambil screenshot dan mengontrol mouse/keyboard. Gunakan **Browser-Use** atau **Playwright MCP Server** untuk memberi agen kontrol atas browser.
+
+ #### 8. Output Terstruktur & Tool Calling
+ Gunakan mode structured output (bukan prompt engineering) untuk jaminan kesesuaian skema JSON. Gunakan Zod (`zodResponseFormat`) untuk validasi type-safe.
+
+ #### 9. Manajemen Biaya
+ - Pantau konsumsi token per sesi di tabel telemetri.
+ - Terapkan semantic caching dengan Redis + vektor.
+ - Gunakan Batch API (OpenAI / Anthropic) untuk workload async volume tinggi dengan biaya 50% lebih hemat.