Building a Production RAG Copilot with n8n, Supabase and LLaMA: Lessons from a Real Implementation

What was built. The author implemented a company-internal knowledge copilot: ask in Telegram, get a cited answer from internal docs or an explicit "I don't know." The stack combines n8n orchestration on Railway, Supabase with pgvector for vector search, Gemini 768-d embeddings, Groq LLaMA 3.3 (70B) as the LLM, and Google Drive as the canonical source. Ingestion chunks PDFs, embeds them, and upserts into Supabase using a modified-time diff to skip unchanged files.

Architectural takeaways. The workflow mixes hybrid retrieval (pgvector + full-text fused via Reciprocal Rank Fusion) and then uses the LLM to re-rank and apply a confidence gate. That pattern addresses both semantic recall and exact-match fidelity - important when documents contain names, clauses, or regulatory text that require precision. The delta-ingestion strategy is practical: avoid reprocessing and keep vector store compact.

Business impact and risk. For leaders, this is a pragmatic template for an internal knowledge assistant that balances utility and safety. However, running a 70B LLM plus embedding pipelines increases operational costs, latency, and risk surface (data leakage, stale docs, hallucinations). The confidence gate helps mitigate hallucination but requires a clear SLA on false negatives (when the system replies "I don't know").

Recommendations. Measure retrieval precision, answer faithfulness, and user friction: track cited-source usage, fallback rates, and manual escalation. Invest in observability (query traces linking retrieval hits to generated answers), refresh policies for embeddings, and synthetic negative tests. Consider cheaper LLMs for re-ranking or a two-tier inference approach (small model for filtering, large for final answers) to optimize cost/performance tradeoffs.