Project S
Project S was a private Retrieval-Augmented Generation platform for an enterprise media and services organization that needed better customer support answers without giving up local control of sensitive information. When I came aboard the project, an earlier proof of concept already existed, but it had plateaued around 50 percent accuracy, so the work became an architectural rebuild rather than a prompt-tuning exercise.
Three-phase RAG overhaul, multimodal ingestion, hybrid retrieval, local open-source inference, and hardware stabilization under strict privacy constraints.
At A Glance
This was not a generic chatbot rollout. It was a private knowledge system that had to recover useful evidence from difficult enterprise material, stay inside local infrastructure, and become reliable enough for real support workflows.
Business Need
Improve support answer quality for complex legacy documentation without exposing internal material to hosted AI defaults.
Core Work
Decouple ingestion, rebuild retrieval, translate visual content into searchable text, and stabilize local inference on shifting hardware.
Outcome
Raised ideal test accuracy from roughly 50 percent to about 90 percent and produced a reusable blueprint for privacy-first enterprise AI.
The Challenge
When I came aboard, the initial prototype had already done its job by proving the concept, but it also exposed the limit of using stock tooling against legacy enterprise knowledge. The source material was sprawling, inconsistent, and written for humans rather than machines. That left the system returning incomplete or misaligned answers too often to be trusted in production.
- Baseline system accuracy hovered around 50 percent, which was too unstable for live support use.
- Documentation contained layout-heavy pages, screenshots, flowcharts, and legacy structure that broke simple text-first ingestion paths.
- Built-in orchestration layers did not offer enough control over chunking, filtering, retrieval logic, or reranking for this domain.
The Strategic Blueprint
The architecture was rebuilt in three phases so each layer solved a concrete bottleneck before the next one was added.
Phase 1
Search Layer Overhaul. I decoupled the vector database from the stock ingestion layout, added custom metadata filtering, and tuned localized semantic indexing. That moved baseline performance from about 50 percent to roughly 75 percent.
Phase 2
Data And Vision Translation. I restructured the corpus for context efficiency, translated more than 570 screenshots and flowcharts into semantic text, and added operational Q and A material drawn from real support experience.
Phase 3
Agentic Reasoning. I iterated on prompts, response behavior, caveats, and localized verification loops so answers behaved more like a careful senior operator than a generic text generator.
Architecture Decisions
- Separate ingestion from the stock app layer so retrieval quality can be tuned directly instead of inherited from a generic interface.
- Use hybrid retrieval with metadata-aware filtering to balance semantic recall against the exact-language precision enterprise support work depends on.
- Convert visual artifacts into searchable semantic text so screenshots and workflow diagrams contribute to retrieval instead of disappearing during preprocessing.
- Keep models and data local to satisfy privacy requirements and maintain operational control over the full stack.
The Infrastructure Pivot
Privacy requirements forced the system onto local open-source inference, but the hardware target changed midstream. The project moved from a stable NVIDIA environment to a bleeding-edge AMD APU stack with immature drivers, which broke tensor execution and pushed inference into CPU fallback.
- Initial stable path: NVIDIA-based local compute.
- Migration risk: pre-release AMD APU stack with unstable driver behavior.
- Observed failure: CPU-bound fallback drove response times beyond 60 seconds per query.
- Target state: a stable local inference base that could respond in roughly 10 to 20 seconds.
Engineering Recovery
Problem
Early-stage APU drivers degraded inference so severely that the system fell back to CPU execution and crossed the one-minute mark for end-to-end responses.
Intervention
I isolated the fault chain, compiled inference binaries on the target machine, moved onto cutting-edge Linux kernels, and configured nightly graphics drivers to bypass the unstable default path.
Result
With the environment stabilized and a higher-performance C++ inference backend in place, latency dropped into a much more usable 10 to 20 second range.
Key Milestones
- System accuracy improved from an unstable 50 percent baseline to roughly 90 percent in target test scenarios.
- More than 570 screenshots and flowcharts were translated into retrieval-ready text, unlocking workflows that were previously invisible to the model.
- Hardware and inference optimization cut local response times from over 60 seconds down to about 10 to 20 seconds.
- The database and retrieval layer were structured for long-term scale, with room to support future internal knowledge bases well beyond the original corpus.
Retrospective
The main lesson from Project S is that enterprise AI reliability is usually decided by data engineering and infrastructure quality before model output style ever becomes interesting. Holding the production beta until the corpus, retrieval path, and local compute layer were stable protected user trust and prevented a fragile deployment from becoming the public definition of the system.
The resulting architecture became a repeatable pattern for private, localized knowledge retrieval that keeps organizational data boundaries intact while still delivering practical operational value.