Building a production‑ready Document Portal with LLMOps (Part-2)
Document Portal Application: Laying the Groundwork
Welcome back! 👋 In our previous post, we introduced our Document Portal's high-level architecture, tools, and frameworks. Now, are you ready to get from theory to practice? It's time to start building the core components that will form the backbone of our application.
In this installment, we'll dive into the essential scaffolding that any production-grade application needs. We'll cover three critical steps that will empower you to create a secure, flexible, and robust system:
Configure Your System: See how a single configuration file can set up your Vector Database, Embedding models, and LLM APIs for inferencing.
Set Up Core Utilities: Master consistent logging and robust exception handling for your application, which are crucial for debugging and maintenance.
Build a Front-End: Learn how to utilize ChatGPT to vibe code your front-end for the application using one-shot prompting.
About the Author:
Arun Subramanian: Arun is a Senior Data Scientist at Amazon with over 13 years of AIML experience and is skilled in crafting strategic analytics roadmap, nurturing talent, collaborating with cross-functional teams, building robust scalable solutions and communicating complex insights to diverse stakeholders.
Configure Your System
A professional application demands a flexible and secure way to manage settings and dependencies. Instead of hardcoding API keys and model names, we centralize this information in a configuration file and create dedicated modules to load it.
Our system starts with config/config.yaml, a simple YAML file that defines all our key parameters. This includes settings for the Vector Database (faiss_db), the embedding model (provider:"google", model_name: "models/text-embedding-004"), and the LLM models for different providers like Google and Groq ("gemini-2.0-flash" from Google and "deepseek-r1-distill-llama-70b" from Groq).
The utils/config_loader.py module's job is to reliably read this file into our application. It handles path resolution, ensuring the configuration file is found whether you run the application from its root directory or from a nested folder. This makes our project portable and easy to run in any environment.
Next, the utils/model_loader.py module acts as your command center for loading models and managing secrets. It's a key part of the scaffolding that makes your application a "production-ready" LLM system.
Depending on whether the app is being run locally or in AWS cloud, the ModelLoader() class loads the LLM API Keys from either the local .env file or from AWS Secrets Manager. This provides a flexible and secure way to manage credentials in both local and production environments.
The ModelLoader contains two key methods:
load_embeddings(): This method uses the settings fromconfig.yamlto load the correct embedding model, such as theGoogleGenerativeAIEmbeddingsmodel.load_llm(): This function is a factory that dynamically loads the appropriate LLM based on theLLM_PROVIDERenvironment variable and the configurations inconfig.yaml. It can seamlessly switch between models from providers like Google and Groq, abstracting away the specifics of each provider's API.
Logging and Exception Handling: Building a Reliable System
For an application to be reliable, you need a way to monitor its behavior and handle unexpected errors gracefully. Our project achieves this through a centralized logging and exception handling system.
Logging
The logger/custom_logger.py module sets up a structured logging system that outputs logs to both the console and a timestamped file. Unlike basic print statements, this module uses structlog, a library that formats log entries as JSON. This is a crucial practice for production applications, as structured logs are easy for cloud services like AWS CloudWatch to parse and analyze, making debugging and monitoring much more efficient.
Exception Handling
Robust applications handle errors gracefully. The exception/custom_exception.py module defines a custom exception class, DocumentPortalException, that centralizes all error handling. This class simplifies error reporting by normalizing error messages and automatically capturing detailed traceback information. When an error occurs, our custom exception captures the exact file name and line number where the issue originated, providing developers with an immediate and clear view of what went wrong.
Vibe coding a Front-End for your application
Let's be real—most data scientists and machine learning engineers aren't professional front-end developers. We're great with Python and data, but crafting beautiful, responsive UIs is often a different ballgame. This is where the power of LLMs like ChatGPT comes in. You don't have to be an expert in HTML or CSS to build a functional and visually appealing front-end. You just need to know how to ask the right questions.
Using a detailed prompt, we can instruct an LLM to generate the entire front-end of our application for us. This saves a huge amount of time and effort, allowing us to focus on the core LLM logic we're building on the back-end.
The Prompt: Your Blueprint for the UI
The key to getting a good result from a generative AI is to be as specific as possible. Our prompt acts as a detailed blueprint for the UI, outlining everything from the layout to the design aesthetics.
Our prompt specified:
Semantic Structure: Use
<header>,<main>, and<footer>for a clean, modern layout.Key Components: The UI needs a title, a logo, and three distinct tabs for “Document Analysis”, “Document Compare”, and “Doc Chat”.
Functionality: Each tab should have a form for file uploads and specific buttons for actions like "Run Analysis" or "Build / Update Index".
Visual Style: We asked for a modern dark theme with gradients, shadows, and a clean font like Inter. We also requested hover effects for buttons.
This level of detail gives the LLM everything it needs to generate high-quality, functional code. The LLM delivered on the request, providing a complete templates/index.html and a static/style.css file. The index.html file creates the three-tab interface, with each tab containing a card section for its specific functionality. It includes all the necessary form fields, buttons, and even validation logic to ensure a smooth user experience. The style.css file brings the design to life, implementing the requested dark theme, gradients, and subtle shadows to give the app a polished feel.
This process demonstrates how you can effectively use AI as a co-pilot for tasks outside your primary expertise. You can get a robust, good-looking UI ready in minutes, allowing you to focus on what you do best: building the intelligent back-end. The prompt I used is in chatgpt_ui/chatgpt_prompt.txt.
Conclusion
In this installment, we built the scaffolding for our Document Portal application. You learned how to use a structured config file to manage all our application settings, from model names to API keys. We then saw how to securely load these configurations and dynamically instantiate the correct LLM and embedding models.
We also implemented a robust logging system, which leverages structlog to produce machine-readable, JSON-formatted logs that are easy to analyze. Then we created a centralized exception handling module that handles errors gracefully by capturing detailed traceback information and providing clear, actionable error messages. This is crucial for debugging and maintaining a reliable application in production.
By building this core scaffolding, we've created a system that is modular, extensible, and ready for the real work to begin. We've moved beyond the blueprint and set up the robust framework that will support our intelligent application.
In the next blog post, we'll dive into the heart of the Document Portal: the Data Ingestion process. You will learn how we handle different file types, split documents into manageable chunks, and use our embedding model to transform text into vectors. We'll then store these vectors in our FAISS vector database, making our documents ready for powerful semantic search and analysis. Get ready to turn raw data into intelligent, searchable information!✨
Yup, it is public
[found the code https://github.com/ArunSubramanian456/document_portal ]
Hi Arun, Is the code base behind this project publicly available? I want to play with it