Application architecture

Last weekend, I had some free time and thought, "How can I make deck building easier?" Abruptly, I turned to the internet. But all I found were meta deck explanations and prebuilt tactical deck guides, great for competitive players, but not much help if you want to build a deck around a non meta Pokémon. Sure, I could ask the community for advice, but that takes time. That’s when I had a lightbulb moment, why not use AI to solve this problem?

I envisioned a simple system:

• A scraper to gather card data from websites

• A database to store the information

• Some retrieval mechanisms to fetch relevant data

• An MCP server to expose this data to AI agents

So, I started with something like this:

TLDR; If you don't want to read the rest of the article and simply want to look at the code, it can be accessed here. Else, let's go on!!

Scraping the cards

The first step was scraping card data. Since I play using the regional Indonesia cards, big websites like Serebii weren’t an option—some card names differ. That left me with the official Pokémon site, which thankfully provides all the necessary details. With that sorted, I could begin extracting the data!

For this task, I chose crawl4ai. It automatically chunks website sections, making it easier for AI tools to process the data—huge time-saver! Once the data was scraped, the next step was storing it in a vector database.

Embedding and storing the data

Since I wanted this tool to be accessible online, I initially leaned toward managed services. Carrying my laptop to casual TCG battles? Not cool. My database of choice is Milvus. There are other capable vector databases like chroma, qdrant, or even supabase. However, Ziliz, the managed Milvus service, is the most generous with it's free offerings. That is the deciding factor. I started with OpenAI for embedding, but reality hit fast, no free trial! That led me to explore alternatives like Gemini, Nomic, and others, but each had limitations for text embeddings. Given the large amount of scraped data, I decided to go local instead. Enter Ollama! Both Ollama and the embedding model can be easily installed with following commands:

curl -fsSL https://ollama.com/install.sh | sh
ollama pull nomic-embed-text

With the necessary embedding system and the database already set up. The next step is to process the data. To keep things simple, I started with just two collections:

• regulations (for game rules and restrictions)

• cards (for all Pokémon TCG card data)

Creating these collections programmatically was straightforward:

    schema = milvus_client.create_schema(
        auto_id=True,
        enable_dynamic_field=True,
    )
    schema.add_field(field_name="id", datatype=DataType.INT64, is_primary=True)
    schema.add_field(field_name="vector", datatype=DataType.FLOAT_VECTOR,dim=768) #3072 for gemini, 1536 for openai, 768 for ollama

    index_params = milvus_client.prepare_index_params()
    index_params.add_index(
        field_name="id",
        index_type="AUTOINDEX"
    )

    index_params.add_index(
        field_name="vector", 
        index_type="AUTOINDEX",
        metric_type="COSINE"
    )

    # Create a collection in Milvus if it doesn't exist
    if not milvus_client.has_collection(collection_name):
        milvus_client.create_collection(
            collection_name=collection_name,
            consistency_level="Strong",
            schema=schema,
            index_params=index_params,
        )

After the code executed, the initiated collections can be seen from Ziliz's dashboard

Each collection has auto-indexing (because, let’s be honest, I’m lazy 😆), and I use the COSINE metric for data relativity. The key takeaway here is that the vector dimension must match the embedding system. If I ever switch to OpenAI, I’ll need to update the schema accordingly. Embedding the data using Ollama is ridiculously simple, it’s just one line of code:

    ollama.embed(model="nomic-embed-text", input=text).embeddings[0]

Bringing the MCP server to life

With everything now standardized, at least in the Python world, spinning up an MCP server is super fast. Thanks to libraries like FastMCP, it takes just a few lines of code to get things up and running:

    # Initialize FastMCP server
    mcp = FastMCP(
        "pokebuilder",
        description="MCP server for Pokemon TCG deck building",
        host=os.getenv("MCP_HOST", "0.0.0.0"),
        port=os.getenv("MCP_PORT", "8051")
    )

And just like that, the MCP server is live! 🎉 Now that the server is up and running, the next step is to build a RAG system to fetch relevant data from the database. This will be exposed as an MCP tool, allowing an AI agent to use it for deck building assistance. For this initial version, the regulations are static, but in the future, I might add features to help build competitive tournament decks—that’s a project for another day! When it comes to selecting relevant cards, the idea is simple:

• The user prompt is passed as a query to the database.

• The system finds relevant cards based on keywords.

• Since most competitive decks use around 15 different cards, I set the query limit to 30 cards, giving some extra flexibility.

Here’s how the query works:

    # Get the context for cards selection
    card_result = milvus_client.search(
        collection_name="cards",
        data=[emb_ollama(query)],
        limit=30,
        search_params={"metric_type": "COSINE", "params": {}},
        output_fields=["text"],
    )
    card_distance = [
        (res["entity"]["text"], res["distance"]) for res in card_result[0]
    ]
    card_context = "\n".join(
        [line_with_distance[0] for line_with_distance in card_distance]
    )

Once the query results are retrieved, I combine the card text based on relevance (distance metric) into a single formatted string. This ensures the information is structured properly for AI processing. Now, it’s time to generate the user prompt! This prompt structure ensures the AI understands the deck-building rules, available cards, and user preferences:

    # Generate the user prompt
    user_prompt = f"""
    Use the following pieces of deck building and battle information enclosed in <regulations> tags to build a pokemon deck with a selection of cards enclosed in <> tags using parameters enclosed in <query> tags.
    <regulations>
    {regulation_context}
    </regulations>
    <cards>
    {card_context}
    </cards>
    <query>
    {query}
    </query>
    """

Now that everything is set up, it’s time for the testing, whether the MCP server is running correctly! Thankfully, FastMCP comes with a built-in MCP inspector tool, making debugging super easy. To check if everything is working, simply run:

    mcp dev src/mcp_server.py

Once the inspector tool is running, you can access it at http://localhost:6274/#tools

Using this tool, we can test the MCP server tools and prompts. Just make sure that the transport type is set to SSE and that it points to the correct URL.

Setting up the UI

Since I plan to deploy this application online, I need a web-based UI. That led me to openwebui, a fantastic web-based interface that can run anywhere! The good news? It’s super flexible! The bad news? It doesn’t support SSE based MCP servers natively. But don’t worry, this is easily fixable! All we need is a proxy in front of our MCP server. For this, I used an openwebui extension called mcpo, which can be installed globally using pipx. To run the mcpo server and target our MCP, simply use:

    mcpo --port 8000 --server-type "sse" -- http://localhost:8051/sse

While Ollama works great locally, I want to access this from my phone. That means the MCP server needs to process prompts using a cloud machine (or maybe I’ll host it in my local lab—still undecided!). For now, I’m keeping things simple, so Gemini it is!

Setting up Gemini with OpenWebUI

First, we need to get the Gemini API key to integrate it with OpenWebUI.

Then, configure the UI to list Gemini as its backend:

• Access the settings and go to Connections

• Add Gemini backend URL https://generativelanguage.googleapis.com/v1beta

• Add the token

• Manually add the model ID, for example gemini-2.0-flash

Connecting OpenWebUI to the MCP

Now, let’s connect the UI to our MCP server:

• Access the settings and go to Tools

• Input the mcpo running proxy URL as the target, by default it is http://localhost:8000/openapi.json

Time to build our deck

With everything set up, we can finally start building our Pokémon TCG deck!

For now, the results feel a bit sketchy, but tweaking the parameters should help refine things. Interestingly, running the prompt directly as a RAG application instead of through MCP yields better results with the current setup, something that still puzzles me. That’s my homework for now! The next step? Deploying this in the cloud—but that’s a project for another weekend. 😆

What differs from IC to Manager?

Let’s start with a confession: looking back at my childhood, I’ve always been drawn to the secondary characters, the ones thrust into leadership roles, not by choice, but by circumstance. Think Locke from Final Fantasy VI or Mat from The Wheel of Time. Ironically, life decided to play out the same trope for me. I started as a game developer, passionately crafting physics equations, debugging code, and dreaming up clever game logic. It was a blast! But fast-forward a few years, and the company decided my "communicative, organized, process-oriented" self was ripe for promotion. And just like that, I was a manager. The transition was rocky. As an IC, I had my groove; just me, my code, and my coffee. Lots of coffee. As a manager? Suddenly, the stakes were different. It wasn't just about me anymore; it was about the team. Success was now measured not in lines of code but in team spirit, strategy, and growth. At first, it felt like trading the thrill of scoring points for the quiet satisfaction of assisting my team to shine. And you know what? Over time, I realized I grew into it.

Manager's life

Another thing that I learned the hard way when I first became a manager was that you are expected to wear multiple hats all at once. Especially when you are working in a startup. Constant juggling between horizontal and vertical communication, deciding which fires to put out and which can smolder for a bit, guiding team members in their growth and career paths, handling conflicts, or championing the team’s needs and achievements to leadership. You're expected to juggle like a seasoned circus performer. A manager's worth is measured when they can bring impact to the company. Mastering soft skills like empathy, conflict resolution, and strategic thinking is a must for a manager to grow and make sure they are being seen.

Truth bomb: a manager’s worth often gets tangled up with the team’s output. It’s a little like basketball, you’ve gone from being a flashy shooting guard to a selfless point guard. Your job isn’t to rack up points but to make sure the ball spins smoothly. Challenging? Totally. Rewarding? Absolutely.

Coding or no coding, that is the question

Like I said before, I miss my coding days. Nothing beats the satisfaction of building something from scratch. The reality being a manager though, your time would be spent more on the people instead of the code itself. Meetings and 1 on 1 are the constants, instead of the variables. In most cases, growing as a manager means broader scope, more responsibility, and more influence; eventually reaching executives’ position. This is where the trap lies. When you step on the managerial ladder, your focus will be on business impact, team building, and growing influence. While on the other hand, as engineering manager, you are expected to excel at engineering!! At first, it's still manageable, but slowly when the company grows, or you grow, it became harder and harder to find the time to code again.

In my case, I still dabbled in technical meetings, PRs and designing system architecture. This allows me to stay connected with the codebase and keeping up with the product development. Spoiler ahead: it's not enough. The reality is it's easy to lose your coding groove. Early on, I found myself lagging behind, struggling with new features, new codes, and submitting PRs that were embarrassingly sloppy. Cue existential crisis: “Am I still an engineer at heart?”. With slow realization, I knew that I started to be left behind.

Then came mistake number two, I code again as a manager. You know where it goes wrong? Yes, trying to code like a full-on player-coach! Bad idea. Being a manager and programmer means more working time. I committed more time to code new feature while growing and coaching the team. The result was disastrous, juggling coding and managerial tasks not only burned me out but slowed the team’s progress. It took a while until I can find the balance again.

Lesson learned, I’m not here to score the three pointers, I’m here to assist. I need to be like Steve Nash. The top priority is ~passing the ball~ managerial tasks. Neglecting that means failure, not only for me but for the team as well. I don't need to always code. The sweet spot? Tackling occasional technical challenges that excite me performance tuning, bootstrapping new features, or fixing pesky bugs. And for staying sharp? Side projects are my sanctuary. I finally found the fun again.

Any regrets so far?

Do I regret becoming a manager? Honestly? Not really. Sure, there have been moments of sleepless nights over decisions. But every challenge has shaped me to be a better person of myself. The learning curves, the laughter shared over coffee breaks, and the pride in watching my team flourish far outweigh any drawbacks. If I were offered a chance to rewind time, would I choose this path again? Probably. Well, just like what my favorite fictional characters said, “What’s life like if you don’t take a chance now and then?”.

With all the AI buzzing left and right, it's hard not to jump on the bandwagon. ChatGPT, Copilot, Gemini... so many choices out there! For the general masses, ChatGPT is more than enough to cover the needs.

Now for the tricky part. Sometimes, we need AI help to handle big documents, brainstorm with the right context, or code with a specialized AI model. Sure, you can copy-paste information as a prompt or use RAG to feed data to the LLM backend. But these easy ways come at a price—your prompt becomes part of the AI model's training data, which can be a BIG problem if it contains important information. This is where local LLMs come to the rescue.

Enter the Local LLM

What is Local LLM? It's basically a large language model running on your machine instead of the cloud. It offers benefits like offline capability and more security, though it may be slower and less accurate unless you have a powerful machine.

LocalLLM setup

My favorite app to run this is Msty. It supports running local models on a CPU (if you have enough memory) and offers GPU acceleration. There are other popular options like LMStudio or llama.cpp, which offer more flexibility, but I prefer a nice UI, and Msty provides just that.

My setup is quite simple, I installed one of the most popular models from HuggingFace, Meta's LLama 3.1 8B for general use. I use Codegemma for code related stuffs, and Mixed Breed Embed Large for RAGs. Those three models alrady covers almost all of my use case.

Use cases

Aside from work, I often use my laptop to write blogs like this or do a bit of coding. My AI use cases also revolve around that. Brainstorming or doing general tasks is straightforward through the prompt UI.

One feature I find super useful for more contextual results is the knowledge stack. It consumes the document we provide through the RAG plugin, tokenizes the content, and stores it in a vector database. This way, I can ask the agent for information from manuals/books without needing to read or search for keywords. Trust me, it saves hours!!

For code assistance, I use a VSCode extension called Continue. Even though it's rarely used, I find it quite helpful to analyze and review proprietary code without breaking any NDA since everything runs locally.

Continue integration with Visual Studio Code

Step 1: Install the Model

First things first, let's get that model installed in Msty. It's super straightforward! Just download it from the Local AI Models. Personally, I love using two models for my coding needs: codegemma:7B for general code completion and starcoder2:7b for tab completion. After you've got those models, make sure your local server is up and running in the background—it usually runs on port 10000 by default.

Step 2: Install the VSCode Extension

Next up, let's get the extension from the VSCode marketplace. Once you've finished installing it, it's time to configure the remote target through the config.json file. You can find all the nitty-gritty details here:

Just access the file settings from the Continue chat tab context menu.

Step 3: Fine-Tuning and Tweaks

Both Local LLMs and cloud AI have their own strengths and weaknesses. The choice between the two depends on your specific needs and priorities. If local context, data privacy, and customization are important, go for local LLMs. However, for versatility, speed, and broad applicability, cloud options like ChatGPT are excellent choices.

Ultimately, the best solution often lies in a hybrid approach, leveraging the strengths of both local LLMs and global models to meet diverse AI requirements effectively.

Conclusion

Both Local LLMs and cloud AI have their own strengths and weaknesses. The choice between the two depends on your specific needs and priorities. If local context, data privacy, and customization are important, go for local LLMs. However, for versatility, speed, and broad applicability, cloud options like ChatGPT are excellent choices.

Ultimately, the best solution often lies in a hybrid approach, leveraging the strengths of both local LLMs and global models to meet diverse AI requirements effectively.