GenAI Blog Series #1 – Build it or Buy it

Its the first question that should come to mind. The decision you make here will transform everything you do after.

And not matter what decision you make, you should also be aware, things are expensive in the AI world. Whether you Buy It/Rent it or Build it, its going to cost A LOT.

Option 1: Buy It/Rent It

There are many options out there to choose from. Each comes with some kind of advantage/disadvantage. Some of these include:

Simple knowledge management agents:
- Microsoft Copilot / Copilot Studio ($30/user/month)
- OpenAI – ChatGPT
SaaS offerings allowing varying levels of customizations and UI friendliness, here are some options in no particular preferenced order:
- Google Vertex AI
- Azure AI Foundry / Prompt Flow
- Amazon Bedrock
- Stack-ai.com
- Relevanceai.com
- CrewAI
- Praison.AI
- Flowiseai.com
- Abacas.ai
- Humanloop
- Klu.ai
- Vishwa.ai
- LangTail
- Tune.app

As you can see from the list above, there are many companies striving to build platforms and SaaS based applications so you don’t have too. However, there are some issues you should consider if you decide to explore these paths, here are just a few:

Features and functionality – if you were to sign up and look at each of the example above, you will find a large set of differences between them and what they feel is important for their customer bases. Your ability to influence them to add features that are important to you, may not be possible or could take a long time to get into their roadmap.
Model support – Some platforms will be solely targeted at the models they are financially invested in or contractually set to use (hosted on AWS, GCP, Azure). Not to say that some platforms won’t allow you to use models in other places, but how difficult will it be to plug your fancy new model into the platform?
Tools – The orchestration layer is probably going to be built into the target platform. It could be LangChain, Semantic Kernel or some other home grown custom one with a high probability that you cannot replace it with your own. The ability to add an agent and its necessary tools is a very important aspect to the system. And being that tools can have tools, the ability for you to have a management UI that allows you to configure these tools and sub tools is a vital part of the solution. Without this, you will be relegated to whatever out of the box agent and tools they give you.
Agent and Tools Extensibility – a topic so important, it is separate from the Tools conversation! Having a set of agent types you can select from is cool and all, but only have a static set of tools to select from is not. You should be able to add your own Agents and Tools. Agents should be able to utilize any tools you load into the system. The only issue you’ll run into, is how do you route these tools?
Router Workflow – Simply calling an agent and getting a single response is pretty…simple. But what if you have multiple agents, or agents with multiple tools? How do you route to these? What if more than one agent or tool needs to respond? Router design becomes a critical part of the system (if you are going past a simple GenAI chat bot anyway). Can you do these types of flows with a Buy It system?
Support – Will it cost you extra? Do they respond quickly enough?
Source Code – Is it open source, or is it closed? Will the company stick around long enough to succeed in the market or will you need a code escrow?

Option 2: Build it

So you have explored the Buy It/Rent It options, and you have come to the conclusion that you need to build it on your own. Understand that the work you are about to undertake will be quite involved and a massive effort.

Resources will be needed.

You are going to need some skilled AI/ML folks (at least two), you will need some front end folks (again, at least two). You will need middleware and backend folks (another couple). Gotta throw in the DevOps team, made up of another 2-3 individuals. Don’t forget, project managers and some testers. So right off the start, you’ll need 12-14 people to Build It. Figure around 100K average per person, you are sitting at $1.2-$1.4m/year just for the GenAI team. Some other roles that I didn’t include here that you likely already have: security and privacy, compliance team members. However, you’d need to allocate some of their time to review any solutions proposed.

This does NOT include the hosting costs, which we will get too in a later post.

The challenge is the same with pretty much every company. Budgets are stretched, you already have competing priorities and projects, legacy applications and processes to support. Teams tend to be overwhelmed with what they already have on their plates. Although most people will be happy to learn more about AI and how to use it and incorporate it, they will need to find the time to do so. AI can help with that…Microsoft Copilot could be that solution to increase productivity and allow them the time to build custom AI solutions. But now we have AI eating AI but hey…why not?

In all seriousness, this blog series will enlighten you to just how hard it is to get a GenAI to a successful end state. It is very likely you will need to bring in some talent that has done this a few times and knows how to keep you out of trouble. Taking classes, watching some videos and doing getting simple POC examples from github up and running does not a GenAI expert make (Yoda, 2025).

This means hitting the recruit staff with some job descriptions and getting them out onto the job boards. I see GenAI jobs popping up everywhere as companies realize, they need help and can’t do it on their own. You should be really specific about what you are looking for. Say what tech stack you are targeting (reference the next section), what hypervisor you use (Azure, AWS, GCP), and be specific that you are looking for GenAI versus Machine Learning. The two are very different things and a GenAI person is not always skilled at building neural networks to train multi-billion parameters models with costly GPUs and nor is a machine learning expert likely skilled at building a GenAI platform.

Components to build.

Now, you won’t completely have to start from scratch, there is plenty of examples of various bits and pieces scattered across GitHub. The real decisions will center in on the main components and what tech stack they will run on and how you will ultimately host them. For example:

Front end (User and Management Portals) – Dotnet, Vue, Node, React, etc. Ultimately these will turn into containers and be hosted somewhere, so as long as your front end folks are comfortable with the tech, you’ll probably be good here. The only nuance, will you need to support accessibility? This is not as easy as one would think and requires quite a bit of tedious work, but required if you are in a country/region/industry that mandates it.
API/Core/Orchestration – What will you write this in? Java? DotNet? Python? How will you support
Gateway layer – build your own API Management layer to offset some costs?
Gatekeeper layer – how will you do prompt attack prevention and sensitive information filtering?
Database – CosmosDB, PostgresSQL
Vector Database – Azure AI Search, CosmosDB, Pine, etc
Agentic Frameworks – Semantic Kernel or LangChain? Will you only support one, or many? Allow bring your own customization? How will that plug into the system?
Security – At what layers will you build the security? What objects and scopes will be secured? Can you build it in a way that scales?
Monitoring – Keeping track of the usage of the system. This includes its performance both in terms of latency and accuracy will be a continual battle. You will need queries and dashboards to help show how the system is running.
Failover/Recovery – If your data center or hosting provider goes down, how business critical is it to get the system back up and running? Can you do a full regional failover (in Azure, from East US2 to Sweden?).

Example

Chris O’Brien has a really nicely written up blog piece that fits into this topic of Buy It or Rent it. It does a good job of pointing out the pros and cons of the two possible paths as well as a focus on the costs involved. The use case is pretty specific, but it still re-iterates the points made here.

Its interesting to note that many of these GenAI SaaS providers (Buy It/Rent It) are still in price discovery mode. They probably didn’t have a good grip on the backend costs of their solution and as they started to scale up, they realized they may have priced too low. Conversely, those that have found efficiencies or realized the value they provide is becoming more of a commodity are starting to bring their prices down to match new market competitors. This alone makes the argument that you should always be evaluating your options are and what “exit strategy” you have for getting off one provider to move to another.

Summary

It may not be obvious which path is the best path to start with. If you go it alone, its going to take some experimentation, mistake making and some lessons learned. I’d encourage you to reach out to folks in the community (like myself or other AI MVPs) or AI conferences and learn from those that have done this before. There really is no reason to reinvent the wheel or make the same mistakes others have.

As you can see from the above, neither choice presents an obvious or easy path to solve your management or stockholders push to use AI in the business. What is obvious, is that GenAI presents opportunities for innovation and business solutions that were not remotely possible before. So its not really a matter of “should we” do it, but “when and how”.

Contact

If you are looking for some help in making your decision, or have questions about anything I mention in this blog series, feel free to reach out anytime, and definitely check out FoundationaLLM!

Email: givenscj@hotmail.com
Twitter: @givenscj
LinkedIn: http://linkedin.com/in/givenscj