Chatbots are everywhere, from answering customer queries on a company website or in an application to providing real-time support to employees. But as a company, how can you ensure that you develop a chatbot that aligns with your culture and meets your clients' needs? In a world full of off-the-shelf solutions, what sets apart a generic chatbot from a custom-designed one? Let’s explore why investing in a tailored solution might be the key to unlocking real value for your organization.
Nowadays Large Language Model (LLM) powered assistants are pervasive in the technological landscape and there are many different chatbots to choose from:

• ChatGPT
• Microsoft 365 Copilot
• Mistral AI's Le Chat
• Google Gemini
• ...

These are great products for specific tasks such as summarizing, translating, improving language and everybody should leverage them - paying attention to confidentiality, ethical and lawfulness issues. When a client comes to us and their use case is well served by these chatbots, then we happily recommend them to use one of these products (with optional training to help the team onboard on these new tools).
At B12 Consulting | part of Yuma, we have also developed our fair share of Large Language Model (LLM)-powered applications, including chatbots. Every project we undertake, including chatbots, is fully custom-built from the ground up, with the client’s unique requirements at the core But with so many chatbot solutions available today, why should our clients invest into building a custom assistant, instead of paying a monthly subscription to one of the existing solutions?

Why Choose a Custom Chatbot?

From our clients' experience we know that users often have needs that are not met by generic chatbots. If you share this experience, then a custom-built chatbot may be for you! Here's the key advantages that a custom chatbot brings to the table compared to off-the-shelf solutions:

• Tailored to your needs
• Seamless integration with internal data sources and documents
• Optimised retrieval and filtering for highly specific data
• Complete control over security and data privacy
• Tailored behaviour and guardrails to meet your unique organisational requirements
• Custom actions and workflows designed to fit your business processes

In this article, we will explore why a generic assistant may fall short and how a custom assistant represents a better investment for delivering added value.

Wouldn't it be great if your chatbot has access to all of your meeting notes and active Jira tickets?

Your data is likely spread across different sources, such as Google Drive, Microsoft Sharepoint and local files. Sometimes the files are even hidden behind an API like Jira tickets, Github issues, etc. Connecting your assistant to your internal knowledge base can significantly boost operational efficiency by giving teams a single point of contact to many different data sources.

Generic off the shelf chatbots like ChatGPT have no access to your internal documents, and thus it cannot answer questions about your data. You can add documents to a chat, but you need to first download them yourself. This is a suboptimal user experience that doesn't scale well.

Other chatbot products like Microsoft 365 Copilot do have access to most files in your Microsoft environment, so it is a step up from ChatGPT. However, giving Copilot access to files outside your Microsoft environment can be tricky or outright infeasible.

Because we build our applications from scratch, we can integrate with many different data sources and parse many different file types. This ensures that the assistant has access to the right data to assist you in your tasks.

Aargh, why doesn't my chatbot find the file I am looking for!?

Imagine asking your chatbot a question, but instead of getting the information you need, it’s pulling up the wrong documents. Frustrating, right? This is where retrieval optimization becomes critical.

Generic chatbots need to cater to their entire user base, and are thus limited in their ability to tune the retrieval to your specific needs. Indeed, many clients experience limitations with generic chatbots and their ability to find exactly the document you are looking for.

There is a large body of literature available to optimize retrieval, e.g. HyDE, Semantic Chunking, Agentic RAG, GraphRAG, LightRAG, etc. We will analyse your data and design a retrieval pipeline that is optimised for your data, data structure and use case. For example, we can implement special rules when retrieving relevant documents, such as filtering by date, location, etc.

Would you feel comfortable if your doctor sent all his notes of your medical appointments to ChatGPT?

Some of our clients operate with highly sensitive data that cannot leave the internal confines of the company. In such cases, even an Enterprise Agreement with a third party provider such as Azure OpenAI may not be an acceptable option.

This immediately rules out a significant portion of the available chatbots, which will send your data to third party providers such as OpenAI, Azure, Mistral, Google, etc.

B12 can set up an on premise LLM server and offer support in acquiring the necessary hardware and maintenance. This allows a perfect silo where no sensitive data leaves your machine.

Help! My chatbot just offended my biggest client!

Several news articles¹ have made front pages due to public facing chatbots that were ill-behaving. To not end up in an embarrassing situation, you should set up a carefully defined policy of how the system should behave.

Generic chatbots will have generic policies that have no or limited configuration options.

We can help define and implement a carefully designed policy that will limit the risk of undesired inputs and outputs of the system.

Why can't my chatbot book my flight for me?

Often, what our clients need is more than just a chatbot. The application should not just answer questions, but also be able to take specific actions or follow predefined steps. For example, an AI agent with the right tools can create a booking, kick off a process, draft a long report, or compare two or more documents against each other.

Generic chatbots are designed for a particular group of users and simple turn-based conversational interactions. They are not designed to handle arbitrary tools and workflows.

Together with our clients we organise workshops to deeply understand their use case and problems. This allows us to build a custom system with exactly the right tools and capabilities that fit your requirements. We find that our clients often have specific requirements that fall outside of the scope of existing chatbot offerings and this is where we can add a lot of value to you and your team.

Imagine a group chat where you, your colleagues and your AI agents are working collaboratively, all together!

Every chatbot on the market today is geared towards one-to-one conversations between you and an AI assistant. This can already add value, but ultimately limits the ability of AI assistants to help you and your team. With our Akgentic framework, B12 is building a future where humans and AI work collaboratively to complete tasks. Humans can ask AI agents for help and AI agents can ask humans for input when they need it.

To our knowledge, there is no system on the market today that comes close to realizing the vision of the Akgentic framework, which is to create true hybrid human-AI squads.

Are you satisfied with your current chatbot?

To summarise, existing offerings on the market try to appeal to a broad group of users. If you fit inside this group, that's great! These chatbots can serve you well. However, if your requirements fall outside their scope, you may find them lacking in ethics, functionality and precision.

Let’s do a quick recap of the different solutions available to you today, ordered by specialisation to your requirements:

• Basic chatbot: Easy to use and setup, but does not have access to your data
• Microsoft Copilot 365: Has access to your Microsoft data, and a limited set of external data sources through plugins. Cannot use special tools.
• Custom Rag chatbot: Can connect to any data source through custom built connectors.
• Custom Agentic chatbot: Can connect to any data source and can use tools to take actions.
• Hybrid Human-AI team: You and your team collaborate with a team of agents. Each agent has a specialized role, tuned to your specific needs. As your requirements evolve, your agentic team evolves together with you.

If your business faces challenges that generic chatbots can’t handle, a custom solution is the way forward. Let’s talk about how we can design a chatbot that meets your exact needs and helps you overcome your current limitations.

‍

¹ News articles:

• https://www.bbc.com/news/technology-68025677
• https://x.com/ChrisJBakke/status/1736533308849443121
• https://www.theguardian.com/technology/2023/feb/17/i-want-to-destroy-whatever-i-want-bings-ai-chatbot-unsettles-us-reporter