AI: Navigating the limits of Large Language Models

Eilon Reshef at Gong explains how to get the most from generative artificial intelligence

Nearly two years on from ChatGPT bursting onto the scene, most companies have at least experimented with generative AI (genAI). But for many business users, the novelty of general-purpose large language models (LLMs) has worn off slightly, as an urgency to prove its value in the tech stack has set in. 

General-purpose LLMs captured the world’s attention because of their versatility and capability to generate content across diverse domains. They can generate bedtime stories, and blog posts on sovereign cloud or summarise lengthy documents with aplomb. However, for companies looking for them to help with more specialised tasks, their ability to address domain-specific challenges leaves much to be desired, as two New York lawyers discovered the hard way.

Instances like these have companies understandably tempering how they invest in genAI, but they shouldn’t scare businesses off entirely. There are still ways companies can embed genAI into workflows to drive value—they just need to think…smaller. 

LLMs are just a piece of the puzzle

While LLMs can provide help authoring emails and documents, summarising materials and more, these tasks do not exist in a vacuum. In business settings, they are usually completed with some context in mind.

For example, an email summary of a customer status might involve information from multiple sources, such as a Customer Relationship Management (CRM) system, a Conversation Intelligence tool or context from previous correspondence. Workers cannot be expected to efficiently copy all of this data and text manually into an LLM dialog window and paste the output where it’s needed.

More often than not, the most effective applications of genAI are embedded directly into a specific, existing workflow, provided by a specialised software vendor. For example, one that helps summarise the status of sales must be integrated into the system sales teams use to review their pipeline and forecast revenue. Or another that authors emails to new prospects would need to connect with their sales engagement solution.

Bigger models are not always better

While large language models such as ChatGPT and Claude grab significant media attention, they are expensive and lack context and specificity.

In contrast, smaller, domain-specific language models are genAI models tailored to individual industries and use-cases. These can include analysing sales calls to improve sales reps’ win rates, writing financial reports or accurately citing case law for court documents.

With significantly fewer parameters as they are fine-tuned using relevant information, these models require less computing power to run, and users benefit from improved accuracy and reduced latency. In fact, these models can cost one-tenth or less of what larger models do to maintain.

Domain-specific models often provide more accuracy too, as by being trained on industry or use-case data, they can deliver more precise - and valuable - results for users. After all, few businesses need a model that can recite the works of Shakespeare, but many could benefit from a model that closely understands their customers’ key pain points and objectives and uses this understanding to generate appropriate next steps or responses. For example, “let’s go” could mean “let’s sign a deal” in a sales context but “let’s leave this party” in others.

That’s one of the major reasons businesses need to look beyond LLMs’ size and power when evaluating their options. A smaller but highly specific generative AI model will not only be less resource intensive but also more tailored to solve the problems it’s being integrated to address. Chances are, it’s more valuable to the organisation hungry to prove the value of its generative AI investments.

The limits of doing it yourself

So if a company wants to turn to a more specialised model for its generative AI workloads, what are its options? Does it set out to find engineering and data science talent to build a model itself, or does it turn to an existing vendor? 

Building a generative AI model in-house is unlikely to deliver the returns that some of the existing tools on the market have already honed. Any company with aspirations of doing things entirely on its own will be held back by three factors.

First, a business would need to build the integrations required to obtain the data needed for the LLM to run in the first place, and connect to the right IT systems so it knows where to pull information from. Then, it would need to design specialised workflows to make it easy for employees to work alongside genAI.

For example, if a company used the technology to create documents from scratch, there would need to be a mechanism that allows the right workers to collaborate on them and continue editing. Or, if one were to generate a customer summary, there would need to be a process for communicating it to the relevant people.

Finally, to yield accuracy and performance, companies would need to train their domain-specific models themselves. In practice, the data available at hand is usually limited within the confines of a single entity, as is the talent capable of fine-tuning a genAI model.

A large company may generate hundreds of data points a day during normal operations, but even that doesn’t compare to solution providers with robust, targeted and contextualised troves of data that can be used to train their models. 

How to choose a genAI product that’s right for you

Companies that offer domain-specific genAI capabilities are appearing across almost every category. Businesses wanting to implement genAI themselves should start with the problem or inefficiency they’re looking to solve and explore the solutions that address their specific challenges.

This element of specificity is the key to getting the value from genAI that so many envisioned when it first hit the market. A smaller model that is trained specifically on parameters that map to the problem it has been designed to solve is going to produce far better outputs than one that has also been trained on less relevant data.    

General-purpose LLMs will continue to be powerful tools for generating potential blog headlines, shopping lists, or TL;DR summaries of long documents because that’s what they’re best suited for. But if a company needs to address the needs of a specific department, thinking smaller will lead to outsized results. 

Eilon Reshef is a co-founder and Chief Product Officer at the revenue intelligence platform Gong

Main image courtesy of iStockPhoto.com and amgun