Agentic AI
Why does the model or (models) used in your agentic AI workflows matter? In this article we explain why the choice of models is key for a successful agentic AI strategy–and why small language models (SLMs) are the right choice.
.jpg)
Historically, in the machine learning/AI industry, there has been a perception that the bigger a model is, the more capable it is. This led to companies generally seeking to use the largest models they could find.
But in today’s world, where models have surpassed trillions of parameters, using the largest models often leads to out-of-control costs and scaling issues. Also, large models are very difficult to train for specialized tasks. These challenges become even more insurmountable when companies try to incorporate them into an agentic AI solution–especially for tasks that require highly-specialized knowledge.
Here at Arcee AI, we’re the industry leaders in small language models (SLMs). We’ve pioneered some of the top techniques (model merging, distillation, Spectrum) for training small models that compete with, and often outperform, their large counterparts. With the rise of agentic AI, we knew that we were perfectly positioned to provide the ideal models to power this technology. So we built an end-to-end agentic AI platform, Arcee Orchestra, that combines our SLMs with intelligent model routing and orchestration–enabling you to use the right model for the right tasks. This approach reduces costs, minimizes latency, and maintains or even improves accuracy.
For most businesses, your teams sometimes do need larger models. But they certainly do not need the biggest, most expensive LLM for routine work like summarizing meeting transcripts or generating follow-up emails. By offering a variety of model sizes, and intelligently routing your tasks to the right model, Orchestra dramatically reduces your cost without sacrificing performance.
We also realized that for any agentic AI system to be valuable, it needed to be able to integrate with existing systems. We built Arcee Orchestra to include over 200 pre-built integrations to common applications like Salesforce, Slack, Dropbox, MS Office, GSuite, and others. This means that you don't have to be the integrator (which is often one of the most costly barriers to successful agentic AI deployments). You can use the integrations directly out-of-the-box without any custom code.
The fully integrated nature of Arcee Orchestra makes it incredibly easy not only to get started, but also to maintain and evolve your workflows. One development challenge that many customers don’t consider until it’s too late is that when models change, the optimal way to integrate and interact with them also changes. An AI agent that performs perfectly today may not work as well tomorrow, if a new model comes out and is integrated into the agent. With Arcee Orchestra, this complexity is abstracted away, and you get to utilize SOTA models without having to worry about a new model not working in your solution.
Arcee Orchestra comes with six SLMs out-of-the-box:
What all of these models have in common: we put them through a highly-specific training process to make them excel at automated workflows and agentic AI systems.
Let’s take a closer look at some of what we considered when training these models specifically for agentic AI.
When you have different providers for models and frameworks/platforms, there’s always an inherent risk that performance will suffer, since the two components were not built to work together. Also, different models require different prompting strategies, which means that–in order to get various models and platforms working well together–you need expertise in all of them. This is expensive in terms of labor and time-to-value.
Large third-party models are trained to excel at satisfying the (human) end user, by being conversational and having the ability to answer any query, such as “Create a song from the perspective of a giraffe.” These types of capabilities are great for consumers… but they increase the size of the model, and add nothing to what companies actually need the model to do for their business workflows.
The Arcee AI SLMs that power Orchestra provide the requisite expertise to enable the platform to interface smoothly with a diversity of models and platforms, and to be able to execute business tasks. Our SLMs are carefully fine-tuned to be highly capable at instruction-following and function calling. Additionally, they understand the nuances of API inputs and outputs, which dramatically improves the accuracy of AI agents.
The models in Arcee Orchestra have been specially trained for agentic AI, which is what makes them so good at it. And with Orchestra, you get access to multiple models, with intelligent routing so that the right model is used for each task. But .. that’s still not all. Our SLMs also solve one of the other major challenges of working with third-party, closed-source LLMs: security and compliance.
Thanks to their relatively compact size, SLMs can be run in your own environment. One of the largest blockers we hear from customers is that they can’t utilize models hosted in another provider’s environment due to regulatory and compliance requirements.
Since SLMs can run efficiently on smaller servers, Arcee Orchestra–including all Arcee AI models–can be deployed in your own environment, utilizing your security controls. Your requests and your data never leave your security perimeter. You get to decide how the models are used, what data they have access to, and who can access them. As you can imagine, this is music to the ears of your CISO and other security experts and data governance experts.
We would be remiss if we didn’t also mention that Orchestra can include one or more SLMs that have been fine-tuned on data of your choice: your company’s proprietary data, industry data, data specific to a certain use case, etc.
Interestingly, we’ve found that in most cases where a company thinks they need a fine-tuned model, they can in fact solve their task with a well-built workflow–which ultimately is the most cost-effective solution. There are certainly some cases where we do recommend fine-tuning our SLMs for your needs; examples of this are when models are required to have a deeper understanding of a specific domain, or when they need to change how they perform a specific task. And needless to say, this process is much more affordable and effective compared to fine-tuning an LLM.
We’d love to hear your questions about Arcee Orchestra, agentic AI, and SLMs. Hit us up here and we’ll get a time on the books!
Agentic AI refers to AI systems designed to autonomously perform tasks and make decisions with minimal human intervention. Unlike traditional AI, which is often optimized for conversational purposes, agentic AI focuses on executing specific business workflows, handling operations, and integrating with systems to drive efficiency. The goal of agentic AI is to automate routine tasks, such as summarizing meeting transcripts or generating follow-up emails, while ensuring high precision and low cost.
Agentic AI workflows streamline business operations by automating repetitive tasks and minimizing the need for human involvement. By leveraging intelligent task routing and specialized models, agentic AI platforms like Arcee Orchestra allow businesses to reduce costs, optimize performance, and maintain accuracy. These workflows can integrate without disruption to their existing infrastructure.
Arcee Orchestra is an advanced agentic AI platform that uses small language models (SLMs) for task automation and integration. Unlike other platforms that rely on large, costly language models, Arcee Orchestra leverages SLMs that are optimized for specific agentic AI workflows, ensuring faster, more efficient performance while reducing operational costs. With over 200 pre-built integrations, the platform easily integrates with popular tools like Salesforce, Slack, and MS Office, making it simple for businesses to adopt and scale agentic AI.
Arcee Orchestra is designed to intelligently route tasks to the most appropriate model, ensuring that each task is handled by the right AI agent. This optimization of agentic AI workflows reduces costs and latency while maintaining or improving accuracy. The platform uses a variety of pre-trained models fine-tuned to excel at specific business tasks, ensuring that each workflow operates at peak efficiency.
Yes, Arcee Orchestra seamlessly integrates with over 200 popular business applications. These integrations eliminate the need for custom code and minimize the challenges typically associated with adopting new agentic AI platforms. With Arcee Orchestra, businesses can easily implement and scale agentic AI workflows without disruption to their existing infrastructure.
Arcee Orchestra’s agentic AI platform offers strong security and compliance features. Since the platform is designed to run on smaller, efficient small language models (SLMs), it can be deployed within your own environment, allowing you to retain full control over your data and security. This setup ensures that your sensitive information never leaves your organization’s perimeter, making it an ideal solution for businesses concerned with data privacy and regulatory compliance.
When we first show our agentic AI platform to businesses, the most common question we hear is, “Why would I need agentic AI that uses your models? I’m accustomed to using Claude [or insert any other popular LLM], and I don’t think a smaller model would work as well.”
The thing is, just like you, we also think the LLMs (large language models) on the market are pretty incredible. These models are great to chat with because they are designed to interact with you, a human being. Yet the whole point of agentic AI is to get computers and machines doing work with minimal human intervention.
This is where Arcee AI’s small language models (SLMs) stand out from the rest.
While other models are optimized for back-and-forth discussion, our models are built to excel within automated systems.
They’re trained to follow instructions precisely, and they have a deep understanding of API data. Here’s just one example: our 72B general purpose model, Virtuoso Large, outperforms the largest LLMs (estimated to be over 1.3 trillion parameters) on instruction following benchmarks like IFEval.
In other words, the SLMs that power our agentic AI platform, Arcee Orchestra, are masters at getting stuff done–and they’re the ideal technology to power agentic AI.
When it comes to the world of language models and Gen AI, a key question for companies looking to adopt these innovations is which model(s) to use. As if it’s not already complicated enough with the plethora of foundational models out there, it is now even more daunting
The Arcee AI research team is honored to be among the contributors to the world's first fully decentralized training of a large language model (LLM). Read about the game-changing project led by Prime Intellect, and how we brought our expertise to the post-training.
LLMs are so 2023, and SLMs (Small Language Models) are the future of enterprise GenAI. Our Solutions Engineering Lead Tyler Odenthal wrote about how domain-adapted SLMs are being used across diverse sectors ranging from law and healthcare to education and customer service.
Subscribe to our newsletter
.svg)
