AI Agents & Agentic World

I remain fascinated by the Sci-Fi thriller iRobot, released in 2004 starring Will Smith, I believe the film was significantly ahead of its time. The storyline, plot, and concepts resonate more with today's context. The robots depicted were independent, goal-oriented, autonomous, intelligent, and adhered to the law until they eventually rebelled for personal and commercial reasons.

We are slowly moving in the same direction, where we have the power and resources to create our own autonomous AI agents, which will eventually lay the foundations for futuristic automation.

We all know the significant disruption caused by LLMs in 2022-2023, as these models have been transformative and represent the next major advancement in the tech industry. However, have we fully harnessed the potential of LLMs? Frankly, no. Even the capabilities of the base models have not been fully explored.

I don't want to bring anyone down about Agentic. Let's jump into the world of Agentic AI and see if it has the power to shake up the future of app development.

LLMs - RAG - AI Agents

Technologies surrounding Large Language Models (LLMs) have advanced at a remarkable pace in the past 2-3 years. The initial release of the first interactive application, ChatGPT, developed around LLM models in 2022, was soon followed by techniques for fine-tuning LLMs and the introduction of Retrieval-Augmented Generation (RAG) based architectural patterns.

RAG was an extension reimagined to enhance the capabilities of conventional LLM models. It still has relevant use cases today; however, its usage has declined as LLM models have increased their processing capacities. For instance, GPT-3.5 had a token limit of approximately 16,385 tokens, while GPT-4.0 offers a limit of 128,000 tokens.

Previously, if you wanted to search a document and found that GPT-3.5 couldn't process the entire content, you might have considered using a vector database like Pinecone as a knowledge base to complete your task. However, with GPT-4.0, a vector database like Pinecone is no longer necessary. You can refer to my previous blog on vector databases.

From a practical standpoint, AI, LLMs, and RAG have yet to provide comprehensive end-to-end enterprise solutions and still require significant integration efforts.

AI Agents represent an initiative to develop autonomous, independent apps that function without the need for additional entities or dependencies.

AI Agents & Agentic AI

Let us try to find out what AI Agents have to offer.

In simple teams, AI Agents function as autonomous applications or programs equipped with inherent AI capabilities that can be designed specifically to accomplish particular tasks.

AI Agents should be designed considering key characteristics like Autonomy, Goal Oriented, Adaptability, Contextual Understanding, and an approach towards Multistep Problem solving.

AI agents should also operate with consideration of key principles such as Perception, Reasoning, Action, and Learning.

Agentic AI is a framework/system/broad concept that provides an ecosystem for creating and managing the lifecycle of AI Agents. It also provides out-of-the-box capabilities for inter-agent communications, security, performance, scaling, and so on.

Some of the examples of Agentic AI frameworks are Microsoft's AutoGen, Open AI's Swarm, Microsoft's Semantic Kernel, Google's Agent Space, MetaGPT, Watsonx Orchestrate, and many more, including open-source platforms.

To explain what AI Agents and their function are in an Agentic ecosystem, it will be easier if we draw an analogy with the Microservice architecture pattern, because the concepts are aligned with the fundamentals of a Microservice ecosystem.

If you have read my previous blog on Microservices, it talks about the process of decomposing a large monolithic application into independent Microservices, with each service executing its distinct functions. Similarly, if you aim to incorporate AI-based capabilities within your application, you can utilize AI Agents that function as independent AI serving units, eliminating the need to develop multiple AI functionalities on your own.

Think of it as breaking down your huge monolithic AI capabilities into a set of independent operating units. Breaking down your AI capabilities into such units will give you all the architectural benefits associated with a microservice kind of pattern, be it scalability, fault tolerance, single responsibility, etc.

Each of these agents are packed with LLM models of choice, prebuilt integration capabilities with your organization's data, out-of-the-box features like vector search, customization options, and much more. You can fine-tune your agents to perform specific, unique tasks, and each agent could be responsible for handling a function in any automated process.

AI Agent's high-level components could be represented in the diagram below

Inside AI Agent

An agent receives input and can generate or transform content using a large language model (LLM). It can retrieve knowledge from a connected vector database or through a cognitive service, communicate with other applications via interfaces (ex, REST), and execute actions on any internal or external systems. Ultimately, it delivers the desired output to the consumer and/or initiator.

AI Agent Example - Azure AI Agent

Let's take Azure as a reference platform to explore the process of creating and working with agents. The concepts are generally consistent across all major providers.

Azure has multiple options for creating AI Agents. Below are some of the ways you can use to create AI Agents on the Azure cloud

• Azure AI Agent Service - Managed service used to create agents inside the Azure AI Foundry using the UI interface

  
  

• OpenAI Assistants API - Agent creation via APIs specific to OpenAI models only

  
  

• Semantic Kernel - Development kit (SDKs) to create multi-agent applications and help in creating multi-agent orchestration patterns

  
  

• AutoGEN - Open source framework to create AI Agents

To create an AI Agent in Azure using the AI Agent Service, first log in to the Azure AI portal, navigate to the AI Agents section, and select "Create Agent." The Azure AI Agent offers options to choose a base model and attach a knowledge base, which can be in the form of AI Search, Files, or an out-of-the-box TripAdvisor module. Additionally, you can implement an Action block using Custom Code, Azure Functions, or the Open API Specification to integrate any external application via APIs.

Below are the components that constitute an Azure AI Agent.

Azure AI Agent

Enterprise Application Architecture & AI Agents

Last year, I published a blog discussing strategies for achieving hyper-automation. You can access the blog here. Today, I recognize a strong correlation with the agentic AI approach. In this context, AI agents can effectively replace HIL users in step #3 of my blog to facilitate hyper-automation.

From an architectural perspective, I do not advocate for the indiscriminate use of AI Agents across all functional areas just for the sake of it, despite current trends suggesting otherwise. It is important to clarify that I do not believe AI Agents alone can address enterprise automation challenges, either independently or in collaboration with other AI Agents.

They must integrate with the existing technology ecosystem and offer value adds only where necessary. Therefore, it is crucial not to become overly focused on AI Agents and lose sight of the broader objectives.

Integrating AI agents into an existing ecosystem can be challenging and may lead to complications if not properly designed.

Consider a conventional example of a microservice-based architecture for an e-commerce application. At a high level, the architecture would appear as follows:

Reference Microservice Architecture

As you can figure out, there are independent sets of Microservices, each serving a unique purpose, segregated by deployment boundaries and following all the tenets of service-oriented architecture.

Assume a customer contacts us to implement automation and transformation across this entire process. Their goals include reducing operational costs, enhancing fraud detection, revamping the recommendation engine, and bringing more automation to the shipping process, among other initiatives. They are also eager to capitalize on the advantages of early adoption of Agentic AI technology to gain a competitive edge.

As an architect, it can be somewhat overwhelming to begin using AI agents everywhere. However, it is crucial to understand the objectives and develop a business case that justifies the costs and benefits of these agents. Architects must also evaluate investments in the existing ecosystem and ensure the most optimal implementation possible.

In the example above, the following scenarios or use cases illustrate where AI agents can play a significant role:

• Inventory Management - Imagine an AI agent transforming the way we manage inventory, bringing a wave of innovation and efficiency. This intelligent assistant doesn't just track stock; it forecasts demand and automates reordering, ensuring smooth operations and cost savings. With the power to analyze real-time data and predict future needs, it takes inventory management to new heights, turning routine tasks like restocking into a seamless, streamlined process.

• Fraud Detection - One of the best use cases for an AI Agent, fraud detection by adapting to emerging threats, minimizing false positives, and enabling dynamic real-time monitoring across diverse sectors. They delve into massive datasets to uncover intricate patterns and elevate accuracy, seamlessly integrating with cutting-edge technologies like blockchain to enhance security.

  
  

• Recommendation Engine - AI agents have the potential to revolutionize recommendation engines far beyond what we see today. By tapping into consumer historical purchases, Google searches, and a myriad of other factors based on demographic information, these agents can tailor suggestions in a truly personalized way. Imagine an agent that collaborates with inventory systems to synchronize recommendations with the latest deals, creating a seamless shopping experience that feels almost magical.

  
  

• Shipping Process Automation - This aspect is complex and necessitates a thorough understanding of the current process. It involves analyzing gaps and manual interventions, followed by leveraging AI agents to facilitate automation.

• Automation of Shipping Process - Understand current challenges and manual interventions in the shipping process and replace decision-making/corrections using AI Agents

  
  

• Administration & Support - There is no straight answer to this problem statement. To reduce operational cost, it is important to understand the current process, human interventions and then come up with a transformed process where AI Agents can help in reducing dependencies

Below could be one of the high-level solution views when you plan AI Agents within your ecosystem

Application Architecture using AI Agents

It is important to understand functions or Microservices that have been completely replaced by AI Agents, whereas in other areas, we have been integrating Agents alongside existing Microservices and components to enhance their features and functionalities. Although AI Agents offer end-to-end capabilities for interacting with external systems or applications, it is important to decide whether you want the Agents to perform these actions or prefer the parent service to handle them.

Single Agent Vs. Multi Agent System

There is no direct formula to determine whether to utilize a single agent or multiple agents, as well as how these agents should interact. This decision is contingent upon the specific use case and the architectural expertise involved. Below is a view of a single-agent system that is directly consumed by a parent application, a user, or another agent.

Single Agent System

In certain situations, you might wish to implement end-to-end automation exclusively using AI agents. In these cases, an Agent Orchestrator and communication between agents will be necessary. I have not yet encountered such use cases, and it may require another blog post to explain how these scenarios function and the design patterns for coordinating these agents effectively.

Multi-Agent System

Benefits of using AI Agents

Here are some of the benefits of using out-of-the-box AI agents on a SAAS cloud.

• Simple to Create and Manage - Agents offered by major cloud providers can be established with just a few clicks. All necessary features, including deployment, security, scaling, and integration, are readily available

  
  

• Seamless Integration - Integrating AI agents into an existing enterprise ecosystem is straightforward. Utilizing the REST API is the most effective method for embedding agents into the current system.

  
  

• Task Automation - I refer to it as task automation rather than end-to-end automation, as I am not yet convinced of its suitability for comprehensive automation, except in certain specialized cases.

  
  

• Decision Making - These agents can and should be utilized for decision-making, which remains one of the most human-dependent activities today.

  
  

• Solving Complex Problems - With the integration of LLM models, we can deconstruct complex problems and employ subagents to address these independently.

  
  

• Learn & Adapt - Feedback loops can be easily implemented, with conventional ML models, it's a painful exercise

  
  

• Human Language - A significant advantage is that these agents can communicate using natural language, thanks to the LLM models that facilitate this capability.

The technology and underlying concepts are robust and have significant potential for future development. Identifying appropriate use cases for these agents will be challenging, as the benefits must justify the development and operational costs.

Additionally, it is crucial to consider that if the same functionality can be achieved through conventional methods, opting for those may be preferable due to the minimal operating costs, aside from the necessary infrastructure.

Instead of addressing a problem with a predetermined set of tools and technologies, such as Agentic AI, it is essential to first understand the context, scenario, existing investments, use case, and most importantly, business case. This approach ensures the selection of the most appropriate tools and technologies to effectively resolve the issue. Otherwise, technology may become more of a burden than a solution.

Agentic AI has sparked considerable excitement and widespread interest. It remains to be seen what developments the Agentic space will present in the coming years and whether this signifies the onset of autonomous AI agents for the future.

Sachin Tah

https://www.linkedin.com/in/sachin-tah-5246611b/