New AI Era in Azure Cloud with CoPilot?

Background:

Artificial Intelligence (AI) is driving a new wave of innovation, revolutionizing how applications are designed, built, and delivered. At Microsoft Ignite (Nov-2023), Azure announced an exciting new service to bring people together to experience the AI transformation. Microsoft Copilot for Azure is an intelligent virtual assistant with expert assistance with designing, operating, optimizing, and troubleshooting your cloud infrastructure and services. The service combines the latest cutting-edge large language models (LLMs) with the robust Azure Resource Model to offer a comprehensive understanding and management of everything happening in Azure, from the cloud to the edge. With Copilot for Azure, you can confidently navigate the complex world of cloud computing and ensure that your organization is making the most of the power of AI technology.

Note: When writing this article, Microsoft Copilot for Azure is in the public preview phase.

Why do we need Azure CoPilot?

Azure cloud management constantly evolves, with Microsoft frequently introducing new services and enhancing the existing ones. This ever-increasing infrastructure must provide more capabilities to meet the growing demands of end-users and applications. However, with the rapidly escalating number and complexity of resources, IT professionals often spend more time searching for information, which can negatively impact their productivity. In such a scenario, an AI assistant can be valuable in keeping pace with the ever-changing cloud technologies. Copilot for Azure is designed to assist end-users and administrators, helping them quickly and efficiently navigate the complex landscape of Azure cloud management.

Microsoft Copilot for Azure is a powerful tool that can significantly enhance your team’s productivity and performance. It utilizes advanced algorithms and machine learning techniques to help you complete complex tasks in less time and with higher accuracy. It can also quickly identify and recommend new capabilities you can leverage to improve your workflows and processes. With its advanced troubleshooting capabilities, you can quickly diagnose and resolve issues, improving your team’s efficiency and effectiveness. Microsoft Copilot for Azure is an indispensable tool to help you achieve your goals and drive your organization’s success.

The Copilot provides the query, which can be reused in your environment. When a user asks how many VMs they have, the answer will be the same as if they executed the query on Resource Graph Explorer.

How does Azure CoPilot work?

Copilot is currently accessible through the Azure portal and will soon become available through the Azure mobile app and CLI. This tool analyses and interprets various data types, such as Azure Resource Manager (ARM), Azure Resource Graph (ARG), cost and usage data, documentation, support, and best practice guidance. Copilot for Azure is a comprehensive solution that can help you reason over and optimize your resources.

Copilot for Azure is a one-of-a-kind solution that utilizes advanced large language models (LLMs) and the Azure Resource Model. It gives users a thorough understanding and streamlined management of all Azure activities, from the cloud to the edge.

The Copilot for Azure offers assistance with four primary capabilities:

1. Design: It helps to create and configure necessary services while ensuring they align with your organization’s policies.

2. Operate: Copilot can answer your questions, author complex commands, and manage resources.

3. Troubleshoot: The tool can orchestrate across Azure services to gather insights about issues, identify their causes, and suggest solutions.

4. Optimize: Copilot recommends improving your environment’s costs, scalability, and reliability.

To keep it simple, how the Copilot works:

1. The user submits prompts in the prompt bar, which are sent to the Copilot’s orchestrator for composing capabilities to answer them.
2. Copilot sends user requests and Copilot capabilities to Azure OpenAI to fulfil the request.
3. Azure Open AI matches the prompt with available capabilities and creates a plan for fulfilling the user’s request. This plan is sent back to the orchestrator. a. If no capability is matched, the response to the user’s prompt is generated using the general knowledge LLM, and the process jumps to step 5.
   b. The general knowledge LLM isn’t explicitly focused on the cloud, which may lead to less accurate responses. Enabling plugins and connecting to knowledge bases can improve Copilot’s resource-specific capabilities.
4. The orchestrator runs the code of selected plugins/capabilities and calls APIs to gather information and take action.
5. The orchestrator receives API responses, which the Copilot further fine-tunes. Finally, the response is combined with the original prompt and returned to Azure OpenAI.
6. Azure OpenAI uses advanced language models to generate human-like responses.
7. The Copilot sends the response to Microsoft’s AI models for review. If it’s safe, it’s sent to the user; otherwise, an error message is generated.
8. The user receives the response from the Copilot.

When using Copilot, it’s important to note that the current user must have permission through role-based access control for Copilot to access data from Azure and perform actions against Azure. All requests to Azure Resource Manager and other APIs are made on the user’s behalf; from a security perspective, the Copilot does not have its own identity.

How to enable the Copilot?

As mentioned earlier, this service is in public preview. Please click here to sign up. Users must complete the form to sign up and enrol in their Azure subscription. Microsoft will onboard customers into the preview every week.

The preview application will only be available for enterprise users. Microsoft will deny access requests from personal email accounts (@gmail.com, etc.).

After the signup, once Microsoft onboarded your subscriptions for the Copilot, you will receive an email. Once you log into the Azure portal, you can see the copilot icon at the top of your subscription; please refer to the image below.

Preview limitations:

It’s crucial to be aware of the following limitations when using Microsoft Copilot for Azure:

• Any action you want to perform on more than 10 resources must be carried out outside of Microsoft Copilot for Azure, as exceeding this limit may result in system errors or performance issues.
  You can only make 15 requests in a single chat and have 10 chats in 24 hours.
• Some responses that display lists will be limited to the top five items.
• You may need to provide an Azure resource ID instead of a resource’s name for specific tasks and queries.
• Microsoft Copilot for Azure (preview) is available only in English.

When asking questions related to subscriptions, such as the cost of a subscription, it’s essential to clarify the scope of the subscription. CoPilot will ask the user to confirm the subscription scope.

Demo of CoPilot:

I have created a demo on how to use CoPilot in Azure. You have also tested CoPilot by asking it a few questions using my subscriptions, and it provided accurate answers.

My question is: How do you design a hub-and-spoke network? The Copilot suggests configuring one using Azure Network Virtual Manager. It lists the steps to perform and provides the reference Azure articles.

Final Thoughts:

Copilot’s features operate within the Azure framework to ensure the security and privacy of our customers’ data. Copilot automatically adopts your organization’s security, compliance, and privacy policies for Azure. The large language models used in Copilot are not trained on customer tenant data. Copilot for Azure is designed to meet the needs of the enterprise.

Using this technology for any production workload is not advisable, as it is still in the preview phase. As a result, we will need to exercise caution and wait for its General Availability before we can confidently deploy it in a production environment.

Please share the article with like-minded readers so we can all learn from each other.