A Deep Dive into Microsoft AI Builder: Advanced Integration with REST APIs, Power Platform, SharePoint, and Developer Tools

Introduction

Microsoft AI Builder provides accessible, no-code and low-code AI capabilities within the Power Platform, enabling developers to infuse AI into applications without specialized machine learning knowledge. For developers with more advanced needs, AI Builder also supports REST API integration, allowing applications to communicate with AI models programmatically. It offers versatile capabilities across Power Automate, Power Apps, and even SharePoint for document automation, image recognition, and more.

In this article, we’ll explore AI Builder’s core functionality, technical integration through REST APIs, and usage within Power Platform tools and SharePoint. Additionally, we’ll cover development tools, options for testing, and free license opportunities to help developers experiment with AI Builder at minimal or no cost.

Core Capabilities of Microsoft AI Builder

AI Builder’s models include pre-built options as well as customizable models suited to various scenarios:

1. Pre-Built Models:

• Form Processing: Uses OCR to extract structured data from forms and documents.
• Object Detection: Recognizes and identifies objects in images, ideal for use cases like inventory tracking.
• Prediction Models: Leverages historical data to provide insights into future trends.

1. Custom Models:

• Custom Category Classification: Customizable for specific data categorization, applicable within Power Apps or Power Automate.
• Text Analytics: Includes entity extraction and sentiment analysis, useful for analyzing customer feedback or extracting keywords.

Technical Integration with REST API

For custom applications outside the Power Platform, AI Builder offers REST API integration. Here’s how developers can interact with AI models directly via API:

Authentication and Setup

To access the AI Builder REST API, developers must:

1. Register an Application in Azure Active Directory (AAD): This will allow OAuth 2.0 authentication, necessary for obtaining an access token.
2. Generate Access Tokens: Authenticate using OAuth 2.0, ensuring secure access to AI Builder’s resources.

API Usage

With REST API access, developers can submit data to AI models, receive structured responses, and integrate results into broader systems.

• Example Use Case: Processing documents with the Form Processing model and receiving extracted data in JSON format.
• API Rate Limits and Capacity: Ensure applications account for API usage limits and implement retries for high-volume processing.

Here’s an example Python code snippet for accessing the API:

import requests

# Obtain access token using OAuth 2.0
token_url = "https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token"
token_data = {
    "grant_type": "client_credentials",
    "client_id": "your_client_id",
    "client_secret": "your_client_secret",
    "scope": "https://api.powerplatform.com/.default"
}

response = requests.post(token_url, data=token_data)
access_token = response.json().get("access_token")

# Define API URL and headers
api_url = "https://api.powerplatform.com/ai-builder/v2.0/forms/models/{model_id}/analyze"
headers = {"Authorization": f"Bearer {access_token}", "Content-Type": "application/json"}

# Send document to model and retrieve results
with open("document.pdf", "rb") as file:
    response = requests.post(api_url, headers=headers, files={"file": file})
extracted_data = response.json()
print(extracted_data)

Integration with Power Automate and Power Apps

AI Builder integrates seamlessly within Power Platform tools, particularly Power Automate and Power Apps, enabling automation and AI-enhanced application development without requiring extensive code.

• Power Automate Integration: Use models like sentiment analysis to automate workflows, such as escalations based on customer feedback.
• Power Apps Integration: In Power Apps, AI Builder can help add real-time AI capabilities, like object detection or form processing, to user-facing applications.

Integrating AI Builder with SharePoint

Microsoft SharePoint, often used for document storage and collaboration, integrates well with AI Builder’s models for document processing, automating data extraction and enhancing document management.

Use Case: Automating Document Classification

AI Builder’s form-processing model can automatically extract and tag metadata from documents uploaded to SharePoint, such as invoices and contracts. Through Power Automate workflows, extracted metadata can populate SharePoint lists, making it easier to categorize and retrieve documents based on metadata.

Developer Tools and Licensing Options

Developer Plans and Free Options

Microsoft provides several ways for developers to test AI Builder and the Power Platform at little or no cost:

1. Power Apps Developer Plan: This free plan gives developers access to a dedicated environment to build and test Power Apps, Power Automate workflows, and AI Builder models.
2. Microsoft 365 Developer Program: This program provides a renewable 90-day free Microsoft 365 E5 sandbox subscription, which includes Power Apps and Power Automate access, ideal for SharePoint and AI Builder testing.
3. AI Builder Trial: Microsoft offers a 30-day free trial for AI Builder, granting 1 million AI Builder credits. This trial can be activated within the Power Platform admin center and is particularly useful for testing high-volume document processing or image recognition workflows.
4. Azure for Students: If you’re an eligible student, the Azure for Students program provides free credits that can also be applied to Azure Cognitive Services, useful for extending AI Builder functionalities with custom AI models.

Cost Considerations

AI Builder uses a consumption-based model for billing. Pricing starts at approximately $500 per month for 1 million credits. AI Builder credits are used based on the complexity and volume of model usage, with some models consuming more credits than others. Usage in REST API integrations may lead to higher costs, especially in scenarios requiring frequent, high-volume requests.

For detailed information on pricing and licensing, refer to Microsoft AI Builder pricing.

Advanced Power Platform Integration: Using AI Builder with REST and Power Platform Tools

The flexibility of AI Builder’s REST API also allows for integration with other tools in the Power Platform, such as Power BI and Power Virtual Agents.

REST API Integration with Power BI

Data processed through AI Builder can feed directly into Power BI for real-time analytics and reporting. For instance:

• Customer Sentiment Analysis: Use AI Builder to analyze customer feedback, extract sentiment scores, and integrate these scores into Power BI to visualize trends in customer satisfaction.
• Form Processing for Data Analytics: Extract structured data from documents via AI Builder and load it into Power BI dashboards for visual representation of KPIs, such as expense tracking or compliance adherence.

Power Virtual Agents

AI Builder can enhance Power Virtual Agents by processing language-based interactions and automating responses. For example, developers can integrate sentiment analysis to automatically detect customer moods and route conversations appropriately.

Power Automate and SharePoint Workflows

Developers can create complex document processing and tagging workflows by connecting AI Builder, Power Automate, and SharePoint. By automating the extraction of information from scanned documents in SharePoint, workflows can:

• Automatically classify, tag, and archive documents.
• Initiate approval workflows for documents based on metadata extracted by AI Builder.

Conclusion

Microsoft AI Builder offers robust tools for integrating AI across the Power Platform, REST APIs, and SharePoint, allowing developers to automate workflows, streamline document management, and enhance customer-facing applications. With developer-focused tools, licensing options, and free trials, AI Builder provides flexibility for exploring AI capabilities without incurring immediate costs.

For those interested in diving deeper into Microsoft AI Builder’s capabilities, integration methods, and pricing details, visit the AI Builder Documentation.

By leveraging these resources and taking advantage of development and free options, developers can harness AI Builder to enhance productivity and create innovative, AI-driven solutions tailored to their organization’s needs.