feat: Add Official Microsoft & Gemini Skills (845+ Total)
🚀 Impact Significantly expands the capabilities of **Antigravity Awesome Skills** by integrating official skill collections from **Microsoft** and **Google Gemini**. This update increases the total skill count to **845+**, making the library even more comprehensive for AI coding assistants. ✨ Key Changes 1. New Official Skills - **Microsoft Skills**: Added a massive collection of official skills from [microsoft/skills](https://github.com/microsoft/skills). - Includes Azure, .NET, Python, TypeScript, and Semantic Kernel skills. - Preserves the original directory structure under `skills/official/microsoft/`. - Includes plugin skills from the `.github/plugins` directory. - **Gemini Skills**: Added official Gemini API development skills under `skills/gemini-api-dev/`. 2. New Scripts & Tooling - **`scripts/sync_microsoft_skills.py`**: A robust synchronization script that: - Clones the official Microsoft repository. - Preserves the original directory heirarchy. - Handles symlinks and plugin locations. - Generates attribution metadata. - **`scripts/tests/inspect_microsoft_repo.py`**: Debug tool to inspect the remote repository structure. - **`scripts/tests/test_comprehensive_coverage.py`**: Verification script to ensure 100% of skills are captured during sync. 3. Core Improvements - **`scripts/generate_index.py`**: Enhanced frontmatter parsing to safely handle unquoted values containing `@` symbols and commas (fixing issues with some Microsoft skill descriptions). - **`package.json`**: Added `sync:microsoft` and `sync:all-official` scripts for easy maintenance. 4. Documentation - Updated `README.md` to reflect the new skill counts (845+) and added Microsoft/Gemini to the provider list. - Updated `CATALOG.md` and `skills_index.json` with the new skills. 🧪 Verification - Ran `scripts/tests/test_comprehensive_coverage.py` to verify all Microsoft skills are detected. - Validated `generate_index.py` fixes by successfully indexing the new skills.
This commit is contained in:
320
skills/official/microsoft/python/compute/botservice/SKILL.md
Normal file
320
skills/official/microsoft/python/compute/botservice/SKILL.md
Normal file
@@ -0,0 +1,320 @@
|
||||
---
|
||||
name: azure-mgmt-botservice-py
|
||||
description: |
|
||||
Azure Bot Service Management SDK for Python. Use for creating, managing, and configuring Azure Bot Service resources.
|
||||
Triggers: "azure-mgmt-botservice", "AzureBotService", "bot management", "conversational AI", "bot channels".
|
||||
---
|
||||
|
||||
# Azure Bot Service Management SDK for Python
|
||||
|
||||
Manage Azure Bot Service resources including bots, channels, and connections.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-mgmt-botservice
|
||||
pip install azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_SUBSCRIPTION_ID=<your-subscription-id>
|
||||
AZURE_RESOURCE_GROUP=<your-resource-group>
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.mgmt.botservice import AzureBotService
|
||||
import os
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
client = AzureBotService(
|
||||
credential=credential,
|
||||
subscription_id=os.environ["AZURE_SUBSCRIPTION_ID"]
|
||||
)
|
||||
```
|
||||
|
||||
## Create a Bot
|
||||
|
||||
```python
|
||||
from azure.mgmt.botservice import AzureBotService
|
||||
from azure.mgmt.botservice.models import Bot, BotProperties, Sku
|
||||
from azure.identity import DefaultAzureCredential
|
||||
import os
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
client = AzureBotService(
|
||||
credential=credential,
|
||||
subscription_id=os.environ["AZURE_SUBSCRIPTION_ID"]
|
||||
)
|
||||
|
||||
resource_group = os.environ["AZURE_RESOURCE_GROUP"]
|
||||
bot_name = "my-chat-bot"
|
||||
|
||||
bot = client.bots.create(
|
||||
resource_group_name=resource_group,
|
||||
resource_name=bot_name,
|
||||
parameters=Bot(
|
||||
location="global",
|
||||
sku=Sku(name="F0"), # Free tier
|
||||
kind="azurebot",
|
||||
properties=BotProperties(
|
||||
display_name="My Chat Bot",
|
||||
description="A conversational AI bot",
|
||||
endpoint="https://my-bot-app.azurewebsites.net/api/messages",
|
||||
msa_app_id="<your-app-id>",
|
||||
msa_app_type="MultiTenant"
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
print(f"Bot created: {bot.name}")
|
||||
```
|
||||
|
||||
## Get Bot Details
|
||||
|
||||
```python
|
||||
bot = client.bots.get(
|
||||
resource_group_name=resource_group,
|
||||
resource_name=bot_name
|
||||
)
|
||||
|
||||
print(f"Bot: {bot.properties.display_name}")
|
||||
print(f"Endpoint: {bot.properties.endpoint}")
|
||||
print(f"SKU: {bot.sku.name}")
|
||||
```
|
||||
|
||||
## List Bots in Resource Group
|
||||
|
||||
```python
|
||||
bots = client.bots.list_by_resource_group(resource_group_name=resource_group)
|
||||
|
||||
for bot in bots:
|
||||
print(f"Bot: {bot.name} - {bot.properties.display_name}")
|
||||
```
|
||||
|
||||
## List All Bots in Subscription
|
||||
|
||||
```python
|
||||
all_bots = client.bots.list()
|
||||
|
||||
for bot in all_bots:
|
||||
print(f"Bot: {bot.name} in {bot.id.split('/')[4]}")
|
||||
```
|
||||
|
||||
## Update Bot
|
||||
|
||||
```python
|
||||
bot = client.bots.update(
|
||||
resource_group_name=resource_group,
|
||||
resource_name=bot_name,
|
||||
properties=BotProperties(
|
||||
display_name="Updated Bot Name",
|
||||
description="Updated description"
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
## Delete Bot
|
||||
|
||||
```python
|
||||
client.bots.delete(
|
||||
resource_group_name=resource_group,
|
||||
resource_name=bot_name
|
||||
)
|
||||
```
|
||||
|
||||
## Configure Channels
|
||||
|
||||
### Add Teams Channel
|
||||
|
||||
```python
|
||||
from azure.mgmt.botservice.models import (
|
||||
BotChannel,
|
||||
MsTeamsChannel,
|
||||
MsTeamsChannelProperties
|
||||
)
|
||||
|
||||
channel = client.channels.create(
|
||||
resource_group_name=resource_group,
|
||||
resource_name=bot_name,
|
||||
channel_name="MsTeamsChannel",
|
||||
parameters=BotChannel(
|
||||
location="global",
|
||||
properties=MsTeamsChannel(
|
||||
properties=MsTeamsChannelProperties(
|
||||
is_enabled=True
|
||||
)
|
||||
)
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
### Add Direct Line Channel
|
||||
|
||||
```python
|
||||
from azure.mgmt.botservice.models import (
|
||||
BotChannel,
|
||||
DirectLineChannel,
|
||||
DirectLineChannelProperties,
|
||||
DirectLineSite
|
||||
)
|
||||
|
||||
channel = client.channels.create(
|
||||
resource_group_name=resource_group,
|
||||
resource_name=bot_name,
|
||||
channel_name="DirectLineChannel",
|
||||
parameters=BotChannel(
|
||||
location="global",
|
||||
properties=DirectLineChannel(
|
||||
properties=DirectLineChannelProperties(
|
||||
sites=[
|
||||
DirectLineSite(
|
||||
site_name="Default Site",
|
||||
is_enabled=True,
|
||||
is_v1_enabled=False,
|
||||
is_v3_enabled=True
|
||||
)
|
||||
]
|
||||
)
|
||||
)
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
### Add Web Chat Channel
|
||||
|
||||
```python
|
||||
from azure.mgmt.botservice.models import (
|
||||
BotChannel,
|
||||
WebChatChannel,
|
||||
WebChatChannelProperties,
|
||||
WebChatSite
|
||||
)
|
||||
|
||||
channel = client.channels.create(
|
||||
resource_group_name=resource_group,
|
||||
resource_name=bot_name,
|
||||
channel_name="WebChatChannel",
|
||||
parameters=BotChannel(
|
||||
location="global",
|
||||
properties=WebChatChannel(
|
||||
properties=WebChatChannelProperties(
|
||||
sites=[
|
||||
WebChatSite(
|
||||
site_name="Default Site",
|
||||
is_enabled=True
|
||||
)
|
||||
]
|
||||
)
|
||||
)
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
## Get Channel Details
|
||||
|
||||
```python
|
||||
channel = client.channels.get(
|
||||
resource_group_name=resource_group,
|
||||
resource_name=bot_name,
|
||||
channel_name="DirectLineChannel"
|
||||
)
|
||||
```
|
||||
|
||||
## List Channel Keys
|
||||
|
||||
```python
|
||||
keys = client.channels.list_with_keys(
|
||||
resource_group_name=resource_group,
|
||||
resource_name=bot_name,
|
||||
channel_name="DirectLineChannel"
|
||||
)
|
||||
|
||||
# Access Direct Line keys
|
||||
if hasattr(keys.properties, 'properties'):
|
||||
for site in keys.properties.properties.sites:
|
||||
print(f"Site: {site.site_name}")
|
||||
print(f"Key: {site.key}")
|
||||
```
|
||||
|
||||
## Bot Connections (OAuth)
|
||||
|
||||
### Create Connection Setting
|
||||
|
||||
```python
|
||||
from azure.mgmt.botservice.models import (
|
||||
ConnectionSetting,
|
||||
ConnectionSettingProperties
|
||||
)
|
||||
|
||||
connection = client.bot_connection.create(
|
||||
resource_group_name=resource_group,
|
||||
resource_name=bot_name,
|
||||
connection_name="graph-connection",
|
||||
parameters=ConnectionSetting(
|
||||
location="global",
|
||||
properties=ConnectionSettingProperties(
|
||||
client_id="<oauth-client-id>",
|
||||
client_secret="<oauth-client-secret>",
|
||||
scopes="User.Read",
|
||||
service_provider_id="<service-provider-id>"
|
||||
)
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
### List Connections
|
||||
|
||||
```python
|
||||
connections = client.bot_connection.list_by_bot_service(
|
||||
resource_group_name=resource_group,
|
||||
resource_name=bot_name
|
||||
)
|
||||
|
||||
for conn in connections:
|
||||
print(f"Connection: {conn.name}")
|
||||
```
|
||||
|
||||
## Client Operations
|
||||
|
||||
| Operation | Method |
|
||||
|-----------|--------|
|
||||
| `client.bots` | Bot CRUD operations |
|
||||
| `client.channels` | Channel configuration |
|
||||
| `client.bot_connection` | OAuth connection settings |
|
||||
| `client.direct_line` | Direct Line channel operations |
|
||||
| `client.email` | Email channel operations |
|
||||
| `client.operations` | Available operations |
|
||||
| `client.host_settings` | Host settings operations |
|
||||
|
||||
## SKU Options
|
||||
|
||||
| SKU | Description |
|
||||
|-----|-------------|
|
||||
| `F0` | Free tier (limited messages) |
|
||||
| `S1` | Standard tier (unlimited messages) |
|
||||
|
||||
## Channel Types
|
||||
|
||||
| Channel | Class | Purpose |
|
||||
|---------|-------|---------|
|
||||
| `MsTeamsChannel` | Microsoft Teams | Teams integration |
|
||||
| `DirectLineChannel` | Direct Line | Custom client integration |
|
||||
| `WebChatChannel` | Web Chat | Embeddable web widget |
|
||||
| `SlackChannel` | Slack | Slack workspace integration |
|
||||
| `FacebookChannel` | Facebook | Messenger integration |
|
||||
| `EmailChannel` | Email | Email communication |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use DefaultAzureCredential** for authentication
|
||||
2. **Start with F0 SKU** for development, upgrade to S1 for production
|
||||
3. **Store MSA App ID/Secret securely** — use Key Vault
|
||||
4. **Enable only needed channels** — reduces attack surface
|
||||
5. **Rotate Direct Line keys** periodically
|
||||
6. **Use managed identity** when possible for bot connections
|
||||
7. **Configure proper CORS** for Web Chat channel
|
||||
@@ -0,0 +1,252 @@
|
||||
---
|
||||
name: azure-containerregistry-py
|
||||
description: |
|
||||
Azure Container Registry SDK for Python. Use for managing container images, artifacts, and repositories.
|
||||
Triggers: "azure-containerregistry", "ContainerRegistryClient", "container images", "docker registry", "ACR".
|
||||
package: azure-containerregistry
|
||||
---
|
||||
|
||||
# Azure Container Registry SDK for Python
|
||||
|
||||
Manage container images, artifacts, and repositories in Azure Container Registry.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-containerregistry
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_CONTAINERREGISTRY_ENDPOINT=https://<registry-name>.azurecr.io
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
### Entra ID (Recommended)
|
||||
|
||||
```python
|
||||
from azure.containerregistry import ContainerRegistryClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
client = ContainerRegistryClient(
|
||||
endpoint=os.environ["AZURE_CONTAINERREGISTRY_ENDPOINT"],
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
```
|
||||
|
||||
### Anonymous Access (Public Registry)
|
||||
|
||||
```python
|
||||
from azure.containerregistry import ContainerRegistryClient
|
||||
|
||||
client = ContainerRegistryClient(
|
||||
endpoint="https://mcr.microsoft.com",
|
||||
credential=None,
|
||||
audience="https://mcr.microsoft.com"
|
||||
)
|
||||
```
|
||||
|
||||
## List Repositories
|
||||
|
||||
```python
|
||||
client = ContainerRegistryClient(endpoint, DefaultAzureCredential())
|
||||
|
||||
for repository in client.list_repository_names():
|
||||
print(repository)
|
||||
```
|
||||
|
||||
## Repository Operations
|
||||
|
||||
### Get Repository Properties
|
||||
|
||||
```python
|
||||
properties = client.get_repository_properties("my-image")
|
||||
print(f"Created: {properties.created_on}")
|
||||
print(f"Modified: {properties.last_updated_on}")
|
||||
print(f"Manifests: {properties.manifest_count}")
|
||||
print(f"Tags: {properties.tag_count}")
|
||||
```
|
||||
|
||||
### Update Repository Properties
|
||||
|
||||
```python
|
||||
from azure.containerregistry import RepositoryProperties
|
||||
|
||||
client.update_repository_properties(
|
||||
"my-image",
|
||||
properties=RepositoryProperties(
|
||||
can_delete=False,
|
||||
can_write=False
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
### Delete Repository
|
||||
|
||||
```python
|
||||
client.delete_repository("my-image")
|
||||
```
|
||||
|
||||
## List Tags
|
||||
|
||||
```python
|
||||
for tag in client.list_tag_properties("my-image"):
|
||||
print(f"{tag.name}: {tag.created_on}")
|
||||
```
|
||||
|
||||
### Filter by Order
|
||||
|
||||
```python
|
||||
from azure.containerregistry import ArtifactTagOrder
|
||||
|
||||
# Most recent first
|
||||
for tag in client.list_tag_properties(
|
||||
"my-image",
|
||||
order_by=ArtifactTagOrder.LAST_UPDATED_ON_DESCENDING
|
||||
):
|
||||
print(f"{tag.name}: {tag.last_updated_on}")
|
||||
```
|
||||
|
||||
## Manifest Operations
|
||||
|
||||
### List Manifests
|
||||
|
||||
```python
|
||||
from azure.containerregistry import ArtifactManifestOrder
|
||||
|
||||
for manifest in client.list_manifest_properties(
|
||||
"my-image",
|
||||
order_by=ArtifactManifestOrder.LAST_UPDATED_ON_DESCENDING
|
||||
):
|
||||
print(f"Digest: {manifest.digest}")
|
||||
print(f"Tags: {manifest.tags}")
|
||||
print(f"Size: {manifest.size_in_bytes}")
|
||||
```
|
||||
|
||||
### Get Manifest Properties
|
||||
|
||||
```python
|
||||
manifest = client.get_manifest_properties("my-image", "latest")
|
||||
print(f"Digest: {manifest.digest}")
|
||||
print(f"Architecture: {manifest.architecture}")
|
||||
print(f"OS: {manifest.operating_system}")
|
||||
```
|
||||
|
||||
### Update Manifest Properties
|
||||
|
||||
```python
|
||||
from azure.containerregistry import ArtifactManifestProperties
|
||||
|
||||
client.update_manifest_properties(
|
||||
"my-image",
|
||||
"latest",
|
||||
properties=ArtifactManifestProperties(
|
||||
can_delete=False,
|
||||
can_write=False
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
### Delete Manifest
|
||||
|
||||
```python
|
||||
# Delete by digest
|
||||
client.delete_manifest("my-image", "sha256:abc123...")
|
||||
|
||||
# Delete by tag
|
||||
manifest = client.get_manifest_properties("my-image", "old-tag")
|
||||
client.delete_manifest("my-image", manifest.digest)
|
||||
```
|
||||
|
||||
## Tag Operations
|
||||
|
||||
### Get Tag Properties
|
||||
|
||||
```python
|
||||
tag = client.get_tag_properties("my-image", "latest")
|
||||
print(f"Digest: {tag.digest}")
|
||||
print(f"Created: {tag.created_on}")
|
||||
```
|
||||
|
||||
### Delete Tag
|
||||
|
||||
```python
|
||||
client.delete_tag("my-image", "old-tag")
|
||||
```
|
||||
|
||||
## Upload and Download Artifacts
|
||||
|
||||
```python
|
||||
from azure.containerregistry import ContainerRegistryClient
|
||||
|
||||
client = ContainerRegistryClient(endpoint, DefaultAzureCredential())
|
||||
|
||||
# Download manifest
|
||||
manifest = client.download_manifest("my-image", "latest")
|
||||
print(f"Media type: {manifest.media_type}")
|
||||
print(f"Digest: {manifest.digest}")
|
||||
|
||||
# Download blob
|
||||
blob = client.download_blob("my-image", "sha256:abc123...")
|
||||
with open("layer.tar.gz", "wb") as f:
|
||||
for chunk in blob:
|
||||
f.write(chunk)
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.containerregistry.aio import ContainerRegistryClient
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def list_repos():
|
||||
credential = DefaultAzureCredential()
|
||||
client = ContainerRegistryClient(endpoint, credential)
|
||||
|
||||
async for repo in client.list_repository_names():
|
||||
print(repo)
|
||||
|
||||
await client.close()
|
||||
await credential.close()
|
||||
```
|
||||
|
||||
## Clean Up Old Images
|
||||
|
||||
```python
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
cutoff = datetime.now(timezone.utc) - timedelta(days=30)
|
||||
|
||||
for manifest in client.list_manifest_properties("my-image"):
|
||||
if manifest.last_updated_on < cutoff and not manifest.tags:
|
||||
print(f"Deleting {manifest.digest}")
|
||||
client.delete_manifest("my-image", manifest.digest)
|
||||
```
|
||||
|
||||
## Client Operations
|
||||
|
||||
| Operation | Description |
|
||||
|-----------|-------------|
|
||||
| `list_repository_names` | List all repositories |
|
||||
| `get_repository_properties` | Get repository metadata |
|
||||
| `delete_repository` | Delete repository and all images |
|
||||
| `list_tag_properties` | List tags in repository |
|
||||
| `get_tag_properties` | Get tag metadata |
|
||||
| `delete_tag` | Delete specific tag |
|
||||
| `list_manifest_properties` | List manifests in repository |
|
||||
| `get_manifest_properties` | Get manifest metadata |
|
||||
| `delete_manifest` | Delete manifest by digest |
|
||||
| `download_manifest` | Download manifest content |
|
||||
| `download_blob` | Download layer blob |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use Entra ID** for authentication in production
|
||||
2. **Delete by digest** not tag to avoid orphaned images
|
||||
3. **Lock production images** with can_delete=False
|
||||
4. **Clean up untagged manifests** regularly
|
||||
5. **Use async client** for high-throughput operations
|
||||
6. **Order by last_updated** to find recent/old images
|
||||
7. **Check manifest.tags** before deleting to avoid removing tagged images
|
||||
258
skills/official/microsoft/python/compute/fabric/SKILL.md
Normal file
258
skills/official/microsoft/python/compute/fabric/SKILL.md
Normal file
@@ -0,0 +1,258 @@
|
||||
---
|
||||
name: azure-mgmt-fabric-py
|
||||
description: |
|
||||
Azure Fabric Management SDK for Python. Use for managing Microsoft Fabric capacities and resources.
|
||||
Triggers: "azure-mgmt-fabric", "FabricMgmtClient", "Fabric capacity", "Microsoft Fabric", "Power BI capacity".
|
||||
---
|
||||
|
||||
# Azure Fabric Management SDK for Python
|
||||
|
||||
Manage Microsoft Fabric capacities and resources programmatically.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-mgmt-fabric
|
||||
pip install azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_SUBSCRIPTION_ID=<your-subscription-id>
|
||||
AZURE_RESOURCE_GROUP=<your-resource-group>
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.mgmt.fabric import FabricMgmtClient
|
||||
import os
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
client = FabricMgmtClient(
|
||||
credential=credential,
|
||||
subscription_id=os.environ["AZURE_SUBSCRIPTION_ID"]
|
||||
)
|
||||
```
|
||||
|
||||
## Create Fabric Capacity
|
||||
|
||||
```python
|
||||
from azure.mgmt.fabric import FabricMgmtClient
|
||||
from azure.mgmt.fabric.models import FabricCapacity, FabricCapacityProperties, CapacitySku
|
||||
from azure.identity import DefaultAzureCredential
|
||||
import os
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
client = FabricMgmtClient(
|
||||
credential=credential,
|
||||
subscription_id=os.environ["AZURE_SUBSCRIPTION_ID"]
|
||||
)
|
||||
|
||||
resource_group = os.environ["AZURE_RESOURCE_GROUP"]
|
||||
capacity_name = "myfabriccapacity"
|
||||
|
||||
capacity = client.fabric_capacities.begin_create_or_update(
|
||||
resource_group_name=resource_group,
|
||||
capacity_name=capacity_name,
|
||||
resource=FabricCapacity(
|
||||
location="eastus",
|
||||
sku=CapacitySku(
|
||||
name="F2", # Fabric SKU
|
||||
tier="Fabric"
|
||||
),
|
||||
properties=FabricCapacityProperties(
|
||||
administration=FabricCapacityAdministration(
|
||||
members=["user@contoso.com"]
|
||||
)
|
||||
)
|
||||
)
|
||||
).result()
|
||||
|
||||
print(f"Capacity created: {capacity.name}")
|
||||
```
|
||||
|
||||
## Get Capacity Details
|
||||
|
||||
```python
|
||||
capacity = client.fabric_capacities.get(
|
||||
resource_group_name=resource_group,
|
||||
capacity_name=capacity_name
|
||||
)
|
||||
|
||||
print(f"Capacity: {capacity.name}")
|
||||
print(f"SKU: {capacity.sku.name}")
|
||||
print(f"State: {capacity.properties.state}")
|
||||
print(f"Location: {capacity.location}")
|
||||
```
|
||||
|
||||
## List Capacities in Resource Group
|
||||
|
||||
```python
|
||||
capacities = client.fabric_capacities.list_by_resource_group(
|
||||
resource_group_name=resource_group
|
||||
)
|
||||
|
||||
for capacity in capacities:
|
||||
print(f"Capacity: {capacity.name} - SKU: {capacity.sku.name}")
|
||||
```
|
||||
|
||||
## List All Capacities in Subscription
|
||||
|
||||
```python
|
||||
all_capacities = client.fabric_capacities.list_by_subscription()
|
||||
|
||||
for capacity in all_capacities:
|
||||
print(f"Capacity: {capacity.name} in {capacity.location}")
|
||||
```
|
||||
|
||||
## Update Capacity
|
||||
|
||||
```python
|
||||
from azure.mgmt.fabric.models import FabricCapacityUpdate, CapacitySku
|
||||
|
||||
updated = client.fabric_capacities.begin_update(
|
||||
resource_group_name=resource_group,
|
||||
capacity_name=capacity_name,
|
||||
properties=FabricCapacityUpdate(
|
||||
sku=CapacitySku(
|
||||
name="F4", # Scale up
|
||||
tier="Fabric"
|
||||
),
|
||||
tags={"environment": "production"}
|
||||
)
|
||||
).result()
|
||||
|
||||
print(f"Updated SKU: {updated.sku.name}")
|
||||
```
|
||||
|
||||
## Suspend Capacity
|
||||
|
||||
Pause capacity to stop billing:
|
||||
|
||||
```python
|
||||
client.fabric_capacities.begin_suspend(
|
||||
resource_group_name=resource_group,
|
||||
capacity_name=capacity_name
|
||||
).result()
|
||||
|
||||
print("Capacity suspended")
|
||||
```
|
||||
|
||||
## Resume Capacity
|
||||
|
||||
Resume a paused capacity:
|
||||
|
||||
```python
|
||||
client.fabric_capacities.begin_resume(
|
||||
resource_group_name=resource_group,
|
||||
capacity_name=capacity_name
|
||||
).result()
|
||||
|
||||
print("Capacity resumed")
|
||||
```
|
||||
|
||||
## Delete Capacity
|
||||
|
||||
```python
|
||||
client.fabric_capacities.begin_delete(
|
||||
resource_group_name=resource_group,
|
||||
capacity_name=capacity_name
|
||||
).result()
|
||||
|
||||
print("Capacity deleted")
|
||||
```
|
||||
|
||||
## Check Name Availability
|
||||
|
||||
```python
|
||||
from azure.mgmt.fabric.models import CheckNameAvailabilityRequest
|
||||
|
||||
result = client.fabric_capacities.check_name_availability(
|
||||
location="eastus",
|
||||
body=CheckNameAvailabilityRequest(
|
||||
name="my-new-capacity",
|
||||
type="Microsoft.Fabric/capacities"
|
||||
)
|
||||
)
|
||||
|
||||
if result.name_available:
|
||||
print("Name is available")
|
||||
else:
|
||||
print(f"Name not available: {result.reason}")
|
||||
```
|
||||
|
||||
## List Available SKUs
|
||||
|
||||
```python
|
||||
skus = client.fabric_capacities.list_skus(
|
||||
resource_group_name=resource_group,
|
||||
capacity_name=capacity_name
|
||||
)
|
||||
|
||||
for sku in skus:
|
||||
print(f"SKU: {sku.name} - Tier: {sku.tier}")
|
||||
```
|
||||
|
||||
## Client Operations
|
||||
|
||||
| Operation | Method |
|
||||
|-----------|--------|
|
||||
| `client.fabric_capacities` | Capacity CRUD operations |
|
||||
| `client.operations` | List available operations |
|
||||
|
||||
## Fabric SKUs
|
||||
|
||||
| SKU | Description | CUs |
|
||||
|-----|-------------|-----|
|
||||
| `F2` | Entry level | 2 Capacity Units |
|
||||
| `F4` | Small | 4 Capacity Units |
|
||||
| `F8` | Medium | 8 Capacity Units |
|
||||
| `F16` | Large | 16 Capacity Units |
|
||||
| `F32` | X-Large | 32 Capacity Units |
|
||||
| `F64` | 2X-Large | 64 Capacity Units |
|
||||
| `F128` | 4X-Large | 128 Capacity Units |
|
||||
| `F256` | 8X-Large | 256 Capacity Units |
|
||||
| `F512` | 16X-Large | 512 Capacity Units |
|
||||
| `F1024` | 32X-Large | 1024 Capacity Units |
|
||||
| `F2048` | 64X-Large | 2048 Capacity Units |
|
||||
|
||||
## Capacity States
|
||||
|
||||
| State | Description |
|
||||
|-------|-------------|
|
||||
| `Active` | Capacity is running |
|
||||
| `Paused` | Capacity is suspended (no billing) |
|
||||
| `Provisioning` | Being created |
|
||||
| `Updating` | Being modified |
|
||||
| `Deleting` | Being removed |
|
||||
| `Failed` | Operation failed |
|
||||
|
||||
## Long-Running Operations
|
||||
|
||||
All mutating operations are long-running (LRO). Use `.result()` to wait:
|
||||
|
||||
```python
|
||||
# Synchronous wait
|
||||
capacity = client.fabric_capacities.begin_create_or_update(...).result()
|
||||
|
||||
# Or poll manually
|
||||
poller = client.fabric_capacities.begin_create_or_update(...)
|
||||
while not poller.done():
|
||||
print(f"Status: {poller.status()}")
|
||||
time.sleep(5)
|
||||
capacity = poller.result()
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use DefaultAzureCredential** for authentication
|
||||
2. **Suspend unused capacities** to reduce costs
|
||||
3. **Start with smaller SKUs** and scale up as needed
|
||||
4. **Use tags** for cost tracking and organization
|
||||
5. **Check name availability** before creating capacities
|
||||
6. **Handle LRO properly** — don't assume immediate completion
|
||||
7. **Set up capacity admins** — specify users who can manage workspaces
|
||||
8. **Monitor capacity usage** via Azure Monitor metrics
|
||||
219
skills/official/microsoft/python/data/blob/SKILL.md
Normal file
219
skills/official/microsoft/python/data/blob/SKILL.md
Normal file
@@ -0,0 +1,219 @@
|
||||
---
|
||||
name: azure-storage-blob-py
|
||||
description: |
|
||||
Azure Blob Storage SDK for Python. Use for uploading, downloading, listing blobs, managing containers, and blob lifecycle.
|
||||
Triggers: "blob storage", "BlobServiceClient", "ContainerClient", "BlobClient", "upload blob", "download blob".
|
||||
package: azure-storage-blob
|
||||
---
|
||||
|
||||
# Azure Blob Storage SDK for Python
|
||||
|
||||
Client library for Azure Blob Storage — object storage for unstructured data.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-storage-blob azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_STORAGE_ACCOUNT_NAME=<your-storage-account>
|
||||
# Or use full URL
|
||||
AZURE_STORAGE_ACCOUNT_URL=https://<account>.blob.core.windows.net
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.storage.blob import BlobServiceClient
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
account_url = "https://<account>.blob.core.windows.net"
|
||||
|
||||
blob_service_client = BlobServiceClient(account_url, credential=credential)
|
||||
```
|
||||
|
||||
## Client Hierarchy
|
||||
|
||||
| Client | Purpose | Get From |
|
||||
|--------|---------|----------|
|
||||
| `BlobServiceClient` | Account-level operations | Direct instantiation |
|
||||
| `ContainerClient` | Container operations | `blob_service_client.get_container_client()` |
|
||||
| `BlobClient` | Single blob operations | `container_client.get_blob_client()` |
|
||||
|
||||
## Core Workflow
|
||||
|
||||
### Create Container
|
||||
|
||||
```python
|
||||
container_client = blob_service_client.get_container_client("mycontainer")
|
||||
container_client.create_container()
|
||||
```
|
||||
|
||||
### Upload Blob
|
||||
|
||||
```python
|
||||
# From file path
|
||||
blob_client = blob_service_client.get_blob_client(
|
||||
container="mycontainer",
|
||||
blob="sample.txt"
|
||||
)
|
||||
|
||||
with open("./local-file.txt", "rb") as data:
|
||||
blob_client.upload_blob(data, overwrite=True)
|
||||
|
||||
# From bytes/string
|
||||
blob_client.upload_blob(b"Hello, World!", overwrite=True)
|
||||
|
||||
# From stream
|
||||
import io
|
||||
stream = io.BytesIO(b"Stream content")
|
||||
blob_client.upload_blob(stream, overwrite=True)
|
||||
```
|
||||
|
||||
### Download Blob
|
||||
|
||||
```python
|
||||
blob_client = blob_service_client.get_blob_client(
|
||||
container="mycontainer",
|
||||
blob="sample.txt"
|
||||
)
|
||||
|
||||
# To file
|
||||
with open("./downloaded.txt", "wb") as file:
|
||||
download_stream = blob_client.download_blob()
|
||||
file.write(download_stream.readall())
|
||||
|
||||
# To memory
|
||||
download_stream = blob_client.download_blob()
|
||||
content = download_stream.readall() # bytes
|
||||
|
||||
# Read into existing buffer
|
||||
stream = io.BytesIO()
|
||||
num_bytes = blob_client.download_blob().readinto(stream)
|
||||
```
|
||||
|
||||
### List Blobs
|
||||
|
||||
```python
|
||||
container_client = blob_service_client.get_container_client("mycontainer")
|
||||
|
||||
# List all blobs
|
||||
for blob in container_client.list_blobs():
|
||||
print(f"{blob.name} - {blob.size} bytes")
|
||||
|
||||
# List with prefix (folder-like)
|
||||
for blob in container_client.list_blobs(name_starts_with="logs/"):
|
||||
print(blob.name)
|
||||
|
||||
# Walk blob hierarchy (virtual directories)
|
||||
for item in container_client.walk_blobs(delimiter="/"):
|
||||
if item.get("prefix"):
|
||||
print(f"Directory: {item['prefix']}")
|
||||
else:
|
||||
print(f"Blob: {item.name}")
|
||||
```
|
||||
|
||||
### Delete Blob
|
||||
|
||||
```python
|
||||
blob_client.delete_blob()
|
||||
|
||||
# Delete with snapshots
|
||||
blob_client.delete_blob(delete_snapshots="include")
|
||||
```
|
||||
|
||||
## Performance Tuning
|
||||
|
||||
```python
|
||||
# Configure chunk sizes for large uploads/downloads
|
||||
blob_client = BlobClient(
|
||||
account_url=account_url,
|
||||
container_name="mycontainer",
|
||||
blob_name="large-file.zip",
|
||||
credential=credential,
|
||||
max_block_size=4 * 1024 * 1024, # 4 MiB blocks
|
||||
max_single_put_size=64 * 1024 * 1024 # 64 MiB single upload limit
|
||||
)
|
||||
|
||||
# Parallel upload
|
||||
blob_client.upload_blob(data, max_concurrency=4)
|
||||
|
||||
# Parallel download
|
||||
download_stream = blob_client.download_blob(max_concurrency=4)
|
||||
```
|
||||
|
||||
## SAS Tokens
|
||||
|
||||
```python
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from azure.storage.blob import generate_blob_sas, BlobSasPermissions
|
||||
|
||||
sas_token = generate_blob_sas(
|
||||
account_name="<account>",
|
||||
container_name="mycontainer",
|
||||
blob_name="sample.txt",
|
||||
account_key="<account-key>", # Or use user delegation key
|
||||
permission=BlobSasPermissions(read=True),
|
||||
expiry=datetime.now(timezone.utc) + timedelta(hours=1)
|
||||
)
|
||||
|
||||
# Use SAS token
|
||||
blob_url = f"https://<account>.blob.core.windows.net/mycontainer/sample.txt?{sas_token}"
|
||||
```
|
||||
|
||||
## Blob Properties and Metadata
|
||||
|
||||
```python
|
||||
# Get properties
|
||||
properties = blob_client.get_blob_properties()
|
||||
print(f"Size: {properties.size}")
|
||||
print(f"Content-Type: {properties.content_settings.content_type}")
|
||||
print(f"Last modified: {properties.last_modified}")
|
||||
|
||||
# Set metadata
|
||||
blob_client.set_blob_metadata(metadata={"category": "logs", "year": "2024"})
|
||||
|
||||
# Set content type
|
||||
from azure.storage.blob import ContentSettings
|
||||
blob_client.set_http_headers(
|
||||
content_settings=ContentSettings(content_type="application/json")
|
||||
)
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
from azure.storage.blob.aio import BlobServiceClient
|
||||
|
||||
async def upload_async():
|
||||
credential = DefaultAzureCredential()
|
||||
|
||||
async with BlobServiceClient(account_url, credential=credential) as client:
|
||||
blob_client = client.get_blob_client("mycontainer", "sample.txt")
|
||||
|
||||
with open("./file.txt", "rb") as data:
|
||||
await blob_client.upload_blob(data, overwrite=True)
|
||||
|
||||
# Download async
|
||||
async def download_async():
|
||||
async with BlobServiceClient(account_url, credential=credential) as client:
|
||||
blob_client = client.get_blob_client("mycontainer", "sample.txt")
|
||||
|
||||
stream = await blob_client.download_blob()
|
||||
data = await stream.readall()
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use DefaultAzureCredential** instead of connection strings
|
||||
2. **Use context managers** for async clients
|
||||
3. **Set `overwrite=True`** explicitly when re-uploading
|
||||
4. **Use `max_concurrency`** for large file transfers
|
||||
5. **Prefer `readinto()`** over `readall()` for memory efficiency
|
||||
6. **Use `walk_blobs()`** for hierarchical listing
|
||||
7. **Set appropriate content types** for web-served blobs
|
||||
239
skills/official/microsoft/python/data/cosmos-db/SKILL.md
Normal file
239
skills/official/microsoft/python/data/cosmos-db/SKILL.md
Normal file
@@ -0,0 +1,239 @@
|
||||
---
|
||||
name: azure-cosmos-db-py
|
||||
description: Build Azure Cosmos DB NoSQL services with Python/FastAPI following production-grade patterns. Use when implementing database client setup with dual auth (DefaultAzureCredential + emulator), service layer classes with CRUD operations, partition key strategies, parameterized queries, or TDD patterns for Cosmos. Triggers on phrases like "Cosmos DB", "NoSQL database", "document store", "add persistence", "database service layer", or "Python Cosmos SDK".
|
||||
package: azure-cosmos
|
||||
---
|
||||
|
||||
# Cosmos DB Service Implementation
|
||||
|
||||
Build production-grade Azure Cosmos DB NoSQL services following clean code, security best practices, and TDD principles.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-cosmos azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
COSMOS_ENDPOINT=https://<account>.documents.azure.com:443/
|
||||
COSMOS_DATABASE_NAME=<database-name>
|
||||
COSMOS_CONTAINER_ID=<container-id>
|
||||
# For emulator only (not production)
|
||||
COSMOS_KEY=<emulator-key>
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
**DefaultAzureCredential (preferred)**:
|
||||
```python
|
||||
from azure.cosmos import CosmosClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
client = CosmosClient(
|
||||
url=os.environ["COSMOS_ENDPOINT"],
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
```
|
||||
|
||||
**Emulator (local development)**:
|
||||
```python
|
||||
from azure.cosmos import CosmosClient
|
||||
|
||||
client = CosmosClient(
|
||||
url="https://localhost:8081",
|
||||
credential=os.environ["COSMOS_KEY"],
|
||||
connection_verify=False
|
||||
)
|
||||
```
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ FastAPI Router │
|
||||
│ - Auth dependencies (get_current_user, get_current_user_required)
|
||||
│ - HTTP error responses (HTTPException) │
|
||||
└──────────────────────────────┬──────────────────────────────────┘
|
||||
│
|
||||
┌──────────────────────────────▼──────────────────────────────────┐
|
||||
│ Service Layer │
|
||||
│ - Business logic and validation │
|
||||
│ - Document ↔ Model conversion │
|
||||
│ - Graceful degradation when Cosmos unavailable │
|
||||
└──────────────────────────────┬──────────────────────────────────┘
|
||||
│
|
||||
┌──────────────────────────────▼──────────────────────────────────┐
|
||||
│ Cosmos DB Client Module │
|
||||
│ - Singleton container initialization │
|
||||
│ - Dual auth: DefaultAzureCredential (Azure) / Key (emulator) │
|
||||
│ - Async wrapper via run_in_threadpool │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Client Module Setup
|
||||
|
||||
Create a singleton Cosmos client with dual authentication:
|
||||
|
||||
```python
|
||||
# db/cosmos.py
|
||||
from azure.cosmos import CosmosClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from starlette.concurrency import run_in_threadpool
|
||||
|
||||
_cosmos_container = None
|
||||
|
||||
def _is_emulator_endpoint(endpoint: str) -> bool:
|
||||
return "localhost" in endpoint or "127.0.0.1" in endpoint
|
||||
|
||||
async def get_container():
|
||||
global _cosmos_container
|
||||
if _cosmos_container is None:
|
||||
if _is_emulator_endpoint(settings.cosmos_endpoint):
|
||||
client = CosmosClient(
|
||||
url=settings.cosmos_endpoint,
|
||||
credential=settings.cosmos_key,
|
||||
connection_verify=False
|
||||
)
|
||||
else:
|
||||
client = CosmosClient(
|
||||
url=settings.cosmos_endpoint,
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
db = client.get_database_client(settings.cosmos_database_name)
|
||||
_cosmos_container = db.get_container_client(settings.cosmos_container_id)
|
||||
return _cosmos_container
|
||||
```
|
||||
|
||||
**Full implementation**: See [references/client-setup.md](references/client-setup.md)
|
||||
|
||||
### 2. Pydantic Model Hierarchy
|
||||
|
||||
Use five-tier model pattern for clean separation:
|
||||
|
||||
```python
|
||||
class ProjectBase(BaseModel): # Shared fields
|
||||
name: str = Field(..., min_length=1, max_length=200)
|
||||
|
||||
class ProjectCreate(ProjectBase): # Creation request
|
||||
workspace_id: str = Field(..., alias="workspaceId")
|
||||
|
||||
class ProjectUpdate(BaseModel): # Partial updates (all optional)
|
||||
name: Optional[str] = Field(None, min_length=1)
|
||||
|
||||
class Project(ProjectBase): # API response
|
||||
id: str
|
||||
created_at: datetime = Field(..., alias="createdAt")
|
||||
|
||||
class ProjectInDB(Project): # Internal with docType
|
||||
doc_type: str = "project"
|
||||
```
|
||||
|
||||
### 3. Service Layer Pattern
|
||||
|
||||
```python
|
||||
class ProjectService:
|
||||
def _use_cosmos(self) -> bool:
|
||||
return get_container() is not None
|
||||
|
||||
async def get_by_id(self, project_id: str, workspace_id: str) -> Project | None:
|
||||
if not self._use_cosmos():
|
||||
return None
|
||||
doc = await get_document(project_id, partition_key=workspace_id)
|
||||
if doc is None:
|
||||
return None
|
||||
return self._doc_to_model(doc)
|
||||
```
|
||||
|
||||
**Full patterns**: See [references/service-layer.md](references/service-layer.md)
|
||||
|
||||
## Core Principles
|
||||
|
||||
### Security Requirements
|
||||
|
||||
1. **RBAC Authentication**: Use `DefaultAzureCredential` in Azure — never store keys in code
|
||||
2. **Emulator-Only Keys**: Hardcode the well-known emulator key only for local development
|
||||
3. **Parameterized Queries**: Always use `@parameter` syntax — never string concatenation
|
||||
4. **Partition Key Validation**: Validate partition key access matches user authorization
|
||||
|
||||
### Clean Code Conventions
|
||||
|
||||
1. **Single Responsibility**: Client module handles connection; services handle business logic
|
||||
2. **Graceful Degradation**: Services return `None`/`[]` when Cosmos unavailable
|
||||
3. **Consistent Naming**: `_doc_to_model()`, `_model_to_doc()`, `_use_cosmos()`
|
||||
4. **Type Hints**: Full typing on all public methods
|
||||
5. **CamelCase Aliases**: Use `Field(alias="camelCase")` for JSON serialization
|
||||
|
||||
### TDD Requirements
|
||||
|
||||
Write tests BEFORE implementation using these patterns:
|
||||
|
||||
```python
|
||||
@pytest.fixture
|
||||
def mock_cosmos_container(mocker):
|
||||
container = mocker.MagicMock()
|
||||
mocker.patch("app.db.cosmos.get_container", return_value=container)
|
||||
return container
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_project_by_id_returns_project(mock_cosmos_container):
|
||||
# Arrange
|
||||
mock_cosmos_container.read_item.return_value = {"id": "123", "name": "Test"}
|
||||
|
||||
# Act
|
||||
result = await project_service.get_by_id("123", "workspace-1")
|
||||
|
||||
# Assert
|
||||
assert result.id == "123"
|
||||
assert result.name == "Test"
|
||||
```
|
||||
|
||||
**Full testing guide**: See [references/testing.md](references/testing.md)
|
||||
|
||||
## Reference Files
|
||||
|
||||
| File | When to Read |
|
||||
|------|--------------|
|
||||
| [references/client-setup.md](references/client-setup.md) | Setting up Cosmos client with dual auth, SSL config, singleton pattern |
|
||||
| [references/service-layer.md](references/service-layer.md) | Implementing full service class with CRUD, conversions, graceful degradation |
|
||||
| [references/testing.md](references/testing.md) | Writing pytest tests, mocking Cosmos, integration test setup |
|
||||
| [references/partitioning.md](references/partitioning.md) | Choosing partition keys, cross-partition queries, move operations |
|
||||
| [references/error-handling.md](references/error-handling.md) | Handling CosmosResourceNotFoundError, logging, HTTP error mapping |
|
||||
|
||||
## Template Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| [assets/cosmos_client_template.py](assets/cosmos_client_template.py) | Ready-to-use client module |
|
||||
| [assets/service_template.py](assets/service_template.py) | Service class skeleton |
|
||||
| [assets/conftest_template.py](assets/conftest_template.py) | pytest fixtures for Cosmos mocking |
|
||||
|
||||
## Quality Attributes (NFRs)
|
||||
|
||||
### Reliability
|
||||
- Graceful degradation when Cosmos unavailable
|
||||
- Retry logic with exponential backoff for transient failures
|
||||
- Connection pooling via singleton pattern
|
||||
|
||||
### Security
|
||||
- Zero secrets in code (RBAC via DefaultAzureCredential)
|
||||
- Parameterized queries prevent injection
|
||||
- Partition key isolation enforces data boundaries
|
||||
|
||||
### Maintainability
|
||||
- Five-tier model pattern enables schema evolution
|
||||
- Service layer decouples business logic from storage
|
||||
- Consistent patterns across all entity services
|
||||
|
||||
### Testability
|
||||
- Dependency injection via `get_container()`
|
||||
- Easy mocking with module-level globals
|
||||
- Clear separation enables unit testing without Cosmos
|
||||
|
||||
### Performance
|
||||
- Partition key queries avoid cross-partition scans
|
||||
- Async wrapping prevents blocking FastAPI event loop
|
||||
- Minimal document conversion overhead
|
||||
280
skills/official/microsoft/python/data/cosmos/SKILL.md
Normal file
280
skills/official/microsoft/python/data/cosmos/SKILL.md
Normal file
@@ -0,0 +1,280 @@
|
||||
---
|
||||
name: azure-cosmos-py
|
||||
description: |
|
||||
Azure Cosmos DB SDK for Python (NoSQL API). Use for document CRUD, queries, containers, and globally distributed data.
|
||||
Triggers: "cosmos db", "CosmosClient", "container", "document", "NoSQL", "partition key".
|
||||
package: azure-cosmos
|
||||
---
|
||||
|
||||
# Azure Cosmos DB SDK for Python
|
||||
|
||||
Client library for Azure Cosmos DB NoSQL API — globally distributed, multi-model database.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-cosmos azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
COSMOS_ENDPOINT=https://<account>.documents.azure.com:443/
|
||||
COSMOS_DATABASE=mydb
|
||||
COSMOS_CONTAINER=mycontainer
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.cosmos import CosmosClient
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
endpoint = "https://<account>.documents.azure.com:443/"
|
||||
|
||||
client = CosmosClient(url=endpoint, credential=credential)
|
||||
```
|
||||
|
||||
## Client Hierarchy
|
||||
|
||||
| Client | Purpose | Get From |
|
||||
|--------|---------|----------|
|
||||
| `CosmosClient` | Account-level operations | Direct instantiation |
|
||||
| `DatabaseProxy` | Database operations | `client.get_database_client()` |
|
||||
| `ContainerProxy` | Container/item operations | `database.get_container_client()` |
|
||||
|
||||
## Core Workflow
|
||||
|
||||
### Setup Database and Container
|
||||
|
||||
```python
|
||||
# Get or create database
|
||||
database = client.create_database_if_not_exists(id="mydb")
|
||||
|
||||
# Get or create container with partition key
|
||||
container = database.create_container_if_not_exists(
|
||||
id="mycontainer",
|
||||
partition_key=PartitionKey(path="/category")
|
||||
)
|
||||
|
||||
# Get existing
|
||||
database = client.get_database_client("mydb")
|
||||
container = database.get_container_client("mycontainer")
|
||||
```
|
||||
|
||||
### Create Item
|
||||
|
||||
```python
|
||||
item = {
|
||||
"id": "item-001", # Required: unique within partition
|
||||
"category": "electronics", # Partition key value
|
||||
"name": "Laptop",
|
||||
"price": 999.99,
|
||||
"tags": ["computer", "portable"]
|
||||
}
|
||||
|
||||
created = container.create_item(body=item)
|
||||
print(f"Created: {created['id']}")
|
||||
```
|
||||
|
||||
### Read Item
|
||||
|
||||
```python
|
||||
# Read requires id AND partition key
|
||||
item = container.read_item(
|
||||
item="item-001",
|
||||
partition_key="electronics"
|
||||
)
|
||||
print(f"Name: {item['name']}")
|
||||
```
|
||||
|
||||
### Update Item (Replace)
|
||||
|
||||
```python
|
||||
item = container.read_item(item="item-001", partition_key="electronics")
|
||||
item["price"] = 899.99
|
||||
item["on_sale"] = True
|
||||
|
||||
updated = container.replace_item(item=item["id"], body=item)
|
||||
```
|
||||
|
||||
### Upsert Item
|
||||
|
||||
```python
|
||||
# Create if not exists, replace if exists
|
||||
item = {
|
||||
"id": "item-002",
|
||||
"category": "electronics",
|
||||
"name": "Tablet",
|
||||
"price": 499.99
|
||||
}
|
||||
|
||||
result = container.upsert_item(body=item)
|
||||
```
|
||||
|
||||
### Delete Item
|
||||
|
||||
```python
|
||||
container.delete_item(
|
||||
item="item-001",
|
||||
partition_key="electronics"
|
||||
)
|
||||
```
|
||||
|
||||
## Queries
|
||||
|
||||
### Basic Query
|
||||
|
||||
```python
|
||||
# Query within a partition (efficient)
|
||||
query = "SELECT * FROM c WHERE c.price < @max_price"
|
||||
items = container.query_items(
|
||||
query=query,
|
||||
parameters=[{"name": "@max_price", "value": 500}],
|
||||
partition_key="electronics"
|
||||
)
|
||||
|
||||
for item in items:
|
||||
print(f"{item['name']}: ${item['price']}")
|
||||
```
|
||||
|
||||
### Cross-Partition Query
|
||||
|
||||
```python
|
||||
# Cross-partition (more expensive, use sparingly)
|
||||
query = "SELECT * FROM c WHERE c.price < @max_price"
|
||||
items = container.query_items(
|
||||
query=query,
|
||||
parameters=[{"name": "@max_price", "value": 500}],
|
||||
enable_cross_partition_query=True
|
||||
)
|
||||
|
||||
for item in items:
|
||||
print(item)
|
||||
```
|
||||
|
||||
### Query with Projection
|
||||
|
||||
```python
|
||||
query = "SELECT c.id, c.name, c.price FROM c WHERE c.category = @category"
|
||||
items = container.query_items(
|
||||
query=query,
|
||||
parameters=[{"name": "@category", "value": "electronics"}],
|
||||
partition_key="electronics"
|
||||
)
|
||||
```
|
||||
|
||||
### Read All Items
|
||||
|
||||
```python
|
||||
# Read all in a partition
|
||||
items = container.read_all_items() # Cross-partition
|
||||
# Or with partition key
|
||||
items = container.query_items(
|
||||
query="SELECT * FROM c",
|
||||
partition_key="electronics"
|
||||
)
|
||||
```
|
||||
|
||||
## Partition Keys
|
||||
|
||||
**Critical**: Always include partition key for efficient operations.
|
||||
|
||||
```python
|
||||
from azure.cosmos import PartitionKey
|
||||
|
||||
# Single partition key
|
||||
container = database.create_container_if_not_exists(
|
||||
id="orders",
|
||||
partition_key=PartitionKey(path="/customer_id")
|
||||
)
|
||||
|
||||
# Hierarchical partition key (preview)
|
||||
container = database.create_container_if_not_exists(
|
||||
id="events",
|
||||
partition_key=PartitionKey(path=["/tenant_id", "/user_id"])
|
||||
)
|
||||
```
|
||||
|
||||
## Throughput
|
||||
|
||||
```python
|
||||
# Create container with provisioned throughput
|
||||
container = database.create_container_if_not_exists(
|
||||
id="mycontainer",
|
||||
partition_key=PartitionKey(path="/pk"),
|
||||
offer_throughput=400 # RU/s
|
||||
)
|
||||
|
||||
# Read current throughput
|
||||
offer = container.read_offer()
|
||||
print(f"Throughput: {offer.offer_throughput} RU/s")
|
||||
|
||||
# Update throughput
|
||||
container.replace_throughput(throughput=1000)
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.cosmos.aio import CosmosClient
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def cosmos_operations():
|
||||
credential = DefaultAzureCredential()
|
||||
|
||||
async with CosmosClient(endpoint, credential=credential) as client:
|
||||
database = client.get_database_client("mydb")
|
||||
container = database.get_container_client("mycontainer")
|
||||
|
||||
# Create
|
||||
await container.create_item(body={"id": "1", "pk": "test"})
|
||||
|
||||
# Read
|
||||
item = await container.read_item(item="1", partition_key="test")
|
||||
|
||||
# Query
|
||||
async for item in container.query_items(
|
||||
query="SELECT * FROM c",
|
||||
partition_key="test"
|
||||
):
|
||||
print(item)
|
||||
|
||||
import asyncio
|
||||
asyncio.run(cosmos_operations())
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
```python
|
||||
from azure.cosmos.exceptions import CosmosHttpResponseError
|
||||
|
||||
try:
|
||||
item = container.read_item(item="nonexistent", partition_key="pk")
|
||||
except CosmosHttpResponseError as e:
|
||||
if e.status_code == 404:
|
||||
print("Item not found")
|
||||
elif e.status_code == 429:
|
||||
print(f"Rate limited. Retry after: {e.headers.get('x-ms-retry-after-ms')}ms")
|
||||
else:
|
||||
raise
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Always specify partition key** for point reads and queries
|
||||
2. **Use parameterized queries** to prevent injection and improve caching
|
||||
3. **Avoid cross-partition queries** when possible
|
||||
4. **Use `upsert_item`** for idempotent writes
|
||||
5. **Use async client** for high-throughput scenarios
|
||||
6. **Design partition key** for even data distribution
|
||||
7. **Use `read_item`** instead of query for single document retrieval
|
||||
|
||||
## Reference Files
|
||||
|
||||
| File | Contents |
|
||||
|------|----------|
|
||||
| [references/partitioning.md](references/partitioning.md) | Partition key strategies, hierarchical keys, hot partition detection and mitigation |
|
||||
| [references/query-patterns.md](references/query-patterns.md) | Query optimization, aggregations, pagination, transactions, change feed |
|
||||
| [scripts/setup_cosmos_container.py](scripts/setup_cosmos_container.py) | CLI tool for creating containers with partitioning, throughput, and indexing |
|
||||
211
skills/official/microsoft/python/data/datalake/SKILL.md
Normal file
211
skills/official/microsoft/python/data/datalake/SKILL.md
Normal file
@@ -0,0 +1,211 @@
|
||||
---
|
||||
name: azure-storage-file-datalake-py
|
||||
description: |
|
||||
Azure Data Lake Storage Gen2 SDK for Python. Use for hierarchical file systems, big data analytics, and file/directory operations.
|
||||
Triggers: "data lake", "DataLakeServiceClient", "FileSystemClient", "ADLS Gen2", "hierarchical namespace".
|
||||
package: azure-storage-file-datalake
|
||||
---
|
||||
|
||||
# Azure Data Lake Storage Gen2 SDK for Python
|
||||
|
||||
Hierarchical file system for big data analytics workloads.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-storage-file-datalake azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_STORAGE_ACCOUNT_URL=https://<account>.dfs.core.windows.net
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.storage.filedatalake import DataLakeServiceClient
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
account_url = "https://<account>.dfs.core.windows.net"
|
||||
|
||||
service_client = DataLakeServiceClient(account_url=account_url, credential=credential)
|
||||
```
|
||||
|
||||
## Client Hierarchy
|
||||
|
||||
| Client | Purpose |
|
||||
|--------|---------|
|
||||
| `DataLakeServiceClient` | Account-level operations |
|
||||
| `FileSystemClient` | Container (file system) operations |
|
||||
| `DataLakeDirectoryClient` | Directory operations |
|
||||
| `DataLakeFileClient` | File operations |
|
||||
|
||||
## File System Operations
|
||||
|
||||
```python
|
||||
# Create file system (container)
|
||||
file_system_client = service_client.create_file_system("myfilesystem")
|
||||
|
||||
# Get existing
|
||||
file_system_client = service_client.get_file_system_client("myfilesystem")
|
||||
|
||||
# Delete
|
||||
service_client.delete_file_system("myfilesystem")
|
||||
|
||||
# List file systems
|
||||
for fs in service_client.list_file_systems():
|
||||
print(fs.name)
|
||||
```
|
||||
|
||||
## Directory Operations
|
||||
|
||||
```python
|
||||
file_system_client = service_client.get_file_system_client("myfilesystem")
|
||||
|
||||
# Create directory
|
||||
directory_client = file_system_client.create_directory("mydir")
|
||||
|
||||
# Create nested directories
|
||||
directory_client = file_system_client.create_directory("path/to/nested/dir")
|
||||
|
||||
# Get directory client
|
||||
directory_client = file_system_client.get_directory_client("mydir")
|
||||
|
||||
# Delete directory
|
||||
directory_client.delete_directory()
|
||||
|
||||
# Rename/move directory
|
||||
directory_client.rename_directory(new_name="myfilesystem/newname")
|
||||
```
|
||||
|
||||
## File Operations
|
||||
|
||||
### Upload File
|
||||
|
||||
```python
|
||||
# Get file client
|
||||
file_client = file_system_client.get_file_client("path/to/file.txt")
|
||||
|
||||
# Upload from local file
|
||||
with open("local-file.txt", "rb") as data:
|
||||
file_client.upload_data(data, overwrite=True)
|
||||
|
||||
# Upload bytes
|
||||
file_client.upload_data(b"Hello, Data Lake!", overwrite=True)
|
||||
|
||||
# Append data (for large files)
|
||||
file_client.append_data(data=b"chunk1", offset=0, length=6)
|
||||
file_client.append_data(data=b"chunk2", offset=6, length=6)
|
||||
file_client.flush_data(12) # Commit the data
|
||||
```
|
||||
|
||||
### Download File
|
||||
|
||||
```python
|
||||
file_client = file_system_client.get_file_client("path/to/file.txt")
|
||||
|
||||
# Download all content
|
||||
download = file_client.download_file()
|
||||
content = download.readall()
|
||||
|
||||
# Download to file
|
||||
with open("downloaded.txt", "wb") as f:
|
||||
download = file_client.download_file()
|
||||
download.readinto(f)
|
||||
|
||||
# Download range
|
||||
download = file_client.download_file(offset=0, length=100)
|
||||
```
|
||||
|
||||
### Delete File
|
||||
|
||||
```python
|
||||
file_client.delete_file()
|
||||
```
|
||||
|
||||
## List Contents
|
||||
|
||||
```python
|
||||
# List paths (files and directories)
|
||||
for path in file_system_client.get_paths():
|
||||
print(f"{'DIR' if path.is_directory else 'FILE'}: {path.name}")
|
||||
|
||||
# List paths in directory
|
||||
for path in file_system_client.get_paths(path="mydir"):
|
||||
print(path.name)
|
||||
|
||||
# Recursive listing
|
||||
for path in file_system_client.get_paths(path="mydir", recursive=True):
|
||||
print(path.name)
|
||||
```
|
||||
|
||||
## File/Directory Properties
|
||||
|
||||
```python
|
||||
# Get properties
|
||||
properties = file_client.get_file_properties()
|
||||
print(f"Size: {properties.size}")
|
||||
print(f"Last modified: {properties.last_modified}")
|
||||
|
||||
# Set metadata
|
||||
file_client.set_metadata(metadata={"processed": "true"})
|
||||
```
|
||||
|
||||
## Access Control (ACL)
|
||||
|
||||
```python
|
||||
# Get ACL
|
||||
acl = directory_client.get_access_control()
|
||||
print(f"Owner: {acl['owner']}")
|
||||
print(f"Permissions: {acl['permissions']}")
|
||||
|
||||
# Set ACL
|
||||
directory_client.set_access_control(
|
||||
owner="user-id",
|
||||
permissions="rwxr-x---"
|
||||
)
|
||||
|
||||
# Update ACL entries
|
||||
from azure.storage.filedatalake import AccessControlChangeResult
|
||||
directory_client.update_access_control_recursive(
|
||||
acl="user:user-id:rwx"
|
||||
)
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.storage.filedatalake.aio import DataLakeServiceClient
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def datalake_operations():
|
||||
credential = DefaultAzureCredential()
|
||||
|
||||
async with DataLakeServiceClient(
|
||||
account_url="https://<account>.dfs.core.windows.net",
|
||||
credential=credential
|
||||
) as service_client:
|
||||
file_system_client = service_client.get_file_system_client("myfilesystem")
|
||||
file_client = file_system_client.get_file_client("test.txt")
|
||||
|
||||
await file_client.upload_data(b"async content", overwrite=True)
|
||||
|
||||
download = await file_client.download_file()
|
||||
content = await download.readall()
|
||||
|
||||
import asyncio
|
||||
asyncio.run(datalake_operations())
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use hierarchical namespace** for file system semantics
|
||||
2. **Use `append_data` + `flush_data`** for large file uploads
|
||||
3. **Set ACLs at directory level** and inherit to children
|
||||
4. **Use async client** for high-throughput scenarios
|
||||
5. **Use `get_paths` with `recursive=True`** for full directory listing
|
||||
6. **Set metadata** for custom file attributes
|
||||
7. **Consider Blob API** for simple object storage use cases
|
||||
238
skills/official/microsoft/python/data/fileshare/SKILL.md
Normal file
238
skills/official/microsoft/python/data/fileshare/SKILL.md
Normal file
@@ -0,0 +1,238 @@
|
||||
---
|
||||
name: azure-storage-file-share-py
|
||||
description: |
|
||||
Azure Storage File Share SDK for Python. Use for SMB file shares, directories, and file operations in the cloud.
|
||||
Triggers: "azure-storage-file-share", "ShareServiceClient", "ShareClient", "file share", "SMB".
|
||||
---
|
||||
|
||||
# Azure Storage File Share SDK for Python
|
||||
|
||||
Manage SMB file shares for cloud-native and lift-and-shift scenarios.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-storage-file-share
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_STORAGE_CONNECTION_STRING=DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...
|
||||
# Or
|
||||
AZURE_STORAGE_ACCOUNT_URL=https://<account>.file.core.windows.net
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
### Connection String
|
||||
|
||||
```python
|
||||
from azure.storage.fileshare import ShareServiceClient
|
||||
|
||||
service = ShareServiceClient.from_connection_string(
|
||||
os.environ["AZURE_STORAGE_CONNECTION_STRING"]
|
||||
)
|
||||
```
|
||||
|
||||
### Entra ID
|
||||
|
||||
```python
|
||||
from azure.storage.fileshare import ShareServiceClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
service = ShareServiceClient(
|
||||
account_url=os.environ["AZURE_STORAGE_ACCOUNT_URL"],
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
```
|
||||
|
||||
## Share Operations
|
||||
|
||||
### Create Share
|
||||
|
||||
```python
|
||||
share = service.create_share("my-share")
|
||||
```
|
||||
|
||||
### List Shares
|
||||
|
||||
```python
|
||||
for share in service.list_shares():
|
||||
print(f"{share.name}: {share.quota} GB")
|
||||
```
|
||||
|
||||
### Get Share Client
|
||||
|
||||
```python
|
||||
share_client = service.get_share_client("my-share")
|
||||
```
|
||||
|
||||
### Delete Share
|
||||
|
||||
```python
|
||||
service.delete_share("my-share")
|
||||
```
|
||||
|
||||
## Directory Operations
|
||||
|
||||
### Create Directory
|
||||
|
||||
```python
|
||||
share_client = service.get_share_client("my-share")
|
||||
share_client.create_directory("my-directory")
|
||||
|
||||
# Nested directory
|
||||
share_client.create_directory("my-directory/sub-directory")
|
||||
```
|
||||
|
||||
### List Directories and Files
|
||||
|
||||
```python
|
||||
directory_client = share_client.get_directory_client("my-directory")
|
||||
|
||||
for item in directory_client.list_directories_and_files():
|
||||
if item["is_directory"]:
|
||||
print(f"[DIR] {item['name']}")
|
||||
else:
|
||||
print(f"[FILE] {item['name']} ({item['size']} bytes)")
|
||||
```
|
||||
|
||||
### Delete Directory
|
||||
|
||||
```python
|
||||
share_client.delete_directory("my-directory")
|
||||
```
|
||||
|
||||
## File Operations
|
||||
|
||||
### Upload File
|
||||
|
||||
```python
|
||||
file_client = share_client.get_file_client("my-directory/file.txt")
|
||||
|
||||
# From string
|
||||
file_client.upload_file("Hello, World!")
|
||||
|
||||
# From file
|
||||
with open("local-file.txt", "rb") as f:
|
||||
file_client.upload_file(f)
|
||||
|
||||
# From bytes
|
||||
file_client.upload_file(b"Binary content")
|
||||
```
|
||||
|
||||
### Download File
|
||||
|
||||
```python
|
||||
file_client = share_client.get_file_client("my-directory/file.txt")
|
||||
|
||||
# To bytes
|
||||
data = file_client.download_file().readall()
|
||||
|
||||
# To file
|
||||
with open("downloaded.txt", "wb") as f:
|
||||
data = file_client.download_file()
|
||||
data.readinto(f)
|
||||
|
||||
# Stream chunks
|
||||
download = file_client.download_file()
|
||||
for chunk in download.chunks():
|
||||
process(chunk)
|
||||
```
|
||||
|
||||
### Get File Properties
|
||||
|
||||
```python
|
||||
properties = file_client.get_file_properties()
|
||||
print(f"Size: {properties.size}")
|
||||
print(f"Content type: {properties.content_settings.content_type}")
|
||||
print(f"Last modified: {properties.last_modified}")
|
||||
```
|
||||
|
||||
### Delete File
|
||||
|
||||
```python
|
||||
file_client.delete_file()
|
||||
```
|
||||
|
||||
### Copy File
|
||||
|
||||
```python
|
||||
source_url = "https://account.file.core.windows.net/share/source.txt"
|
||||
dest_client = share_client.get_file_client("destination.txt")
|
||||
dest_client.start_copy_from_url(source_url)
|
||||
```
|
||||
|
||||
## Range Operations
|
||||
|
||||
### Upload Range
|
||||
|
||||
```python
|
||||
# Upload to specific range
|
||||
file_client.upload_range(data=b"content", offset=0, length=7)
|
||||
```
|
||||
|
||||
### Download Range
|
||||
|
||||
```python
|
||||
# Download specific range
|
||||
download = file_client.download_file(offset=0, length=100)
|
||||
data = download.readall()
|
||||
```
|
||||
|
||||
## Snapshot Operations
|
||||
|
||||
### Create Snapshot
|
||||
|
||||
```python
|
||||
snapshot = share_client.create_snapshot()
|
||||
print(f"Snapshot: {snapshot['snapshot']}")
|
||||
```
|
||||
|
||||
### Access Snapshot
|
||||
|
||||
```python
|
||||
snapshot_client = service.get_share_client(
|
||||
"my-share",
|
||||
snapshot=snapshot["snapshot"]
|
||||
)
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.storage.fileshare.aio import ShareServiceClient
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def upload_file():
|
||||
credential = DefaultAzureCredential()
|
||||
service = ShareServiceClient(account_url, credential=credential)
|
||||
|
||||
share = service.get_share_client("my-share")
|
||||
file_client = share.get_file_client("test.txt")
|
||||
|
||||
await file_client.upload_file("Hello!")
|
||||
|
||||
await service.close()
|
||||
await credential.close()
|
||||
```
|
||||
|
||||
## Client Types
|
||||
|
||||
| Client | Purpose |
|
||||
|--------|---------|
|
||||
| `ShareServiceClient` | Account-level operations |
|
||||
| `ShareClient` | Share operations |
|
||||
| `ShareDirectoryClient` | Directory operations |
|
||||
| `ShareFileClient` | File operations |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use connection string** for simplest setup
|
||||
2. **Use Entra ID** for production with RBAC
|
||||
3. **Stream large files** using chunks() to avoid memory issues
|
||||
4. **Create snapshots** before major changes
|
||||
5. **Set quotas** to prevent unexpected storage costs
|
||||
6. **Use ranges** for partial file updates
|
||||
7. **Close async clients** explicitly
|
||||
213
skills/official/microsoft/python/data/queue/SKILL.md
Normal file
213
skills/official/microsoft/python/data/queue/SKILL.md
Normal file
@@ -0,0 +1,213 @@
|
||||
---
|
||||
name: azure-storage-queue-py
|
||||
description: |
|
||||
Azure Queue Storage SDK for Python. Use for reliable message queuing, task distribution, and asynchronous processing.
|
||||
Triggers: "queue storage", "QueueServiceClient", "QueueClient", "message queue", "dequeue".
|
||||
package: azure-storage-queue
|
||||
---
|
||||
|
||||
# Azure Queue Storage SDK for Python
|
||||
|
||||
Simple, cost-effective message queuing for asynchronous communication.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-storage-queue azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_STORAGE_ACCOUNT_URL=https://<account>.queue.core.windows.net
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.storage.queue import QueueServiceClient, QueueClient
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
account_url = "https://<account>.queue.core.windows.net"
|
||||
|
||||
# Service client
|
||||
service_client = QueueServiceClient(account_url=account_url, credential=credential)
|
||||
|
||||
# Queue client
|
||||
queue_client = QueueClient(account_url=account_url, queue_name="myqueue", credential=credential)
|
||||
```
|
||||
|
||||
## Queue Operations
|
||||
|
||||
```python
|
||||
# Create queue
|
||||
service_client.create_queue("myqueue")
|
||||
|
||||
# Get queue client
|
||||
queue_client = service_client.get_queue_client("myqueue")
|
||||
|
||||
# Delete queue
|
||||
service_client.delete_queue("myqueue")
|
||||
|
||||
# List queues
|
||||
for queue in service_client.list_queues():
|
||||
print(queue.name)
|
||||
```
|
||||
|
||||
## Send Messages
|
||||
|
||||
```python
|
||||
# Send message (string)
|
||||
queue_client.send_message("Hello, Queue!")
|
||||
|
||||
# Send with options
|
||||
queue_client.send_message(
|
||||
content="Delayed message",
|
||||
visibility_timeout=60, # Hidden for 60 seconds
|
||||
time_to_live=3600 # Expires in 1 hour
|
||||
)
|
||||
|
||||
# Send JSON
|
||||
import json
|
||||
data = {"task": "process", "id": 123}
|
||||
queue_client.send_message(json.dumps(data))
|
||||
```
|
||||
|
||||
## Receive Messages
|
||||
|
||||
```python
|
||||
# Receive messages (makes them invisible temporarily)
|
||||
messages = queue_client.receive_messages(
|
||||
messages_per_page=10,
|
||||
visibility_timeout=30 # 30 seconds to process
|
||||
)
|
||||
|
||||
for message in messages:
|
||||
print(f"ID: {message.id}")
|
||||
print(f"Content: {message.content}")
|
||||
print(f"Dequeue count: {message.dequeue_count}")
|
||||
|
||||
# Process message...
|
||||
|
||||
# Delete after processing
|
||||
queue_client.delete_message(message)
|
||||
```
|
||||
|
||||
## Peek Messages
|
||||
|
||||
```python
|
||||
# Peek without hiding (doesn't affect visibility)
|
||||
messages = queue_client.peek_messages(max_messages=5)
|
||||
|
||||
for message in messages:
|
||||
print(message.content)
|
||||
```
|
||||
|
||||
## Update Message
|
||||
|
||||
```python
|
||||
# Extend visibility or update content
|
||||
messages = queue_client.receive_messages()
|
||||
for message in messages:
|
||||
# Extend timeout (need more time)
|
||||
queue_client.update_message(
|
||||
message,
|
||||
visibility_timeout=60
|
||||
)
|
||||
|
||||
# Update content and timeout
|
||||
queue_client.update_message(
|
||||
message,
|
||||
content="Updated content",
|
||||
visibility_timeout=60
|
||||
)
|
||||
```
|
||||
|
||||
## Delete Message
|
||||
|
||||
```python
|
||||
# Delete after successful processing
|
||||
messages = queue_client.receive_messages()
|
||||
for message in messages:
|
||||
try:
|
||||
# Process...
|
||||
queue_client.delete_message(message)
|
||||
except Exception:
|
||||
# Message becomes visible again after timeout
|
||||
pass
|
||||
```
|
||||
|
||||
## Clear Queue
|
||||
|
||||
```python
|
||||
# Delete all messages
|
||||
queue_client.clear_messages()
|
||||
```
|
||||
|
||||
## Queue Properties
|
||||
|
||||
```python
|
||||
# Get queue properties
|
||||
properties = queue_client.get_queue_properties()
|
||||
print(f"Approximate message count: {properties.approximate_message_count}")
|
||||
|
||||
# Set/get metadata
|
||||
queue_client.set_queue_metadata(metadata={"environment": "production"})
|
||||
properties = queue_client.get_queue_properties()
|
||||
print(properties.metadata)
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.storage.queue.aio import QueueServiceClient, QueueClient
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def queue_operations():
|
||||
credential = DefaultAzureCredential()
|
||||
|
||||
async with QueueClient(
|
||||
account_url="https://<account>.queue.core.windows.net",
|
||||
queue_name="myqueue",
|
||||
credential=credential
|
||||
) as client:
|
||||
# Send
|
||||
await client.send_message("Async message")
|
||||
|
||||
# Receive
|
||||
async for message in client.receive_messages():
|
||||
print(message.content)
|
||||
await client.delete_message(message)
|
||||
|
||||
import asyncio
|
||||
asyncio.run(queue_operations())
|
||||
```
|
||||
|
||||
## Base64 Encoding
|
||||
|
||||
```python
|
||||
from azure.storage.queue import QueueClient, BinaryBase64EncodePolicy, BinaryBase64DecodePolicy
|
||||
|
||||
# For binary data
|
||||
queue_client = QueueClient(
|
||||
account_url=account_url,
|
||||
queue_name="myqueue",
|
||||
credential=credential,
|
||||
message_encode_policy=BinaryBase64EncodePolicy(),
|
||||
message_decode_policy=BinaryBase64DecodePolicy()
|
||||
)
|
||||
|
||||
# Send bytes
|
||||
queue_client.send_message(b"Binary content")
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Delete messages after processing** to prevent reprocessing
|
||||
2. **Set appropriate visibility timeout** based on processing time
|
||||
3. **Handle `dequeue_count`** for poison message detection
|
||||
4. **Use async client** for high-throughput scenarios
|
||||
5. **Use `peek_messages`** for monitoring without affecting queue
|
||||
6. **Set `time_to_live`** to prevent stale messages
|
||||
7. **Consider Service Bus** for advanced features (sessions, topics)
|
||||
243
skills/official/microsoft/python/data/tables/SKILL.md
Normal file
243
skills/official/microsoft/python/data/tables/SKILL.md
Normal file
@@ -0,0 +1,243 @@
|
||||
---
|
||||
name: azure-data-tables-py
|
||||
description: |
|
||||
Azure Tables SDK for Python (Storage and Cosmos DB). Use for NoSQL key-value storage, entity CRUD, and batch operations.
|
||||
Triggers: "table storage", "TableServiceClient", "TableClient", "entities", "PartitionKey", "RowKey".
|
||||
package: azure-data-tables
|
||||
---
|
||||
|
||||
# Azure Tables SDK for Python
|
||||
|
||||
NoSQL key-value store for structured data (Azure Storage Tables or Cosmos DB Table API).
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-data-tables azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
# Azure Storage Tables
|
||||
AZURE_STORAGE_ACCOUNT_URL=https://<account>.table.core.windows.net
|
||||
|
||||
# Cosmos DB Table API
|
||||
COSMOS_TABLE_ENDPOINT=https://<account>.table.cosmos.azure.com
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.data.tables import TableServiceClient, TableClient
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
endpoint = "https://<account>.table.core.windows.net"
|
||||
|
||||
# Service client (manage tables)
|
||||
service_client = TableServiceClient(endpoint=endpoint, credential=credential)
|
||||
|
||||
# Table client (work with entities)
|
||||
table_client = TableClient(endpoint=endpoint, table_name="mytable", credential=credential)
|
||||
```
|
||||
|
||||
## Client Types
|
||||
|
||||
| Client | Purpose |
|
||||
|--------|---------|
|
||||
| `TableServiceClient` | Create/delete tables, list tables |
|
||||
| `TableClient` | Entity CRUD, queries |
|
||||
|
||||
## Table Operations
|
||||
|
||||
```python
|
||||
# Create table
|
||||
service_client.create_table("mytable")
|
||||
|
||||
# Create if not exists
|
||||
service_client.create_table_if_not_exists("mytable")
|
||||
|
||||
# Delete table
|
||||
service_client.delete_table("mytable")
|
||||
|
||||
# List tables
|
||||
for table in service_client.list_tables():
|
||||
print(table.name)
|
||||
|
||||
# Get table client
|
||||
table_client = service_client.get_table_client("mytable")
|
||||
```
|
||||
|
||||
## Entity Operations
|
||||
|
||||
**Important**: Every entity requires `PartitionKey` and `RowKey` (together form unique ID).
|
||||
|
||||
### Create Entity
|
||||
|
||||
```python
|
||||
entity = {
|
||||
"PartitionKey": "sales",
|
||||
"RowKey": "order-001",
|
||||
"product": "Widget",
|
||||
"quantity": 5,
|
||||
"price": 9.99,
|
||||
"shipped": False
|
||||
}
|
||||
|
||||
# Create (fails if exists)
|
||||
table_client.create_entity(entity=entity)
|
||||
|
||||
# Upsert (create or replace)
|
||||
table_client.upsert_entity(entity=entity)
|
||||
```
|
||||
|
||||
### Get Entity
|
||||
|
||||
```python
|
||||
# Get by key (fastest)
|
||||
entity = table_client.get_entity(
|
||||
partition_key="sales",
|
||||
row_key="order-001"
|
||||
)
|
||||
print(f"Product: {entity['product']}")
|
||||
```
|
||||
|
||||
### Update Entity
|
||||
|
||||
```python
|
||||
# Replace entire entity
|
||||
entity["quantity"] = 10
|
||||
table_client.update_entity(entity=entity, mode="replace")
|
||||
|
||||
# Merge (update specific fields only)
|
||||
update = {
|
||||
"PartitionKey": "sales",
|
||||
"RowKey": "order-001",
|
||||
"shipped": True
|
||||
}
|
||||
table_client.update_entity(entity=update, mode="merge")
|
||||
```
|
||||
|
||||
### Delete Entity
|
||||
|
||||
```python
|
||||
table_client.delete_entity(
|
||||
partition_key="sales",
|
||||
row_key="order-001"
|
||||
)
|
||||
```
|
||||
|
||||
## Query Entities
|
||||
|
||||
### Query Within Partition
|
||||
|
||||
```python
|
||||
# Query by partition (efficient)
|
||||
entities = table_client.query_entities(
|
||||
query_filter="PartitionKey eq 'sales'"
|
||||
)
|
||||
for entity in entities:
|
||||
print(entity)
|
||||
```
|
||||
|
||||
### Query with Filters
|
||||
|
||||
```python
|
||||
# Filter by properties
|
||||
entities = table_client.query_entities(
|
||||
query_filter="PartitionKey eq 'sales' and quantity gt 3"
|
||||
)
|
||||
|
||||
# With parameters (safer)
|
||||
entities = table_client.query_entities(
|
||||
query_filter="PartitionKey eq @pk and price lt @max_price",
|
||||
parameters={"pk": "sales", "max_price": 50.0}
|
||||
)
|
||||
```
|
||||
|
||||
### Select Specific Properties
|
||||
|
||||
```python
|
||||
entities = table_client.query_entities(
|
||||
query_filter="PartitionKey eq 'sales'",
|
||||
select=["RowKey", "product", "price"]
|
||||
)
|
||||
```
|
||||
|
||||
### List All Entities
|
||||
|
||||
```python
|
||||
# List all (cross-partition - use sparingly)
|
||||
for entity in table_client.list_entities():
|
||||
print(entity)
|
||||
```
|
||||
|
||||
## Batch Operations
|
||||
|
||||
```python
|
||||
from azure.data.tables import TableTransactionError
|
||||
|
||||
# Batch operations (same partition only!)
|
||||
operations = [
|
||||
("create", {"PartitionKey": "batch", "RowKey": "1", "data": "first"}),
|
||||
("create", {"PartitionKey": "batch", "RowKey": "2", "data": "second"}),
|
||||
("upsert", {"PartitionKey": "batch", "RowKey": "3", "data": "third"}),
|
||||
]
|
||||
|
||||
try:
|
||||
table_client.submit_transaction(operations)
|
||||
except TableTransactionError as e:
|
||||
print(f"Transaction failed: {e}")
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.data.tables.aio import TableServiceClient, TableClient
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def table_operations():
|
||||
credential = DefaultAzureCredential()
|
||||
|
||||
async with TableClient(
|
||||
endpoint="https://<account>.table.core.windows.net",
|
||||
table_name="mytable",
|
||||
credential=credential
|
||||
) as client:
|
||||
# Create
|
||||
await client.create_entity(entity={
|
||||
"PartitionKey": "async",
|
||||
"RowKey": "1",
|
||||
"data": "test"
|
||||
})
|
||||
|
||||
# Query
|
||||
async for entity in client.query_entities("PartitionKey eq 'async'"):
|
||||
print(entity)
|
||||
|
||||
import asyncio
|
||||
asyncio.run(table_operations())
|
||||
```
|
||||
|
||||
## Data Types
|
||||
|
||||
| Python Type | Table Storage Type |
|
||||
|-------------|-------------------|
|
||||
| `str` | String |
|
||||
| `int` | Int64 |
|
||||
| `float` | Double |
|
||||
| `bool` | Boolean |
|
||||
| `datetime` | DateTime |
|
||||
| `bytes` | Binary |
|
||||
| `UUID` | Guid |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Design partition keys** for query patterns and even distribution
|
||||
2. **Query within partitions** whenever possible (cross-partition is expensive)
|
||||
3. **Use batch operations** for multiple entities in same partition
|
||||
4. **Use `upsert_entity`** for idempotent writes
|
||||
5. **Use parameterized queries** to prevent injection
|
||||
6. **Keep entities small** — max 1MB per entity
|
||||
7. **Use async client** for high-throughput scenarios
|
||||
192
skills/official/microsoft/python/entra/azure-identity/SKILL.md
Normal file
192
skills/official/microsoft/python/entra/azure-identity/SKILL.md
Normal file
@@ -0,0 +1,192 @@
|
||||
---
|
||||
name: azure-identity-py
|
||||
description: |
|
||||
Azure Identity SDK for Python authentication. Use for DefaultAzureCredential, managed identity, service principals, and token caching.
|
||||
Triggers: "azure-identity", "DefaultAzureCredential", "authentication", "managed identity", "service principal", "credential".
|
||||
package: azure-identity
|
||||
---
|
||||
|
||||
# Azure Identity SDK for Python
|
||||
|
||||
Authentication library for Azure SDK clients using Microsoft Entra ID (formerly Azure AD).
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
# Service Principal (for production/CI)
|
||||
AZURE_TENANT_ID=<your-tenant-id>
|
||||
AZURE_CLIENT_ID=<your-client-id>
|
||||
AZURE_CLIENT_SECRET=<your-client-secret>
|
||||
|
||||
# User-assigned Managed Identity (optional)
|
||||
AZURE_CLIENT_ID=<managed-identity-client-id>
|
||||
```
|
||||
|
||||
## DefaultAzureCredential
|
||||
|
||||
The recommended credential for most scenarios. Tries multiple authentication methods in order:
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.storage.blob import BlobServiceClient
|
||||
|
||||
# Works in local dev AND production without code changes
|
||||
credential = DefaultAzureCredential()
|
||||
|
||||
client = BlobServiceClient(
|
||||
account_url="https://<account>.blob.core.windows.net",
|
||||
credential=credential
|
||||
)
|
||||
```
|
||||
|
||||
### Credential Chain Order
|
||||
|
||||
| Order | Credential | Environment |
|
||||
|-------|-----------|-------------|
|
||||
| 1 | EnvironmentCredential | CI/CD, containers |
|
||||
| 2 | WorkloadIdentityCredential | Kubernetes |
|
||||
| 3 | ManagedIdentityCredential | Azure VMs, App Service, Functions |
|
||||
| 4 | SharedTokenCacheCredential | Windows only |
|
||||
| 5 | VisualStudioCodeCredential | VS Code with Azure extension |
|
||||
| 6 | AzureCliCredential | `az login` |
|
||||
| 7 | AzurePowerShellCredential | `Connect-AzAccount` |
|
||||
| 8 | AzureDeveloperCliCredential | `azd auth login` |
|
||||
|
||||
### Customizing DefaultAzureCredential
|
||||
|
||||
```python
|
||||
# Exclude credentials you don't need
|
||||
credential = DefaultAzureCredential(
|
||||
exclude_environment_credential=True,
|
||||
exclude_shared_token_cache_credential=True,
|
||||
managed_identity_client_id="<user-assigned-mi-client-id>" # For user-assigned MI
|
||||
)
|
||||
|
||||
# Enable interactive browser (disabled by default)
|
||||
credential = DefaultAzureCredential(
|
||||
exclude_interactive_browser_credential=False
|
||||
)
|
||||
```
|
||||
|
||||
## Specific Credential Types
|
||||
|
||||
### ManagedIdentityCredential
|
||||
|
||||
For Azure-hosted resources (VMs, App Service, Functions, AKS):
|
||||
|
||||
```python
|
||||
from azure.identity import ManagedIdentityCredential
|
||||
|
||||
# System-assigned managed identity
|
||||
credential = ManagedIdentityCredential()
|
||||
|
||||
# User-assigned managed identity
|
||||
credential = ManagedIdentityCredential(
|
||||
client_id="<user-assigned-mi-client-id>"
|
||||
)
|
||||
```
|
||||
|
||||
### ClientSecretCredential
|
||||
|
||||
For service principal with secret:
|
||||
|
||||
```python
|
||||
from azure.identity import ClientSecretCredential
|
||||
|
||||
credential = ClientSecretCredential(
|
||||
tenant_id=os.environ["AZURE_TENANT_ID"],
|
||||
client_id=os.environ["AZURE_CLIENT_ID"],
|
||||
client_secret=os.environ["AZURE_CLIENT_SECRET"]
|
||||
)
|
||||
```
|
||||
|
||||
### AzureCliCredential
|
||||
|
||||
Uses the account from `az login`:
|
||||
|
||||
```python
|
||||
from azure.identity import AzureCliCredential
|
||||
|
||||
credential = AzureCliCredential()
|
||||
```
|
||||
|
||||
### ChainedTokenCredential
|
||||
|
||||
Custom credential chain:
|
||||
|
||||
```python
|
||||
from azure.identity import (
|
||||
ChainedTokenCredential,
|
||||
ManagedIdentityCredential,
|
||||
AzureCliCredential
|
||||
)
|
||||
|
||||
# Try managed identity first, fall back to CLI
|
||||
credential = ChainedTokenCredential(
|
||||
ManagedIdentityCredential(client_id="<user-assigned-mi-client-id>"),
|
||||
AzureCliCredential()
|
||||
)
|
||||
```
|
||||
|
||||
## Credential Types Table
|
||||
|
||||
| Credential | Use Case | Auth Method |
|
||||
|------------|----------|-------------|
|
||||
| `DefaultAzureCredential` | Most scenarios | Auto-detect |
|
||||
| `ManagedIdentityCredential` | Azure-hosted apps | Managed Identity |
|
||||
| `ClientSecretCredential` | Service principal | Client secret |
|
||||
| `ClientCertificateCredential` | Service principal | Certificate |
|
||||
| `AzureCliCredential` | Local development | Azure CLI |
|
||||
| `AzureDeveloperCliCredential` | Local development | Azure Developer CLI |
|
||||
| `InteractiveBrowserCredential` | User sign-in | Browser OAuth |
|
||||
| `DeviceCodeCredential` | Headless/SSH | Device code flow |
|
||||
|
||||
## Getting Tokens Directly
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
|
||||
# Get token for a specific scope
|
||||
token = credential.get_token("https://management.azure.com/.default")
|
||||
print(f"Token expires: {token.expires_on}")
|
||||
|
||||
# For Azure Database for PostgreSQL
|
||||
token = credential.get_token("https://ossrdbms-aad.database.windows.net/.default")
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
from azure.storage.blob.aio import BlobServiceClient
|
||||
|
||||
async def main():
|
||||
credential = DefaultAzureCredential()
|
||||
|
||||
async with BlobServiceClient(
|
||||
account_url="https://<account>.blob.core.windows.net",
|
||||
credential=credential
|
||||
) as client:
|
||||
# ... async operations
|
||||
pass
|
||||
|
||||
await credential.close()
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use DefaultAzureCredential** for code that runs locally and in Azure
|
||||
2. **Never hardcode credentials** — use environment variables or managed identity
|
||||
3. **Prefer managed identity** in production Azure deployments
|
||||
4. **Use ChainedTokenCredential** when you need a custom credential order
|
||||
5. **Close async credentials** explicitly or use context managers
|
||||
6. **Set AZURE_CLIENT_ID** for user-assigned managed identities
|
||||
7. **Exclude unused credentials** to speed up authentication
|
||||
247
skills/official/microsoft/python/entra/keyvault/SKILL.md
Normal file
247
skills/official/microsoft/python/entra/keyvault/SKILL.md
Normal file
@@ -0,0 +1,247 @@
|
||||
---
|
||||
name: azure-keyvault-py
|
||||
description: |
|
||||
Azure Key Vault SDK for Python. Use for secrets, keys, and certificates management with secure storage.
|
||||
Triggers: "key vault", "SecretClient", "KeyClient", "CertificateClient", "secrets", "encryption keys".
|
||||
package: azure-keyvault-secrets, azure-keyvault-keys, azure-keyvault-certificates
|
||||
---
|
||||
|
||||
# Azure Key Vault SDK for Python
|
||||
|
||||
Secure storage and management for secrets, cryptographic keys, and certificates.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
# Secrets
|
||||
pip install azure-keyvault-secrets azure-identity
|
||||
|
||||
# Keys (cryptographic operations)
|
||||
pip install azure-keyvault-keys azure-identity
|
||||
|
||||
# Certificates
|
||||
pip install azure-keyvault-certificates azure-identity
|
||||
|
||||
# All
|
||||
pip install azure-keyvault-secrets azure-keyvault-keys azure-keyvault-certificates azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_KEYVAULT_URL=https://<vault-name>.vault.azure.net/
|
||||
```
|
||||
|
||||
## Secrets
|
||||
|
||||
### SecretClient Setup
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.keyvault.secrets import SecretClient
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
vault_url = "https://<vault-name>.vault.azure.net/"
|
||||
|
||||
client = SecretClient(vault_url=vault_url, credential=credential)
|
||||
```
|
||||
|
||||
### Secret Operations
|
||||
|
||||
```python
|
||||
# Set secret
|
||||
secret = client.set_secret("database-password", "super-secret-value")
|
||||
print(f"Created: {secret.name}, version: {secret.properties.version}")
|
||||
|
||||
# Get secret
|
||||
secret = client.get_secret("database-password")
|
||||
print(f"Value: {secret.value}")
|
||||
|
||||
# Get specific version
|
||||
secret = client.get_secret("database-password", version="abc123")
|
||||
|
||||
# List secrets (names only, not values)
|
||||
for secret_properties in client.list_properties_of_secrets():
|
||||
print(f"Secret: {secret_properties.name}")
|
||||
|
||||
# List versions
|
||||
for version in client.list_properties_of_secret_versions("database-password"):
|
||||
print(f"Version: {version.version}, Created: {version.created_on}")
|
||||
|
||||
# Delete secret (soft delete)
|
||||
poller = client.begin_delete_secret("database-password")
|
||||
deleted_secret = poller.result()
|
||||
|
||||
# Purge (permanent delete, if soft-delete enabled)
|
||||
client.purge_deleted_secret("database-password")
|
||||
|
||||
# Recover deleted secret
|
||||
client.begin_recover_deleted_secret("database-password").result()
|
||||
```
|
||||
|
||||
## Keys
|
||||
|
||||
### KeyClient Setup
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.keyvault.keys import KeyClient
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
vault_url = "https://<vault-name>.vault.azure.net/"
|
||||
|
||||
client = KeyClient(vault_url=vault_url, credential=credential)
|
||||
```
|
||||
|
||||
### Key Operations
|
||||
|
||||
```python
|
||||
from azure.keyvault.keys import KeyType
|
||||
|
||||
# Create RSA key
|
||||
rsa_key = client.create_rsa_key("rsa-key", size=2048)
|
||||
|
||||
# Create EC key
|
||||
ec_key = client.create_ec_key("ec-key", curve="P-256")
|
||||
|
||||
# Get key
|
||||
key = client.get_key("rsa-key")
|
||||
print(f"Key type: {key.key_type}")
|
||||
|
||||
# List keys
|
||||
for key_properties in client.list_properties_of_keys():
|
||||
print(f"Key: {key_properties.name}")
|
||||
|
||||
# Delete key
|
||||
poller = client.begin_delete_key("rsa-key")
|
||||
deleted_key = poller.result()
|
||||
```
|
||||
|
||||
### Cryptographic Operations
|
||||
|
||||
```python
|
||||
from azure.keyvault.keys.crypto import CryptographyClient, EncryptionAlgorithm
|
||||
|
||||
# Get crypto client for a specific key
|
||||
crypto_client = CryptographyClient(key, credential=credential)
|
||||
# Or from key ID
|
||||
crypto_client = CryptographyClient(
|
||||
"https://<vault>.vault.azure.net/keys/<key-name>/<version>",
|
||||
credential=credential
|
||||
)
|
||||
|
||||
# Encrypt
|
||||
plaintext = b"Hello, Key Vault!"
|
||||
result = crypto_client.encrypt(EncryptionAlgorithm.rsa_oaep, plaintext)
|
||||
ciphertext = result.ciphertext
|
||||
|
||||
# Decrypt
|
||||
result = crypto_client.decrypt(EncryptionAlgorithm.rsa_oaep, ciphertext)
|
||||
decrypted = result.plaintext
|
||||
|
||||
# Sign
|
||||
from azure.keyvault.keys.crypto import SignatureAlgorithm
|
||||
import hashlib
|
||||
|
||||
digest = hashlib.sha256(b"data to sign").digest()
|
||||
result = crypto_client.sign(SignatureAlgorithm.rs256, digest)
|
||||
signature = result.signature
|
||||
|
||||
# Verify
|
||||
result = crypto_client.verify(SignatureAlgorithm.rs256, digest, signature)
|
||||
print(f"Valid: {result.is_valid}")
|
||||
```
|
||||
|
||||
## Certificates
|
||||
|
||||
### CertificateClient Setup
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.keyvault.certificates import CertificateClient, CertificatePolicy
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
vault_url = "https://<vault-name>.vault.azure.net/"
|
||||
|
||||
client = CertificateClient(vault_url=vault_url, credential=credential)
|
||||
```
|
||||
|
||||
### Certificate Operations
|
||||
|
||||
```python
|
||||
# Create self-signed certificate
|
||||
policy = CertificatePolicy.get_default()
|
||||
poller = client.begin_create_certificate("my-cert", policy=policy)
|
||||
certificate = poller.result()
|
||||
|
||||
# Get certificate
|
||||
certificate = client.get_certificate("my-cert")
|
||||
print(f"Thumbprint: {certificate.properties.x509_thumbprint.hex()}")
|
||||
|
||||
# Get certificate with private key (as secret)
|
||||
from azure.keyvault.secrets import SecretClient
|
||||
secret_client = SecretClient(vault_url=vault_url, credential=credential)
|
||||
cert_secret = secret_client.get_secret("my-cert")
|
||||
# cert_secret.value contains PEM or PKCS12
|
||||
|
||||
# List certificates
|
||||
for cert in client.list_properties_of_certificates():
|
||||
print(f"Certificate: {cert.name}")
|
||||
|
||||
# Delete certificate
|
||||
poller = client.begin_delete_certificate("my-cert")
|
||||
deleted = poller.result()
|
||||
```
|
||||
|
||||
## Client Types Table
|
||||
|
||||
| Client | Package | Purpose |
|
||||
|--------|---------|---------|
|
||||
| `SecretClient` | `azure-keyvault-secrets` | Store/retrieve secrets |
|
||||
| `KeyClient` | `azure-keyvault-keys` | Manage cryptographic keys |
|
||||
| `CryptographyClient` | `azure-keyvault-keys` | Encrypt/decrypt/sign/verify |
|
||||
| `CertificateClient` | `azure-keyvault-certificates` | Manage certificates |
|
||||
|
||||
## Async Clients
|
||||
|
||||
```python
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
from azure.keyvault.secrets.aio import SecretClient
|
||||
|
||||
async def get_secret():
|
||||
credential = DefaultAzureCredential()
|
||||
client = SecretClient(vault_url=vault_url, credential=credential)
|
||||
|
||||
async with client:
|
||||
secret = await client.get_secret("my-secret")
|
||||
print(secret.value)
|
||||
|
||||
import asyncio
|
||||
asyncio.run(get_secret())
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
```python
|
||||
from azure.core.exceptions import ResourceNotFoundError, HttpResponseError
|
||||
|
||||
try:
|
||||
secret = client.get_secret("nonexistent")
|
||||
except ResourceNotFoundError:
|
||||
print("Secret not found")
|
||||
except HttpResponseError as e:
|
||||
if e.status_code == 403:
|
||||
print("Access denied - check RBAC permissions")
|
||||
raise
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use DefaultAzureCredential** for authentication
|
||||
2. **Use managed identity** in Azure-hosted applications
|
||||
3. **Enable soft-delete** for recovery (enabled by default)
|
||||
4. **Use RBAC** over access policies for fine-grained control
|
||||
5. **Rotate secrets** regularly using versioning
|
||||
6. **Use Key Vault references** in App Service/Functions config
|
||||
7. **Cache secrets** appropriately to reduce API calls
|
||||
8. **Use async clients** for high-throughput scenarios
|
||||
@@ -0,0 +1,333 @@
|
||||
---
|
||||
name: agent-framework-azure-ai-py
|
||||
description: Build Azure AI Foundry agents using the Microsoft Agent Framework Python SDK (agent-framework-azure-ai). Use when creating persistent agents with AzureAIAgentsProvider, using hosted tools (code interpreter, file search, web search), integrating MCP servers, managing conversation threads, or implementing streaming responses. Covers function tools, structured outputs, and multi-tool agents.
|
||||
package: agent-framework-azure-ai
|
||||
---
|
||||
|
||||
# Agent Framework Azure Hosted Agents
|
||||
|
||||
Build persistent agents on Azure AI Foundry using the Microsoft Agent Framework Python SDK.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
User Query → AzureAIAgentsProvider → Azure AI Agent Service (Persistent)
|
||||
↓
|
||||
Agent.run() / Agent.run_stream()
|
||||
↓
|
||||
Tools: Functions | Hosted (Code/Search/Web) | MCP
|
||||
↓
|
||||
AgentThread (conversation persistence)
|
||||
```
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
# Full framework (recommended)
|
||||
pip install agent-framework --pre
|
||||
|
||||
# Or Azure-specific package only
|
||||
pip install agent-framework-azure-ai --pre
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
export AZURE_AI_PROJECT_ENDPOINT="https://<project>.services.ai.azure.com/api/projects/<project-id>"
|
||||
export AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4o-mini"
|
||||
export BING_CONNECTION_ID="your-bing-connection-id" # For web search
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.identity.aio import AzureCliCredential, DefaultAzureCredential
|
||||
|
||||
# Development
|
||||
credential = AzureCliCredential()
|
||||
|
||||
# Production
|
||||
credential = DefaultAzureCredential()
|
||||
```
|
||||
|
||||
## Core Workflow
|
||||
|
||||
### Basic Agent
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
from agent_framework.azure import AzureAIAgentsProvider
|
||||
from azure.identity.aio import AzureCliCredential
|
||||
|
||||
async def main():
|
||||
async with (
|
||||
AzureCliCredential() as credential,
|
||||
AzureAIAgentsProvider(credential=credential) as provider,
|
||||
):
|
||||
agent = await provider.create_agent(
|
||||
name="MyAgent",
|
||||
instructions="You are a helpful assistant.",
|
||||
)
|
||||
|
||||
result = await agent.run("Hello!")
|
||||
print(result.text)
|
||||
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
### Agent with Function Tools
|
||||
|
||||
```python
|
||||
from typing import Annotated
|
||||
from pydantic import Field
|
||||
from agent_framework.azure import AzureAIAgentsProvider
|
||||
from azure.identity.aio import AzureCliCredential
|
||||
|
||||
def get_weather(
|
||||
location: Annotated[str, Field(description="City name to get weather for")],
|
||||
) -> str:
|
||||
"""Get the current weather for a location."""
|
||||
return f"Weather in {location}: 72°F, sunny"
|
||||
|
||||
def get_current_time() -> str:
|
||||
"""Get the current UTC time."""
|
||||
from datetime import datetime, timezone
|
||||
return datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
|
||||
|
||||
async def main():
|
||||
async with (
|
||||
AzureCliCredential() as credential,
|
||||
AzureAIAgentsProvider(credential=credential) as provider,
|
||||
):
|
||||
agent = await provider.create_agent(
|
||||
name="WeatherAgent",
|
||||
instructions="You help with weather and time queries.",
|
||||
tools=[get_weather, get_current_time], # Pass functions directly
|
||||
)
|
||||
|
||||
result = await agent.run("What's the weather in Seattle?")
|
||||
print(result.text)
|
||||
```
|
||||
|
||||
### Agent with Hosted Tools
|
||||
|
||||
```python
|
||||
from agent_framework import (
|
||||
HostedCodeInterpreterTool,
|
||||
HostedFileSearchTool,
|
||||
HostedWebSearchTool,
|
||||
)
|
||||
from agent_framework.azure import AzureAIAgentsProvider
|
||||
from azure.identity.aio import AzureCliCredential
|
||||
|
||||
async def main():
|
||||
async with (
|
||||
AzureCliCredential() as credential,
|
||||
AzureAIAgentsProvider(credential=credential) as provider,
|
||||
):
|
||||
agent = await provider.create_agent(
|
||||
name="MultiToolAgent",
|
||||
instructions="You can execute code, search files, and search the web.",
|
||||
tools=[
|
||||
HostedCodeInterpreterTool(),
|
||||
HostedWebSearchTool(name="Bing"),
|
||||
],
|
||||
)
|
||||
|
||||
result = await agent.run("Calculate the factorial of 20 in Python")
|
||||
print(result.text)
|
||||
```
|
||||
|
||||
### Streaming Responses
|
||||
|
||||
```python
|
||||
async def main():
|
||||
async with (
|
||||
AzureCliCredential() as credential,
|
||||
AzureAIAgentsProvider(credential=credential) as provider,
|
||||
):
|
||||
agent = await provider.create_agent(
|
||||
name="StreamingAgent",
|
||||
instructions="You are a helpful assistant.",
|
||||
)
|
||||
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream("Tell me a short story"):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print()
|
||||
```
|
||||
|
||||
### Conversation Threads
|
||||
|
||||
```python
|
||||
from agent_framework.azure import AzureAIAgentsProvider
|
||||
from azure.identity.aio import AzureCliCredential
|
||||
|
||||
async def main():
|
||||
async with (
|
||||
AzureCliCredential() as credential,
|
||||
AzureAIAgentsProvider(credential=credential) as provider,
|
||||
):
|
||||
agent = await provider.create_agent(
|
||||
name="ChatAgent",
|
||||
instructions="You are a helpful assistant.",
|
||||
tools=[get_weather],
|
||||
)
|
||||
|
||||
# Create thread for conversation persistence
|
||||
thread = agent.get_new_thread()
|
||||
|
||||
# First turn
|
||||
result1 = await agent.run("What's the weather in Seattle?", thread=thread)
|
||||
print(f"Agent: {result1.text}")
|
||||
|
||||
# Second turn - context is maintained
|
||||
result2 = await agent.run("What about Portland?", thread=thread)
|
||||
print(f"Agent: {result2.text}")
|
||||
|
||||
# Save thread ID for later resumption
|
||||
print(f"Conversation ID: {thread.conversation_id}")
|
||||
```
|
||||
|
||||
### Structured Outputs
|
||||
|
||||
```python
|
||||
from pydantic import BaseModel, ConfigDict
|
||||
from agent_framework.azure import AzureAIAgentsProvider
|
||||
from azure.identity.aio import AzureCliCredential
|
||||
|
||||
class WeatherResponse(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
location: str
|
||||
temperature: float
|
||||
unit: str
|
||||
conditions: str
|
||||
|
||||
async def main():
|
||||
async with (
|
||||
AzureCliCredential() as credential,
|
||||
AzureAIAgentsProvider(credential=credential) as provider,
|
||||
):
|
||||
agent = await provider.create_agent(
|
||||
name="StructuredAgent",
|
||||
instructions="Provide weather information in structured format.",
|
||||
response_format=WeatherResponse,
|
||||
)
|
||||
|
||||
result = await agent.run("Weather in Seattle?")
|
||||
weather = WeatherResponse.model_validate_json(result.text)
|
||||
print(f"{weather.location}: {weather.temperature}°{weather.unit}")
|
||||
```
|
||||
|
||||
## Provider Methods
|
||||
|
||||
| Method | Description |
|
||||
|--------|-------------|
|
||||
| `create_agent()` | Create new agent on Azure AI service |
|
||||
| `get_agent(agent_id)` | Retrieve existing agent by ID |
|
||||
| `as_agent(sdk_agent)` | Wrap SDK Agent object (no HTTP call) |
|
||||
|
||||
## Hosted Tools Quick Reference
|
||||
|
||||
| Tool | Import | Purpose |
|
||||
|------|--------|---------|
|
||||
| `HostedCodeInterpreterTool` | `from agent_framework import HostedCodeInterpreterTool` | Execute Python code |
|
||||
| `HostedFileSearchTool` | `from agent_framework import HostedFileSearchTool` | Search vector stores |
|
||||
| `HostedWebSearchTool` | `from agent_framework import HostedWebSearchTool` | Bing web search |
|
||||
| `HostedMCPTool` | `from agent_framework import HostedMCPTool` | Service-managed MCP |
|
||||
| `MCPStreamableHTTPTool` | `from agent_framework import MCPStreamableHTTPTool` | Client-managed MCP |
|
||||
|
||||
## Complete Example
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
from typing import Annotated
|
||||
from pydantic import BaseModel, Field
|
||||
from agent_framework import (
|
||||
HostedCodeInterpreterTool,
|
||||
HostedWebSearchTool,
|
||||
MCPStreamableHTTPTool,
|
||||
)
|
||||
from agent_framework.azure import AzureAIAgentsProvider
|
||||
from azure.identity.aio import AzureCliCredential
|
||||
|
||||
|
||||
def get_weather(
|
||||
location: Annotated[str, Field(description="City name")],
|
||||
) -> str:
|
||||
"""Get weather for a location."""
|
||||
return f"Weather in {location}: 72°F, sunny"
|
||||
|
||||
|
||||
class AnalysisResult(BaseModel):
|
||||
summary: str
|
||||
key_findings: list[str]
|
||||
confidence: float
|
||||
|
||||
|
||||
async def main():
|
||||
async with (
|
||||
AzureCliCredential() as credential,
|
||||
MCPStreamableHTTPTool(
|
||||
name="Docs MCP",
|
||||
url="https://learn.microsoft.com/api/mcp",
|
||||
) as mcp_tool,
|
||||
AzureAIAgentsProvider(credential=credential) as provider,
|
||||
):
|
||||
agent = await provider.create_agent(
|
||||
name="ResearchAssistant",
|
||||
instructions="You are a research assistant with multiple capabilities.",
|
||||
tools=[
|
||||
get_weather,
|
||||
HostedCodeInterpreterTool(),
|
||||
HostedWebSearchTool(name="Bing"),
|
||||
mcp_tool,
|
||||
],
|
||||
)
|
||||
|
||||
thread = agent.get_new_thread()
|
||||
|
||||
# Non-streaming
|
||||
result = await agent.run(
|
||||
"Search for Python best practices and summarize",
|
||||
thread=thread,
|
||||
)
|
||||
print(f"Response: {result.text}")
|
||||
|
||||
# Streaming
|
||||
print("\nStreaming: ", end="")
|
||||
async for chunk in agent.run_stream("Continue with examples", thread=thread):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print()
|
||||
|
||||
# Structured output
|
||||
result = await agent.run(
|
||||
"Analyze findings",
|
||||
thread=thread,
|
||||
response_format=AnalysisResult,
|
||||
)
|
||||
analysis = AnalysisResult.model_validate_json(result.text)
|
||||
print(f"\nConfidence: {analysis.confidence}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
||||
- Always use async context managers: `async with provider:`
|
||||
- Pass functions directly to `tools=` parameter (auto-converted to AIFunction)
|
||||
- Use `Annotated[type, Field(description=...)]` for function parameters
|
||||
- Use `get_new_thread()` for multi-turn conversations
|
||||
- Prefer `HostedMCPTool` for service-managed MCP, `MCPStreamableHTTPTool` for client-managed
|
||||
|
||||
## Reference Files
|
||||
|
||||
- [references/tools.md](references/tools.md): Detailed hosted tool patterns
|
||||
- [references/mcp.md](references/mcp.md): MCP integration (hosted + local)
|
||||
- [references/threads.md](references/threads.md): Thread and conversation management
|
||||
- [references/advanced.md](references/advanced.md): OpenAPI, citations, structured outputs
|
||||
325
skills/official/microsoft/python/foundry/agents-v2/SKILL.md
Normal file
325
skills/official/microsoft/python/foundry/agents-v2/SKILL.md
Normal file
@@ -0,0 +1,325 @@
|
||||
---
|
||||
name: agents-v2-py
|
||||
description: |
|
||||
Build container-based Foundry Agents using Azure AI Projects SDK with ImageBasedHostedAgentDefinition.
|
||||
Use when creating hosted agents that run custom code in Azure AI Foundry with your own container images.
|
||||
Triggers: "ImageBasedHostedAgentDefinition", "hosted agent", "container agent", "Foundry Agent",
|
||||
"create_version", "ProtocolVersionRecord", "AgentProtocol.RESPONSES", "custom agent image".
|
||||
package: azure-ai-projects
|
||||
---
|
||||
|
||||
# Azure AI Hosted Agents (Python)
|
||||
|
||||
Build container-based hosted agents using `ImageBasedHostedAgentDefinition` from the Azure AI Projects SDK.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-ai-projects>=2.0.0b3 azure-identity
|
||||
```
|
||||
|
||||
**Minimum SDK Version:** `2.0.0b3` or later required for hosted agent support.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_AI_PROJECT_ENDPOINT=https://<resource>.services.ai.azure.com/api/projects/<project>
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before creating hosted agents:
|
||||
|
||||
1. **Container Image** - Build and push to Azure Container Registry (ACR)
|
||||
2. **ACR Pull Permissions** - Grant your project's managed identity `AcrPull` role on the ACR
|
||||
3. **Capability Host** - Account-level capability host with `enablePublicHostingEnvironment=true`
|
||||
4. **SDK Version** - Ensure `azure-ai-projects>=2.0.0b3`
|
||||
|
||||
## Authentication
|
||||
|
||||
Always use `DefaultAzureCredential`:
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.ai.projects import AIProjectClient
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
client = AIProjectClient(
|
||||
endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"],
|
||||
credential=credential
|
||||
)
|
||||
```
|
||||
|
||||
## Core Workflow
|
||||
|
||||
### 1. Imports
|
||||
|
||||
```python
|
||||
import os
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.ai.projects import AIProjectClient
|
||||
from azure.ai.projects.models import (
|
||||
ImageBasedHostedAgentDefinition,
|
||||
ProtocolVersionRecord,
|
||||
AgentProtocol,
|
||||
)
|
||||
```
|
||||
|
||||
### 2. Create Hosted Agent
|
||||
|
||||
```python
|
||||
client = AIProjectClient(
|
||||
endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"],
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
|
||||
agent = client.agents.create_version(
|
||||
agent_name="my-hosted-agent",
|
||||
definition=ImageBasedHostedAgentDefinition(
|
||||
container_protocol_versions=[
|
||||
ProtocolVersionRecord(protocol=AgentProtocol.RESPONSES, version="v1")
|
||||
],
|
||||
cpu="1",
|
||||
memory="2Gi",
|
||||
image="myregistry.azurecr.io/my-agent:latest",
|
||||
tools=[{"type": "code_interpreter"}],
|
||||
environment_variables={
|
||||
"AZURE_AI_PROJECT_ENDPOINT": os.environ["AZURE_AI_PROJECT_ENDPOINT"],
|
||||
"MODEL_NAME": "gpt-4o-mini"
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
print(f"Created agent: {agent.name} (version: {agent.version})")
|
||||
```
|
||||
|
||||
### 3. List Agent Versions
|
||||
|
||||
```python
|
||||
versions = client.agents.list_versions(agent_name="my-hosted-agent")
|
||||
for version in versions:
|
||||
print(f"Version: {version.version}, State: {version.state}")
|
||||
```
|
||||
|
||||
### 4. Delete Agent Version
|
||||
|
||||
```python
|
||||
client.agents.delete_version(
|
||||
agent_name="my-hosted-agent",
|
||||
version=agent.version
|
||||
)
|
||||
```
|
||||
|
||||
## ImageBasedHostedAgentDefinition Parameters
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
|-----------|------|----------|-------------|
|
||||
| `container_protocol_versions` | `list[ProtocolVersionRecord]` | Yes | Protocol versions the agent supports |
|
||||
| `image` | `str` | Yes | Full container image path (registry/image:tag) |
|
||||
| `cpu` | `str` | No | CPU allocation (e.g., "1", "2") |
|
||||
| `memory` | `str` | No | Memory allocation (e.g., "2Gi", "4Gi") |
|
||||
| `tools` | `list[dict]` | No | Tools available to the agent |
|
||||
| `environment_variables` | `dict[str, str]` | No | Environment variables for the container |
|
||||
|
||||
## Protocol Versions
|
||||
|
||||
The `container_protocol_versions` parameter specifies which protocols your agent supports:
|
||||
|
||||
```python
|
||||
from azure.ai.projects.models import ProtocolVersionRecord, AgentProtocol
|
||||
|
||||
# RESPONSES protocol - standard agent responses
|
||||
container_protocol_versions=[
|
||||
ProtocolVersionRecord(protocol=AgentProtocol.RESPONSES, version="v1")
|
||||
]
|
||||
```
|
||||
|
||||
**Available Protocols:**
|
||||
| Protocol | Description |
|
||||
|----------|-------------|
|
||||
| `AgentProtocol.RESPONSES` | Standard response protocol for agent interactions |
|
||||
|
||||
## Resource Allocation
|
||||
|
||||
Specify CPU and memory for your container:
|
||||
|
||||
```python
|
||||
definition=ImageBasedHostedAgentDefinition(
|
||||
container_protocol_versions=[...],
|
||||
image="myregistry.azurecr.io/my-agent:latest",
|
||||
cpu="2", # 2 CPU cores
|
||||
memory="4Gi" # 4 GiB memory
|
||||
)
|
||||
```
|
||||
|
||||
**Resource Limits:**
|
||||
| Resource | Min | Max | Default |
|
||||
|----------|-----|-----|---------|
|
||||
| CPU | 0.5 | 4 | 1 |
|
||||
| Memory | 1Gi | 8Gi | 2Gi |
|
||||
|
||||
## Tools Configuration
|
||||
|
||||
Add tools to your hosted agent:
|
||||
|
||||
### Code Interpreter
|
||||
|
||||
```python
|
||||
tools=[{"type": "code_interpreter"}]
|
||||
```
|
||||
|
||||
### MCP Tools
|
||||
|
||||
```python
|
||||
tools=[
|
||||
{"type": "code_interpreter"},
|
||||
{
|
||||
"type": "mcp",
|
||||
"server_label": "my-mcp-server",
|
||||
"server_url": "https://my-mcp-server.example.com"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
### Multiple Tools
|
||||
|
||||
```python
|
||||
tools=[
|
||||
{"type": "code_interpreter"},
|
||||
{"type": "file_search"},
|
||||
{
|
||||
"type": "mcp",
|
||||
"server_label": "custom-tool",
|
||||
"server_url": "https://custom-tool.example.com"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Pass configuration to your container:
|
||||
|
||||
```python
|
||||
environment_variables={
|
||||
"AZURE_AI_PROJECT_ENDPOINT": os.environ["AZURE_AI_PROJECT_ENDPOINT"],
|
||||
"MODEL_NAME": "gpt-4o-mini",
|
||||
"LOG_LEVEL": "INFO",
|
||||
"CUSTOM_CONFIG": "value"
|
||||
}
|
||||
```
|
||||
|
||||
**Best Practice:** Never hardcode secrets. Use environment variables or Azure Key Vault.
|
||||
|
||||
## Complete Example
|
||||
|
||||
```python
|
||||
import os
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.ai.projects import AIProjectClient
|
||||
from azure.ai.projects.models import (
|
||||
ImageBasedHostedAgentDefinition,
|
||||
ProtocolVersionRecord,
|
||||
AgentProtocol,
|
||||
)
|
||||
|
||||
def create_hosted_agent():
|
||||
"""Create a hosted agent with custom container image."""
|
||||
|
||||
client = AIProjectClient(
|
||||
endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"],
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
|
||||
agent = client.agents.create_version(
|
||||
agent_name="data-processor-agent",
|
||||
definition=ImageBasedHostedAgentDefinition(
|
||||
container_protocol_versions=[
|
||||
ProtocolVersionRecord(
|
||||
protocol=AgentProtocol.RESPONSES,
|
||||
version="v1"
|
||||
)
|
||||
],
|
||||
image="myregistry.azurecr.io/data-processor:v1.0",
|
||||
cpu="2",
|
||||
memory="4Gi",
|
||||
tools=[
|
||||
{"type": "code_interpreter"},
|
||||
{"type": "file_search"}
|
||||
],
|
||||
environment_variables={
|
||||
"AZURE_AI_PROJECT_ENDPOINT": os.environ["AZURE_AI_PROJECT_ENDPOINT"],
|
||||
"MODEL_NAME": "gpt-4o-mini",
|
||||
"MAX_RETRIES": "3"
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
print(f"Created hosted agent: {agent.name}")
|
||||
print(f"Version: {agent.version}")
|
||||
print(f"State: {agent.state}")
|
||||
|
||||
return agent
|
||||
|
||||
if __name__ == "__main__":
|
||||
create_hosted_agent()
|
||||
```
|
||||
|
||||
## Async Pattern
|
||||
|
||||
```python
|
||||
import os
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
from azure.ai.projects.aio import AIProjectClient
|
||||
from azure.ai.projects.models import (
|
||||
ImageBasedHostedAgentDefinition,
|
||||
ProtocolVersionRecord,
|
||||
AgentProtocol,
|
||||
)
|
||||
|
||||
async def create_hosted_agent_async():
|
||||
"""Create a hosted agent asynchronously."""
|
||||
|
||||
async with DefaultAzureCredential() as credential:
|
||||
async with AIProjectClient(
|
||||
endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"],
|
||||
credential=credential
|
||||
) as client:
|
||||
agent = await client.agents.create_version(
|
||||
agent_name="async-agent",
|
||||
definition=ImageBasedHostedAgentDefinition(
|
||||
container_protocol_versions=[
|
||||
ProtocolVersionRecord(
|
||||
protocol=AgentProtocol.RESPONSES,
|
||||
version="v1"
|
||||
)
|
||||
],
|
||||
image="myregistry.azurecr.io/async-agent:latest",
|
||||
cpu="1",
|
||||
memory="2Gi"
|
||||
)
|
||||
)
|
||||
return agent
|
||||
```
|
||||
|
||||
## Common Errors
|
||||
|
||||
| Error | Cause | Solution |
|
||||
|-------|-------|----------|
|
||||
| `ImagePullBackOff` | ACR pull permission denied | Grant `AcrPull` role to project's managed identity |
|
||||
| `InvalidContainerImage` | Image not found | Verify image path and tag exist in ACR |
|
||||
| `CapabilityHostNotFound` | No capability host configured | Create account-level capability host |
|
||||
| `ProtocolVersionNotSupported` | Invalid protocol version | Use `AgentProtocol.RESPONSES` with version `"v1"` |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Version Your Images** - Use specific tags, not `latest` in production
|
||||
2. **Minimal Resources** - Start with minimum CPU/memory, scale up as needed
|
||||
3. **Environment Variables** - Use for all configuration, never hardcode
|
||||
4. **Error Handling** - Wrap agent creation in try/except blocks
|
||||
5. **Cleanup** - Delete unused agent versions to free resources
|
||||
|
||||
## Reference Links
|
||||
|
||||
- [Azure AI Projects SDK](https://pypi.org/project/azure-ai-projects/)
|
||||
- [Hosted Agents Documentation](https://learn.microsoft.com/azure/ai-services/agents/how-to/hosted-agents)
|
||||
- [Azure Container Registry](https://learn.microsoft.com/azure/container-registry/)
|
||||
214
skills/official/microsoft/python/foundry/contentsafety/SKILL.md
Normal file
214
skills/official/microsoft/python/foundry/contentsafety/SKILL.md
Normal file
@@ -0,0 +1,214 @@
|
||||
---
|
||||
name: azure-ai-contentsafety-py
|
||||
description: |
|
||||
Azure AI Content Safety SDK for Python. Use for detecting harmful content in text and images with multi-severity classification.
|
||||
Triggers: "azure-ai-contentsafety", "ContentSafetyClient", "content moderation", "harmful content", "text analysis", "image analysis".
|
||||
package: azure-ai-contentsafety
|
||||
---
|
||||
|
||||
# Azure AI Content Safety SDK for Python
|
||||
|
||||
Detect harmful user-generated and AI-generated content in applications.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-ai-contentsafety
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
CONTENT_SAFETY_ENDPOINT=https://<resource>.cognitiveservices.azure.com
|
||||
CONTENT_SAFETY_KEY=<your-api-key>
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
### API Key
|
||||
|
||||
```python
|
||||
from azure.ai.contentsafety import ContentSafetyClient
|
||||
from azure.core.credentials import AzureKeyCredential
|
||||
import os
|
||||
|
||||
client = ContentSafetyClient(
|
||||
endpoint=os.environ["CONTENT_SAFETY_ENDPOINT"],
|
||||
credential=AzureKeyCredential(os.environ["CONTENT_SAFETY_KEY"])
|
||||
)
|
||||
```
|
||||
|
||||
### Entra ID
|
||||
|
||||
```python
|
||||
from azure.ai.contentsafety import ContentSafetyClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
client = ContentSafetyClient(
|
||||
endpoint=os.environ["CONTENT_SAFETY_ENDPOINT"],
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
```
|
||||
|
||||
## Analyze Text
|
||||
|
||||
```python
|
||||
from azure.ai.contentsafety import ContentSafetyClient
|
||||
from azure.ai.contentsafety.models import AnalyzeTextOptions, TextCategory
|
||||
from azure.core.credentials import AzureKeyCredential
|
||||
|
||||
client = ContentSafetyClient(endpoint, AzureKeyCredential(key))
|
||||
|
||||
request = AnalyzeTextOptions(text="Your text content to analyze")
|
||||
response = client.analyze_text(request)
|
||||
|
||||
# Check each category
|
||||
for category in [TextCategory.HATE, TextCategory.SELF_HARM,
|
||||
TextCategory.SEXUAL, TextCategory.VIOLENCE]:
|
||||
result = next((r for r in response.categories_analysis
|
||||
if r.category == category), None)
|
||||
if result:
|
||||
print(f"{category}: severity {result.severity}")
|
||||
```
|
||||
|
||||
## Analyze Image
|
||||
|
||||
```python
|
||||
from azure.ai.contentsafety import ContentSafetyClient
|
||||
from azure.ai.contentsafety.models import AnalyzeImageOptions, ImageData
|
||||
from azure.core.credentials import AzureKeyCredential
|
||||
import base64
|
||||
|
||||
client = ContentSafetyClient(endpoint, AzureKeyCredential(key))
|
||||
|
||||
# From file
|
||||
with open("image.jpg", "rb") as f:
|
||||
image_data = base64.b64encode(f.read()).decode("utf-8")
|
||||
|
||||
request = AnalyzeImageOptions(
|
||||
image=ImageData(content=image_data)
|
||||
)
|
||||
|
||||
response = client.analyze_image(request)
|
||||
|
||||
for result in response.categories_analysis:
|
||||
print(f"{result.category}: severity {result.severity}")
|
||||
```
|
||||
|
||||
### Image from URL
|
||||
|
||||
```python
|
||||
from azure.ai.contentsafety.models import AnalyzeImageOptions, ImageData
|
||||
|
||||
request = AnalyzeImageOptions(
|
||||
image=ImageData(blob_url="https://example.com/image.jpg")
|
||||
)
|
||||
|
||||
response = client.analyze_image(request)
|
||||
```
|
||||
|
||||
## Text Blocklist Management
|
||||
|
||||
### Create Blocklist
|
||||
|
||||
```python
|
||||
from azure.ai.contentsafety import BlocklistClient
|
||||
from azure.ai.contentsafety.models import TextBlocklist
|
||||
from azure.core.credentials import AzureKeyCredential
|
||||
|
||||
blocklist_client = BlocklistClient(endpoint, AzureKeyCredential(key))
|
||||
|
||||
blocklist = TextBlocklist(
|
||||
blocklist_name="my-blocklist",
|
||||
description="Custom terms to block"
|
||||
)
|
||||
|
||||
result = blocklist_client.create_or_update_text_blocklist(
|
||||
blocklist_name="my-blocklist",
|
||||
options=blocklist
|
||||
)
|
||||
```
|
||||
|
||||
### Add Block Items
|
||||
|
||||
```python
|
||||
from azure.ai.contentsafety.models import AddOrUpdateTextBlocklistItemsOptions, TextBlocklistItem
|
||||
|
||||
items = AddOrUpdateTextBlocklistItemsOptions(
|
||||
blocklist_items=[
|
||||
TextBlocklistItem(text="blocked-term-1"),
|
||||
TextBlocklistItem(text="blocked-term-2")
|
||||
]
|
||||
)
|
||||
|
||||
result = blocklist_client.add_or_update_blocklist_items(
|
||||
blocklist_name="my-blocklist",
|
||||
options=items
|
||||
)
|
||||
```
|
||||
|
||||
### Analyze with Blocklist
|
||||
|
||||
```python
|
||||
from azure.ai.contentsafety.models import AnalyzeTextOptions
|
||||
|
||||
request = AnalyzeTextOptions(
|
||||
text="Text containing blocked-term-1",
|
||||
blocklist_names=["my-blocklist"],
|
||||
halt_on_blocklist_hit=True
|
||||
)
|
||||
|
||||
response = client.analyze_text(request)
|
||||
|
||||
if response.blocklists_match:
|
||||
for match in response.blocklists_match:
|
||||
print(f"Blocked: {match.blocklist_item_text}")
|
||||
```
|
||||
|
||||
## Severity Levels
|
||||
|
||||
Text analysis returns 4 severity levels (0, 2, 4, 6) by default. For 8 levels (0-7):
|
||||
|
||||
```python
|
||||
from azure.ai.contentsafety.models import AnalyzeTextOptions, AnalyzeTextOutputType
|
||||
|
||||
request = AnalyzeTextOptions(
|
||||
text="Your text",
|
||||
output_type=AnalyzeTextOutputType.EIGHT_SEVERITY_LEVELS
|
||||
)
|
||||
```
|
||||
|
||||
## Harm Categories
|
||||
|
||||
| Category | Description |
|
||||
|----------|-------------|
|
||||
| `Hate` | Attacks based on identity (race, religion, gender, etc.) |
|
||||
| `Sexual` | Sexual content, relationships, anatomy |
|
||||
| `Violence` | Physical harm, weapons, injury |
|
||||
| `SelfHarm` | Self-injury, suicide, eating disorders |
|
||||
|
||||
## Severity Scale
|
||||
|
||||
| Level | Text Range | Image Range | Meaning |
|
||||
|-------|------------|-------------|---------|
|
||||
| 0 | Safe | Safe | No harmful content |
|
||||
| 2 | Low | Low | Mild references |
|
||||
| 4 | Medium | Medium | Moderate content |
|
||||
| 6 | High | High | Severe content |
|
||||
|
||||
## Client Types
|
||||
|
||||
| Client | Purpose |
|
||||
|--------|---------|
|
||||
| `ContentSafetyClient` | Analyze text and images |
|
||||
| `BlocklistClient` | Manage custom blocklists |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use blocklists** for domain-specific terms
|
||||
2. **Set severity thresholds** appropriate for your use case
|
||||
3. **Handle multiple categories** — content can be harmful in multiple ways
|
||||
4. **Use halt_on_blocklist_hit** for immediate rejection
|
||||
5. **Log analysis results** for audit and improvement
|
||||
6. **Consider 8-severity mode** for finer-grained control
|
||||
7. **Pre-moderate AI outputs** before showing to users
|
||||
@@ -0,0 +1,273 @@
|
||||
---
|
||||
name: azure-ai-contentunderstanding-py
|
||||
description: |
|
||||
Azure AI Content Understanding SDK for Python. Use for multimodal content extraction from documents, images, audio, and video.
|
||||
Triggers: "azure-ai-contentunderstanding", "ContentUnderstandingClient", "multimodal analysis", "document extraction", "video analysis", "audio transcription".
|
||||
package: azure-ai-contentunderstanding
|
||||
---
|
||||
|
||||
# Azure AI Content Understanding SDK for Python
|
||||
|
||||
Multimodal AI service that extracts semantic content from documents, video, audio, and image files for RAG and automated workflows.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-ai-contentunderstanding
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
CONTENTUNDERSTANDING_ENDPOINT=https://<resource>.cognitiveservices.azure.com/
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
import os
|
||||
from azure.ai.contentunderstanding import ContentUnderstandingClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
endpoint = os.environ["CONTENTUNDERSTANDING_ENDPOINT"]
|
||||
credential = DefaultAzureCredential()
|
||||
client = ContentUnderstandingClient(endpoint=endpoint, credential=credential)
|
||||
```
|
||||
|
||||
## Core Workflow
|
||||
|
||||
Content Understanding operations are asynchronous long-running operations:
|
||||
|
||||
1. **Begin Analysis** — Start the analysis operation with `begin_analyze()` (returns a poller)
|
||||
2. **Poll for Results** — Poll until analysis completes (SDK handles this with `.result()`)
|
||||
3. **Process Results** — Extract structured results from `AnalyzeResult.contents`
|
||||
|
||||
## Prebuilt Analyzers
|
||||
|
||||
| Analyzer | Content Type | Purpose |
|
||||
|----------|--------------|---------|
|
||||
| `prebuilt-documentSearch` | Documents | Extract markdown for RAG applications |
|
||||
| `prebuilt-imageSearch` | Images | Extract content from images |
|
||||
| `prebuilt-audioSearch` | Audio | Transcribe audio with timing |
|
||||
| `prebuilt-videoSearch` | Video | Extract frames, transcripts, summaries |
|
||||
| `prebuilt-invoice` | Documents | Extract invoice fields |
|
||||
|
||||
## Analyze Document
|
||||
|
||||
```python
|
||||
import os
|
||||
from azure.ai.contentunderstanding import ContentUnderstandingClient
|
||||
from azure.ai.contentunderstanding.models import AnalyzeInput
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
endpoint = os.environ["CONTENTUNDERSTANDING_ENDPOINT"]
|
||||
client = ContentUnderstandingClient(
|
||||
endpoint=endpoint,
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
|
||||
# Analyze document from URL
|
||||
poller = client.begin_analyze(
|
||||
analyzer_id="prebuilt-documentSearch",
|
||||
inputs=[AnalyzeInput(url="https://example.com/document.pdf")]
|
||||
)
|
||||
|
||||
result = poller.result()
|
||||
|
||||
# Access markdown content (contents is a list)
|
||||
content = result.contents[0]
|
||||
print(content.markdown)
|
||||
```
|
||||
|
||||
## Access Document Content Details
|
||||
|
||||
```python
|
||||
from azure.ai.contentunderstanding.models import MediaContentKind, DocumentContent
|
||||
|
||||
content = result.contents[0]
|
||||
if content.kind == MediaContentKind.DOCUMENT:
|
||||
document_content: DocumentContent = content # type: ignore
|
||||
print(document_content.start_page_number)
|
||||
```
|
||||
|
||||
## Analyze Image
|
||||
|
||||
```python
|
||||
from azure.ai.contentunderstanding.models import AnalyzeInput
|
||||
|
||||
poller = client.begin_analyze(
|
||||
analyzer_id="prebuilt-imageSearch",
|
||||
inputs=[AnalyzeInput(url="https://example.com/image.jpg")]
|
||||
)
|
||||
result = poller.result()
|
||||
content = result.contents[0]
|
||||
print(content.markdown)
|
||||
```
|
||||
|
||||
## Analyze Video
|
||||
|
||||
```python
|
||||
from azure.ai.contentunderstanding.models import AnalyzeInput
|
||||
|
||||
poller = client.begin_analyze(
|
||||
analyzer_id="prebuilt-videoSearch",
|
||||
inputs=[AnalyzeInput(url="https://example.com/video.mp4")]
|
||||
)
|
||||
|
||||
result = poller.result()
|
||||
|
||||
# Access video content (AudioVisualContent)
|
||||
content = result.contents[0]
|
||||
|
||||
# Get transcript phrases with timing
|
||||
for phrase in content.transcript_phrases:
|
||||
print(f"[{phrase.start_time} - {phrase.end_time}]: {phrase.text}")
|
||||
|
||||
# Get key frames (for video)
|
||||
for frame in content.key_frames:
|
||||
print(f"Frame at {frame.time}: {frame.description}")
|
||||
```
|
||||
|
||||
## Analyze Audio
|
||||
|
||||
```python
|
||||
from azure.ai.contentunderstanding.models import AnalyzeInput
|
||||
|
||||
poller = client.begin_analyze(
|
||||
analyzer_id="prebuilt-audioSearch",
|
||||
inputs=[AnalyzeInput(url="https://example.com/audio.mp3")]
|
||||
)
|
||||
|
||||
result = poller.result()
|
||||
|
||||
# Access audio transcript
|
||||
content = result.contents[0]
|
||||
for phrase in content.transcript_phrases:
|
||||
print(f"[{phrase.start_time}] {phrase.text}")
|
||||
```
|
||||
|
||||
## Custom Analyzers
|
||||
|
||||
Create custom analyzers with field schemas for specialized extraction:
|
||||
|
||||
```python
|
||||
# Create custom analyzer
|
||||
analyzer = client.create_analyzer(
|
||||
analyzer_id="my-invoice-analyzer",
|
||||
analyzer={
|
||||
"description": "Custom invoice analyzer",
|
||||
"base_analyzer_id": "prebuilt-documentSearch",
|
||||
"field_schema": {
|
||||
"fields": {
|
||||
"vendor_name": {"type": "string"},
|
||||
"invoice_total": {"type": "number"},
|
||||
"line_items": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"description": {"type": "string"},
|
||||
"amount": {"type": "number"}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
)
|
||||
|
||||
# Use custom analyzer
|
||||
from azure.ai.contentunderstanding.models import AnalyzeInput
|
||||
|
||||
poller = client.begin_analyze(
|
||||
analyzer_id="my-invoice-analyzer",
|
||||
inputs=[AnalyzeInput(url="https://example.com/invoice.pdf")]
|
||||
)
|
||||
|
||||
result = poller.result()
|
||||
|
||||
# Access extracted fields
|
||||
print(result.fields["vendor_name"])
|
||||
print(result.fields["invoice_total"])
|
||||
```
|
||||
|
||||
## Analyzer Management
|
||||
|
||||
```python
|
||||
# List all analyzers
|
||||
analyzers = client.list_analyzers()
|
||||
for analyzer in analyzers:
|
||||
print(f"{analyzer.analyzer_id}: {analyzer.description}")
|
||||
|
||||
# Get specific analyzer
|
||||
analyzer = client.get_analyzer("prebuilt-documentSearch")
|
||||
|
||||
# Delete custom analyzer
|
||||
client.delete_analyzer("my-custom-analyzer")
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
import os
|
||||
from azure.ai.contentunderstanding.aio import ContentUnderstandingClient
|
||||
from azure.ai.contentunderstanding.models import AnalyzeInput
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def analyze_document():
|
||||
endpoint = os.environ["CONTENTUNDERSTANDING_ENDPOINT"]
|
||||
credential = DefaultAzureCredential()
|
||||
|
||||
async with ContentUnderstandingClient(
|
||||
endpoint=endpoint,
|
||||
credential=credential
|
||||
) as client:
|
||||
poller = await client.begin_analyze(
|
||||
analyzer_id="prebuilt-documentSearch",
|
||||
inputs=[AnalyzeInput(url="https://example.com/doc.pdf")]
|
||||
)
|
||||
result = await poller.result()
|
||||
content = result.contents[0]
|
||||
return content.markdown
|
||||
|
||||
asyncio.run(analyze_document())
|
||||
```
|
||||
|
||||
## Content Types
|
||||
|
||||
| Class | For | Provides |
|
||||
|-------|-----|----------|
|
||||
| `DocumentContent` | PDF, images, Office docs | Pages, tables, figures, paragraphs |
|
||||
| `AudioVisualContent` | Audio, video files | Transcript phrases, timing, key frames |
|
||||
|
||||
Both derive from `MediaContent` which provides basic info and markdown representation.
|
||||
|
||||
## Model Imports
|
||||
|
||||
```python
|
||||
from azure.ai.contentunderstanding.models import (
|
||||
AnalyzeInput,
|
||||
AnalyzeResult,
|
||||
MediaContentKind,
|
||||
DocumentContent,
|
||||
AudioVisualContent,
|
||||
)
|
||||
```
|
||||
|
||||
## Client Types
|
||||
|
||||
| Client | Purpose |
|
||||
|--------|---------|
|
||||
| `ContentUnderstandingClient` | Sync client for all operations |
|
||||
| `ContentUnderstandingClient` (aio) | Async client for all operations |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use `begin_analyze` with `AnalyzeInput`** — this is the correct method signature
|
||||
2. **Access results via `result.contents[0]`** — results are returned as a list
|
||||
3. **Use prebuilt analyzers** for common scenarios (document/image/audio/video search)
|
||||
4. **Create custom analyzers** only for domain-specific field extraction
|
||||
5. **Use async client** for high-throughput scenarios with `azure.identity.aio` credentials
|
||||
6. **Handle long-running operations** — video/audio analysis can take minutes
|
||||
7. **Use URL sources** when possible to avoid upload overhead
|
||||
271
skills/official/microsoft/python/foundry/ml/SKILL.md
Normal file
271
skills/official/microsoft/python/foundry/ml/SKILL.md
Normal file
@@ -0,0 +1,271 @@
|
||||
---
|
||||
name: azure-ai-ml-py
|
||||
description: |
|
||||
Azure Machine Learning SDK v2 for Python. Use for ML workspaces, jobs, models, datasets, compute, and pipelines.
|
||||
Triggers: "azure-ai-ml", "MLClient", "workspace", "model registry", "training jobs", "datasets".
|
||||
package: azure-ai-ml
|
||||
---
|
||||
|
||||
# Azure Machine Learning SDK v2 for Python
|
||||
|
||||
Client library for managing Azure ML resources: workspaces, jobs, models, data, and compute.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-ai-ml
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_SUBSCRIPTION_ID=<your-subscription-id>
|
||||
AZURE_RESOURCE_GROUP=<your-resource-group>
|
||||
AZURE_ML_WORKSPACE_NAME=<your-workspace-name>
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.ai.ml import MLClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
ml_client = MLClient(
|
||||
credential=DefaultAzureCredential(),
|
||||
subscription_id=os.environ["AZURE_SUBSCRIPTION_ID"],
|
||||
resource_group_name=os.environ["AZURE_RESOURCE_GROUP"],
|
||||
workspace_name=os.environ["AZURE_ML_WORKSPACE_NAME"]
|
||||
)
|
||||
```
|
||||
|
||||
### From Config File
|
||||
|
||||
```python
|
||||
from azure.ai.ml import MLClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
# Uses config.json in current directory or parent
|
||||
ml_client = MLClient.from_config(
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
```
|
||||
|
||||
## Workspace Management
|
||||
|
||||
### Create Workspace
|
||||
|
||||
```python
|
||||
from azure.ai.ml.entities import Workspace
|
||||
|
||||
ws = Workspace(
|
||||
name="my-workspace",
|
||||
location="eastus",
|
||||
display_name="My Workspace",
|
||||
description="ML workspace for experiments",
|
||||
tags={"purpose": "demo"}
|
||||
)
|
||||
|
||||
ml_client.workspaces.begin_create(ws).result()
|
||||
```
|
||||
|
||||
### List Workspaces
|
||||
|
||||
```python
|
||||
for ws in ml_client.workspaces.list():
|
||||
print(f"{ws.name}: {ws.location}")
|
||||
```
|
||||
|
||||
## Data Assets
|
||||
|
||||
### Register Data
|
||||
|
||||
```python
|
||||
from azure.ai.ml.entities import Data
|
||||
from azure.ai.ml.constants import AssetTypes
|
||||
|
||||
# Register a file
|
||||
my_data = Data(
|
||||
name="my-dataset",
|
||||
version="1",
|
||||
path="azureml://datastores/workspaceblobstore/paths/data/train.csv",
|
||||
type=AssetTypes.URI_FILE,
|
||||
description="Training data"
|
||||
)
|
||||
|
||||
ml_client.data.create_or_update(my_data)
|
||||
```
|
||||
|
||||
### Register Folder
|
||||
|
||||
```python
|
||||
my_data = Data(
|
||||
name="my-folder-dataset",
|
||||
version="1",
|
||||
path="azureml://datastores/workspaceblobstore/paths/data/",
|
||||
type=AssetTypes.URI_FOLDER
|
||||
)
|
||||
|
||||
ml_client.data.create_or_update(my_data)
|
||||
```
|
||||
|
||||
## Model Registry
|
||||
|
||||
### Register Model
|
||||
|
||||
```python
|
||||
from azure.ai.ml.entities import Model
|
||||
from azure.ai.ml.constants import AssetTypes
|
||||
|
||||
model = Model(
|
||||
name="my-model",
|
||||
version="1",
|
||||
path="./model/",
|
||||
type=AssetTypes.CUSTOM_MODEL,
|
||||
description="My trained model"
|
||||
)
|
||||
|
||||
ml_client.models.create_or_update(model)
|
||||
```
|
||||
|
||||
### List Models
|
||||
|
||||
```python
|
||||
for model in ml_client.models.list(name="my-model"):
|
||||
print(f"{model.name} v{model.version}")
|
||||
```
|
||||
|
||||
## Compute
|
||||
|
||||
### Create Compute Cluster
|
||||
|
||||
```python
|
||||
from azure.ai.ml.entities import AmlCompute
|
||||
|
||||
cluster = AmlCompute(
|
||||
name="cpu-cluster",
|
||||
type="amlcompute",
|
||||
size="Standard_DS3_v2",
|
||||
min_instances=0,
|
||||
max_instances=4,
|
||||
idle_time_before_scale_down=120
|
||||
)
|
||||
|
||||
ml_client.compute.begin_create_or_update(cluster).result()
|
||||
```
|
||||
|
||||
### List Compute
|
||||
|
||||
```python
|
||||
for compute in ml_client.compute.list():
|
||||
print(f"{compute.name}: {compute.type}")
|
||||
```
|
||||
|
||||
## Jobs
|
||||
|
||||
### Command Job
|
||||
|
||||
```python
|
||||
from azure.ai.ml import command, Input
|
||||
|
||||
job = command(
|
||||
code="./src",
|
||||
command="python train.py --data ${{inputs.data}} --lr ${{inputs.learning_rate}}",
|
||||
inputs={
|
||||
"data": Input(type="uri_folder", path="azureml:my-dataset:1"),
|
||||
"learning_rate": 0.01
|
||||
},
|
||||
environment="AzureML-sklearn-1.0-ubuntu20.04-py38-cpu@latest",
|
||||
compute="cpu-cluster",
|
||||
display_name="training-job"
|
||||
)
|
||||
|
||||
returned_job = ml_client.jobs.create_or_update(job)
|
||||
print(f"Job URL: {returned_job.studio_url}")
|
||||
```
|
||||
|
||||
### Monitor Job
|
||||
|
||||
```python
|
||||
ml_client.jobs.stream(returned_job.name)
|
||||
```
|
||||
|
||||
## Pipelines
|
||||
|
||||
```python
|
||||
from azure.ai.ml import dsl, Input, Output
|
||||
from azure.ai.ml.entities import Pipeline
|
||||
|
||||
@dsl.pipeline(
|
||||
compute="cpu-cluster",
|
||||
description="Training pipeline"
|
||||
)
|
||||
def training_pipeline(data_input):
|
||||
prep_step = prep_component(data=data_input)
|
||||
train_step = train_component(
|
||||
data=prep_step.outputs.output_data,
|
||||
learning_rate=0.01
|
||||
)
|
||||
return {"model": train_step.outputs.model}
|
||||
|
||||
pipeline = training_pipeline(
|
||||
data_input=Input(type="uri_folder", path="azureml:my-dataset:1")
|
||||
)
|
||||
|
||||
pipeline_job = ml_client.jobs.create_or_update(pipeline)
|
||||
```
|
||||
|
||||
## Environments
|
||||
|
||||
### Create Custom Environment
|
||||
|
||||
```python
|
||||
from azure.ai.ml.entities import Environment
|
||||
|
||||
env = Environment(
|
||||
name="my-env",
|
||||
version="1",
|
||||
image="mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04",
|
||||
conda_file="./environment.yml"
|
||||
)
|
||||
|
||||
ml_client.environments.create_or_update(env)
|
||||
```
|
||||
|
||||
## Datastores
|
||||
|
||||
### List Datastores
|
||||
|
||||
```python
|
||||
for ds in ml_client.datastores.list():
|
||||
print(f"{ds.name}: {ds.type}")
|
||||
```
|
||||
|
||||
### Get Default Datastore
|
||||
|
||||
```python
|
||||
default_ds = ml_client.datastores.get_default()
|
||||
print(f"Default: {default_ds.name}")
|
||||
```
|
||||
|
||||
## MLClient Operations
|
||||
|
||||
| Property | Operations |
|
||||
|----------|------------|
|
||||
| `workspaces` | create, get, list, delete |
|
||||
| `jobs` | create_or_update, get, list, stream, cancel |
|
||||
| `models` | create_or_update, get, list, archive |
|
||||
| `data` | create_or_update, get, list |
|
||||
| `compute` | begin_create_or_update, get, list, delete |
|
||||
| `environments` | create_or_update, get, list |
|
||||
| `datastores` | create_or_update, get, list, get_default |
|
||||
| `components` | create_or_update, get, list |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use versioning** for data, models, and environments
|
||||
2. **Configure idle scale-down** to reduce compute costs
|
||||
3. **Use environments** for reproducible training
|
||||
4. **Stream job logs** to monitor progress
|
||||
5. **Register models** after successful training jobs
|
||||
6. **Use pipelines** for multi-step workflows
|
||||
7. **Tag resources** for organization and cost tracking
|
||||
295
skills/official/microsoft/python/foundry/projects/SKILL.md
Normal file
295
skills/official/microsoft/python/foundry/projects/SKILL.md
Normal file
@@ -0,0 +1,295 @@
|
||||
---
|
||||
name: azure-ai-projects-py
|
||||
description: Build AI applications using the Azure AI Projects Python SDK (azure-ai-projects). Use when working with Foundry project clients, creating versioned agents with PromptAgentDefinition, running evaluations, managing connections/deployments/datasets/indexes, or using OpenAI-compatible clients. This is the high-level Foundry SDK - for low-level agent operations, use azure-ai-agents-python skill.
|
||||
package: azure-ai-projects
|
||||
---
|
||||
|
||||
# Azure AI Projects Python SDK (Foundry SDK)
|
||||
|
||||
Build AI applications on Microsoft Foundry using the `azure-ai-projects` SDK.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-ai-projects azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_AI_PROJECT_ENDPOINT="https://<resource>.services.ai.azure.com/api/projects/<project>"
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4o-mini"
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
import os
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.ai.projects import AIProjectClient
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
client = AIProjectClient(
|
||||
endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"],
|
||||
credential=credential,
|
||||
)
|
||||
```
|
||||
|
||||
## Client Operations Overview
|
||||
|
||||
| Operation | Access | Purpose |
|
||||
|-----------|--------|---------|
|
||||
| `client.agents` | `.agents.*` | Agent CRUD, versions, threads, runs |
|
||||
| `client.connections` | `.connections.*` | List/get project connections |
|
||||
| `client.deployments` | `.deployments.*` | List model deployments |
|
||||
| `client.datasets` | `.datasets.*` | Dataset management |
|
||||
| `client.indexes` | `.indexes.*` | Index management |
|
||||
| `client.evaluations` | `.evaluations.*` | Run evaluations |
|
||||
| `client.red_teams` | `.red_teams.*` | Red team operations |
|
||||
|
||||
## Two Client Approaches
|
||||
|
||||
### 1. AIProjectClient (Native Foundry)
|
||||
|
||||
```python
|
||||
from azure.ai.projects import AIProjectClient
|
||||
|
||||
client = AIProjectClient(
|
||||
endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"],
|
||||
credential=DefaultAzureCredential(),
|
||||
)
|
||||
|
||||
# Use Foundry-native operations
|
||||
agent = client.agents.create_agent(
|
||||
model=os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
|
||||
name="my-agent",
|
||||
instructions="You are helpful.",
|
||||
)
|
||||
```
|
||||
|
||||
### 2. OpenAI-Compatible Client
|
||||
|
||||
```python
|
||||
# Get OpenAI-compatible client from project
|
||||
openai_client = client.get_openai_client()
|
||||
|
||||
# Use standard OpenAI API
|
||||
response = openai_client.chat.completions.create(
|
||||
model=os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
|
||||
messages=[{"role": "user", "content": "Hello!"}],
|
||||
)
|
||||
```
|
||||
|
||||
## Agent Operations
|
||||
|
||||
### Create Agent (Basic)
|
||||
|
||||
```python
|
||||
agent = client.agents.create_agent(
|
||||
model=os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
|
||||
name="my-agent",
|
||||
instructions="You are a helpful assistant.",
|
||||
)
|
||||
```
|
||||
|
||||
### Create Agent with Tools
|
||||
|
||||
```python
|
||||
from azure.ai.agents import CodeInterpreterTool, FileSearchTool
|
||||
|
||||
agent = client.agents.create_agent(
|
||||
model=os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
|
||||
name="tool-agent",
|
||||
instructions="You can execute code and search files.",
|
||||
tools=[CodeInterpreterTool(), FileSearchTool()],
|
||||
)
|
||||
```
|
||||
|
||||
### Versioned Agents with PromptAgentDefinition
|
||||
|
||||
```python
|
||||
from azure.ai.projects.models import PromptAgentDefinition
|
||||
|
||||
# Create a versioned agent
|
||||
agent_version = client.agents.create_version(
|
||||
agent_name="customer-support-agent",
|
||||
definition=PromptAgentDefinition(
|
||||
model=os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
|
||||
instructions="You are a customer support specialist.",
|
||||
tools=[], # Add tools as needed
|
||||
),
|
||||
version_label="v1.0",
|
||||
)
|
||||
```
|
||||
|
||||
See [references/agents.md](references/agents.md) for detailed agent patterns.
|
||||
|
||||
## Tools Overview
|
||||
|
||||
| Tool | Class | Use Case |
|
||||
|------|-------|----------|
|
||||
| Code Interpreter | `CodeInterpreterTool` | Execute Python, generate files |
|
||||
| File Search | `FileSearchTool` | RAG over uploaded documents |
|
||||
| Bing Grounding | `BingGroundingTool` | Web search (requires connection) |
|
||||
| Azure AI Search | `AzureAISearchTool` | Search your indexes |
|
||||
| Function Calling | `FunctionTool` | Call your Python functions |
|
||||
| OpenAPI | `OpenApiTool` | Call REST APIs |
|
||||
| MCP | `McpTool` | Model Context Protocol servers |
|
||||
| Memory Search | `MemorySearchTool` | Search agent memory stores |
|
||||
| SharePoint | `SharepointGroundingTool` | Search SharePoint content |
|
||||
|
||||
See [references/tools.md](references/tools.md) for all tool patterns.
|
||||
|
||||
## Thread and Message Flow
|
||||
|
||||
```python
|
||||
# 1. Create thread
|
||||
thread = client.agents.threads.create()
|
||||
|
||||
# 2. Add message
|
||||
client.agents.messages.create(
|
||||
thread_id=thread.id,
|
||||
role="user",
|
||||
content="What's the weather like?",
|
||||
)
|
||||
|
||||
# 3. Create and process run
|
||||
run = client.agents.runs.create_and_process(
|
||||
thread_id=thread.id,
|
||||
agent_id=agent.id,
|
||||
)
|
||||
|
||||
# 4. Get response
|
||||
if run.status == "completed":
|
||||
messages = client.agents.messages.list(thread_id=thread.id)
|
||||
for msg in messages:
|
||||
if msg.role == "assistant":
|
||||
print(msg.content[0].text.value)
|
||||
```
|
||||
|
||||
## Connections
|
||||
|
||||
```python
|
||||
# List all connections
|
||||
connections = client.connections.list()
|
||||
for conn in connections:
|
||||
print(f"{conn.name}: {conn.connection_type}")
|
||||
|
||||
# Get specific connection
|
||||
connection = client.connections.get(connection_name="my-search-connection")
|
||||
```
|
||||
|
||||
See [references/connections.md](references/connections.md) for connection patterns.
|
||||
|
||||
## Deployments
|
||||
|
||||
```python
|
||||
# List available model deployments
|
||||
deployments = client.deployments.list()
|
||||
for deployment in deployments:
|
||||
print(f"{deployment.name}: {deployment.model}")
|
||||
```
|
||||
|
||||
See [references/deployments.md](references/deployments.md) for deployment patterns.
|
||||
|
||||
## Datasets and Indexes
|
||||
|
||||
```python
|
||||
# List datasets
|
||||
datasets = client.datasets.list()
|
||||
|
||||
# List indexes
|
||||
indexes = client.indexes.list()
|
||||
```
|
||||
|
||||
See [references/datasets-indexes.md](references/datasets-indexes.md) for data operations.
|
||||
|
||||
## Evaluation
|
||||
|
||||
```python
|
||||
# Using OpenAI client for evals
|
||||
openai_client = client.get_openai_client()
|
||||
|
||||
# Create evaluation with built-in evaluators
|
||||
eval_run = openai_client.evals.runs.create(
|
||||
eval_id="my-eval",
|
||||
name="quality-check",
|
||||
data_source={
|
||||
"type": "custom",
|
||||
"item_references": [{"item_id": "test-1"}],
|
||||
},
|
||||
testing_criteria=[
|
||||
{"type": "fluency"},
|
||||
{"type": "task_adherence"},
|
||||
],
|
||||
)
|
||||
```
|
||||
|
||||
See [references/evaluation.md](references/evaluation.md) for evaluation patterns.
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.ai.projects.aio import AIProjectClient
|
||||
|
||||
async with AIProjectClient(
|
||||
endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"],
|
||||
credential=DefaultAzureCredential(),
|
||||
) as client:
|
||||
agent = await client.agents.create_agent(...)
|
||||
# ... async operations
|
||||
```
|
||||
|
||||
See [references/async-patterns.md](references/async-patterns.md) for async patterns.
|
||||
|
||||
## Memory Stores
|
||||
|
||||
```python
|
||||
# Create memory store for agent
|
||||
memory_store = client.agents.create_memory_store(
|
||||
name="conversation-memory",
|
||||
)
|
||||
|
||||
# Attach to agent for persistent memory
|
||||
agent = client.agents.create_agent(
|
||||
model=os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
|
||||
name="memory-agent",
|
||||
tools=[MemorySearchTool()],
|
||||
tool_resources={"memory": {"store_ids": [memory_store.id]}},
|
||||
)
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use context managers** for async client: `async with AIProjectClient(...) as client:`
|
||||
2. **Clean up agents** when done: `client.agents.delete_agent(agent.id)`
|
||||
3. **Use `create_and_process`** for simple runs, **streaming** for real-time UX
|
||||
4. **Use versioned agents** for production deployments
|
||||
5. **Prefer connections** for external service integration (AI Search, Bing, etc.)
|
||||
|
||||
## SDK Comparison
|
||||
|
||||
| Feature | `azure-ai-projects` | `azure-ai-agents` |
|
||||
|---------|---------------------|-------------------|
|
||||
| Level | High-level (Foundry) | Low-level (Agents) |
|
||||
| Client | `AIProjectClient` | `AgentsClient` |
|
||||
| Versioning | `create_version()` | Not available |
|
||||
| Connections | Yes | No |
|
||||
| Deployments | Yes | No |
|
||||
| Datasets/Indexes | Yes | No |
|
||||
| Evaluation | Via OpenAI client | No |
|
||||
| When to use | Full Foundry integration | Standalone agent apps |
|
||||
|
||||
## Reference Files
|
||||
|
||||
- [references/agents.md](references/agents.md): Agent operations with PromptAgentDefinition
|
||||
- [references/tools.md](references/tools.md): All agent tools with examples
|
||||
- [references/evaluation.md](references/evaluation.md): Evaluation operations overview
|
||||
- [references/built-in-evaluators.md](references/built-in-evaluators.md): Complete built-in evaluator reference
|
||||
- [references/custom-evaluators.md](references/custom-evaluators.md): Code and prompt-based evaluator patterns
|
||||
- [references/connections.md](references/connections.md): Connection operations
|
||||
- [references/deployments.md](references/deployments.md): Deployment enumeration
|
||||
- [references/datasets-indexes.md](references/datasets-indexes.md): Dataset and index operations
|
||||
- [references/async-patterns.md](references/async-patterns.md): Async client usage
|
||||
- [references/api-reference.md](references/api-reference.md): Complete API reference for all 373 SDK exports (v2.0.0b4)
|
||||
- [scripts/run_batch_evaluation.py](scripts/run_batch_evaluation.py): CLI tool for batch evaluations
|
||||
@@ -0,0 +1,528 @@
|
||||
---
|
||||
name: azure-search-documents-py
|
||||
description: |
|
||||
Azure AI Search SDK for Python. Use for vector search, hybrid search, semantic ranking, indexing, and skillsets.
|
||||
Triggers: "azure-search-documents", "SearchClient", "SearchIndexClient", "vector search", "hybrid search", "semantic search".
|
||||
package: azure-search-documents
|
||||
---
|
||||
|
||||
# Azure AI Search SDK for Python
|
||||
|
||||
Full-text, vector, and hybrid search with AI enrichment capabilities.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-search-documents
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_SEARCH_ENDPOINT=https://<service-name>.search.windows.net
|
||||
AZURE_SEARCH_API_KEY=<your-api-key>
|
||||
AZURE_SEARCH_INDEX_NAME=<your-index-name>
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
### API Key
|
||||
|
||||
```python
|
||||
from azure.search.documents import SearchClient
|
||||
from azure.core.credentials import AzureKeyCredential
|
||||
|
||||
client = SearchClient(
|
||||
endpoint=os.environ["AZURE_SEARCH_ENDPOINT"],
|
||||
index_name=os.environ["AZURE_SEARCH_INDEX_NAME"],
|
||||
credential=AzureKeyCredential(os.environ["AZURE_SEARCH_API_KEY"])
|
||||
)
|
||||
```
|
||||
|
||||
### Entra ID (Recommended)
|
||||
|
||||
```python
|
||||
from azure.search.documents import SearchClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
client = SearchClient(
|
||||
endpoint=os.environ["AZURE_SEARCH_ENDPOINT"],
|
||||
index_name=os.environ["AZURE_SEARCH_INDEX_NAME"],
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
```
|
||||
|
||||
## Client Types
|
||||
|
||||
| Client | Purpose |
|
||||
|--------|---------|
|
||||
| `SearchClient` | Search and document operations |
|
||||
| `SearchIndexClient` | Index management, synonym maps |
|
||||
| `SearchIndexerClient` | Indexers, data sources, skillsets |
|
||||
|
||||
## Create Index with Vector Field
|
||||
|
||||
```python
|
||||
from azure.search.documents.indexes import SearchIndexClient
|
||||
from azure.search.documents.indexes.models import (
|
||||
SearchIndex,
|
||||
SearchField,
|
||||
SearchFieldDataType,
|
||||
VectorSearch,
|
||||
HnswAlgorithmConfiguration,
|
||||
VectorSearchProfile,
|
||||
SearchableField,
|
||||
SimpleField
|
||||
)
|
||||
|
||||
index_client = SearchIndexClient(endpoint, AzureKeyCredential(key))
|
||||
|
||||
fields = [
|
||||
SimpleField(name="id", type=SearchFieldDataType.String, key=True),
|
||||
SearchableField(name="title", type=SearchFieldDataType.String),
|
||||
SearchableField(name="content", type=SearchFieldDataType.String),
|
||||
SearchField(
|
||||
name="content_vector",
|
||||
type=SearchFieldDataType.Collection(SearchFieldDataType.Single),
|
||||
searchable=True,
|
||||
vector_search_dimensions=1536,
|
||||
vector_search_profile_name="my-vector-profile"
|
||||
)
|
||||
]
|
||||
|
||||
vector_search = VectorSearch(
|
||||
algorithms=[
|
||||
HnswAlgorithmConfiguration(name="my-hnsw")
|
||||
],
|
||||
profiles=[
|
||||
VectorSearchProfile(
|
||||
name="my-vector-profile",
|
||||
algorithm_configuration_name="my-hnsw"
|
||||
)
|
||||
]
|
||||
)
|
||||
|
||||
index = SearchIndex(
|
||||
name="my-index",
|
||||
fields=fields,
|
||||
vector_search=vector_search
|
||||
)
|
||||
|
||||
index_client.create_or_update_index(index)
|
||||
```
|
||||
|
||||
## Upload Documents
|
||||
|
||||
```python
|
||||
from azure.search.documents import SearchClient
|
||||
|
||||
client = SearchClient(endpoint, "my-index", AzureKeyCredential(key))
|
||||
|
||||
documents = [
|
||||
{
|
||||
"id": "1",
|
||||
"title": "Azure AI Search",
|
||||
"content": "Full-text and vector search service",
|
||||
"content_vector": [0.1, 0.2, ...] # 1536 dimensions
|
||||
}
|
||||
]
|
||||
|
||||
result = client.upload_documents(documents)
|
||||
print(f"Uploaded {len(result)} documents")
|
||||
```
|
||||
|
||||
## Keyword Search
|
||||
|
||||
```python
|
||||
results = client.search(
|
||||
search_text="azure search",
|
||||
select=["id", "title", "content"],
|
||||
top=10
|
||||
)
|
||||
|
||||
for result in results:
|
||||
print(f"{result['title']}: {result['@search.score']}")
|
||||
```
|
||||
|
||||
## Vector Search
|
||||
|
||||
```python
|
||||
from azure.search.documents.models import VectorizedQuery
|
||||
|
||||
# Your query embedding (1536 dimensions)
|
||||
query_vector = get_embedding("semantic search capabilities")
|
||||
|
||||
vector_query = VectorizedQuery(
|
||||
vector=query_vector,
|
||||
k_nearest_neighbors=10,
|
||||
fields="content_vector"
|
||||
)
|
||||
|
||||
results = client.search(
|
||||
vector_queries=[vector_query],
|
||||
select=["id", "title", "content"]
|
||||
)
|
||||
|
||||
for result in results:
|
||||
print(f"{result['title']}: {result['@search.score']}")
|
||||
```
|
||||
|
||||
## Hybrid Search (Vector + Keyword)
|
||||
|
||||
```python
|
||||
from azure.search.documents.models import VectorizedQuery
|
||||
|
||||
vector_query = VectorizedQuery(
|
||||
vector=query_vector,
|
||||
k_nearest_neighbors=10,
|
||||
fields="content_vector"
|
||||
)
|
||||
|
||||
results = client.search(
|
||||
search_text="azure search",
|
||||
vector_queries=[vector_query],
|
||||
select=["id", "title", "content"],
|
||||
top=10
|
||||
)
|
||||
```
|
||||
|
||||
## Semantic Ranking
|
||||
|
||||
```python
|
||||
from azure.search.documents.models import QueryType
|
||||
|
||||
results = client.search(
|
||||
search_text="what is azure search",
|
||||
query_type=QueryType.SEMANTIC,
|
||||
semantic_configuration_name="my-semantic-config",
|
||||
select=["id", "title", "content"],
|
||||
top=10
|
||||
)
|
||||
|
||||
for result in results:
|
||||
print(f"{result['title']}")
|
||||
if result.get("@search.captions"):
|
||||
print(f" Caption: {result['@search.captions'][0].text}")
|
||||
```
|
||||
|
||||
## Filters
|
||||
|
||||
```python
|
||||
results = client.search(
|
||||
search_text="*",
|
||||
filter="category eq 'Technology' and rating gt 4",
|
||||
order_by=["rating desc"],
|
||||
select=["id", "title", "category", "rating"]
|
||||
)
|
||||
```
|
||||
|
||||
## Facets
|
||||
|
||||
```python
|
||||
results = client.search(
|
||||
search_text="*",
|
||||
facets=["category,count:10", "rating"],
|
||||
top=0 # Only get facets, no documents
|
||||
)
|
||||
|
||||
for facet_name, facet_values in results.get_facets().items():
|
||||
print(f"{facet_name}:")
|
||||
for facet in facet_values:
|
||||
print(f" {facet['value']}: {facet['count']}")
|
||||
```
|
||||
|
||||
## Autocomplete & Suggest
|
||||
|
||||
```python
|
||||
# Autocomplete
|
||||
results = client.autocomplete(
|
||||
search_text="sea",
|
||||
suggester_name="my-suggester",
|
||||
mode="twoTerms"
|
||||
)
|
||||
|
||||
# Suggest
|
||||
results = client.suggest(
|
||||
search_text="sea",
|
||||
suggester_name="my-suggester",
|
||||
select=["title"]
|
||||
)
|
||||
```
|
||||
|
||||
## Indexer with Skillset
|
||||
|
||||
```python
|
||||
from azure.search.documents.indexes import SearchIndexerClient
|
||||
from azure.search.documents.indexes.models import (
|
||||
SearchIndexer,
|
||||
SearchIndexerDataSourceConnection,
|
||||
SearchIndexerSkillset,
|
||||
EntityRecognitionSkill,
|
||||
InputFieldMappingEntry,
|
||||
OutputFieldMappingEntry
|
||||
)
|
||||
|
||||
indexer_client = SearchIndexerClient(endpoint, AzureKeyCredential(key))
|
||||
|
||||
# Create data source
|
||||
data_source = SearchIndexerDataSourceConnection(
|
||||
name="my-datasource",
|
||||
type="azureblob",
|
||||
connection_string=connection_string,
|
||||
container={"name": "documents"}
|
||||
)
|
||||
indexer_client.create_or_update_data_source_connection(data_source)
|
||||
|
||||
# Create skillset
|
||||
skillset = SearchIndexerSkillset(
|
||||
name="my-skillset",
|
||||
skills=[
|
||||
EntityRecognitionSkill(
|
||||
inputs=[InputFieldMappingEntry(name="text", source="/document/content")],
|
||||
outputs=[OutputFieldMappingEntry(name="organizations", target_name="organizations")]
|
||||
)
|
||||
]
|
||||
)
|
||||
indexer_client.create_or_update_skillset(skillset)
|
||||
|
||||
# Create indexer
|
||||
indexer = SearchIndexer(
|
||||
name="my-indexer",
|
||||
data_source_name="my-datasource",
|
||||
target_index_name="my-index",
|
||||
skillset_name="my-skillset"
|
||||
)
|
||||
indexer_client.create_or_update_indexer(indexer)
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use hybrid search** for best relevance combining vector and keyword
|
||||
2. **Enable semantic ranking** for natural language queries
|
||||
3. **Index in batches** of 100-1000 documents for efficiency
|
||||
4. **Use filters** to narrow results before ranking
|
||||
5. **Configure vector dimensions** to match your embedding model
|
||||
6. **Use HNSW algorithm** for large-scale vector search
|
||||
7. **Create suggesters** at index creation time (cannot add later)
|
||||
|
||||
## Reference Files
|
||||
|
||||
| File | Contents |
|
||||
|------|----------|
|
||||
| [references/vector-search.md](references/vector-search.md) | HNSW configuration, integrated vectorization, multi-vector queries |
|
||||
| [references/semantic-ranking.md](references/semantic-ranking.md) | Semantic configuration, captions, answers, hybrid patterns |
|
||||
| [scripts/setup_vector_index.py](scripts/setup_vector_index.py) | CLI script to create vector-enabled search index |
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Additional Azure AI Search Patterns
|
||||
|
||||
# Azure AI Search Python SDK
|
||||
|
||||
Write clean, idiomatic Python code for Azure AI Search using `azure-search-documents`.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-search-documents azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_SEARCH_ENDPOINT=https://<search-service>.search.windows.net
|
||||
AZURE_SEARCH_INDEX_NAME=<index-name>
|
||||
# For API key auth (not recommended for production)
|
||||
AZURE_SEARCH_API_KEY=<api-key>
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
**DefaultAzureCredential (preferred)**:
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.search.documents import SearchClient
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
client = SearchClient(endpoint, index_name, credential)
|
||||
```
|
||||
|
||||
**API Key**:
|
||||
```python
|
||||
from azure.core.credentials import AzureKeyCredential
|
||||
from azure.search.documents import SearchClient
|
||||
|
||||
client = SearchClient(endpoint, index_name, AzureKeyCredential(api_key))
|
||||
```
|
||||
|
||||
## Client Selection
|
||||
|
||||
| Client | Purpose |
|
||||
|--------|---------|
|
||||
| `SearchClient` | Query indexes, upload/update/delete documents |
|
||||
| `SearchIndexClient` | Create/manage indexes, knowledge sources, knowledge bases |
|
||||
| `SearchIndexerClient` | Manage indexers, skillsets, data sources |
|
||||
| `KnowledgeBaseRetrievalClient` | Agentic retrieval with LLM-powered Q&A |
|
||||
|
||||
## Index Creation Pattern
|
||||
|
||||
```python
|
||||
from azure.search.documents.indexes import SearchIndexClient
|
||||
from azure.search.documents.indexes.models import (
|
||||
SearchIndex, SearchField, VectorSearch, VectorSearchProfile,
|
||||
HnswAlgorithmConfiguration, AzureOpenAIVectorizer,
|
||||
AzureOpenAIVectorizerParameters, SemanticSearch,
|
||||
SemanticConfiguration, SemanticPrioritizedFields, SemanticField
|
||||
)
|
||||
|
||||
index = SearchIndex(
|
||||
name=index_name,
|
||||
fields=[
|
||||
SearchField(name="id", type="Edm.String", key=True),
|
||||
SearchField(name="content", type="Edm.String", searchable=True),
|
||||
SearchField(name="embedding", type="Collection(Edm.Single)",
|
||||
vector_search_dimensions=3072,
|
||||
vector_search_profile_name="vector-profile"),
|
||||
],
|
||||
vector_search=VectorSearch(
|
||||
profiles=[VectorSearchProfile(
|
||||
name="vector-profile",
|
||||
algorithm_configuration_name="hnsw-algo",
|
||||
vectorizer_name="openai-vectorizer"
|
||||
)],
|
||||
algorithms=[HnswAlgorithmConfiguration(name="hnsw-algo")],
|
||||
vectorizers=[AzureOpenAIVectorizer(
|
||||
vectorizer_name="openai-vectorizer",
|
||||
parameters=AzureOpenAIVectorizerParameters(
|
||||
resource_url=aoai_endpoint,
|
||||
deployment_name=embedding_deployment,
|
||||
model_name=embedding_model
|
||||
)
|
||||
)]
|
||||
),
|
||||
semantic_search=SemanticSearch(
|
||||
default_configuration_name="semantic-config",
|
||||
configurations=[SemanticConfiguration(
|
||||
name="semantic-config",
|
||||
prioritized_fields=SemanticPrioritizedFields(
|
||||
content_fields=[SemanticField(field_name="content")]
|
||||
)
|
||||
)]
|
||||
)
|
||||
)
|
||||
|
||||
index_client = SearchIndexClient(endpoint, credential)
|
||||
index_client.create_or_update_index(index)
|
||||
```
|
||||
|
||||
## Document Operations
|
||||
|
||||
```python
|
||||
from azure.search.documents import SearchIndexingBufferedSender
|
||||
|
||||
# Batch upload with automatic batching
|
||||
with SearchIndexingBufferedSender(endpoint, index_name, credential) as sender:
|
||||
sender.upload_documents(documents)
|
||||
|
||||
# Direct operations via SearchClient
|
||||
search_client = SearchClient(endpoint, index_name, credential)
|
||||
search_client.upload_documents(documents) # Add new
|
||||
search_client.merge_documents(documents) # Update existing
|
||||
search_client.merge_or_upload_documents(documents) # Upsert
|
||||
search_client.delete_documents(documents) # Remove
|
||||
```
|
||||
|
||||
## Search Patterns
|
||||
|
||||
```python
|
||||
# Basic search
|
||||
results = search_client.search(search_text="query")
|
||||
|
||||
# Vector search
|
||||
from azure.search.documents.models import VectorizedQuery
|
||||
|
||||
results = search_client.search(
|
||||
search_text=None,
|
||||
vector_queries=[VectorizedQuery(
|
||||
vector=embedding,
|
||||
k_nearest_neighbors=5,
|
||||
fields="embedding"
|
||||
)]
|
||||
)
|
||||
|
||||
# Hybrid search (vector + keyword)
|
||||
results = search_client.search(
|
||||
search_text="query",
|
||||
vector_queries=[VectorizedQuery(vector=embedding, k_nearest_neighbors=5, fields="embedding")],
|
||||
query_type="semantic",
|
||||
semantic_configuration_name="semantic-config"
|
||||
)
|
||||
|
||||
# With filters
|
||||
results = search_client.search(
|
||||
search_text="query",
|
||||
filter="category eq 'technology'",
|
||||
select=["id", "title", "content"],
|
||||
top=10
|
||||
)
|
||||
```
|
||||
|
||||
## Agentic Retrieval (Knowledge Bases)
|
||||
|
||||
For LLM-powered Q&A with answer synthesis, see [references/agentic-retrieval.md](references/agentic-retrieval.md).
|
||||
|
||||
Key concepts:
|
||||
- **Knowledge Source**: Points to a search index
|
||||
- **Knowledge Base**: Wraps knowledge sources + LLM for query planning and synthesis
|
||||
- **Output modes**: `EXTRACTIVE_DATA` (raw chunks) or `ANSWER_SYNTHESIS` (LLM-generated answers)
|
||||
|
||||
## Async Pattern
|
||||
|
||||
```python
|
||||
from azure.search.documents.aio import SearchClient
|
||||
|
||||
async with SearchClient(endpoint, index_name, credential) as client:
|
||||
results = await client.search(search_text="query")
|
||||
async for result in results:
|
||||
print(result["title"])
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use environment variables** for endpoints, keys, and deployment names
|
||||
2. **Prefer `DefaultAzureCredential`** over API keys for production
|
||||
3. **Use `SearchIndexingBufferedSender`** for batch uploads (handles batching/retries)
|
||||
4. **Always define semantic configuration** for agentic retrieval indexes
|
||||
5. **Use `create_or_update_index`** for idempotent index creation
|
||||
6. **Close clients** with context managers or explicit `close()`
|
||||
|
||||
## Field Types Reference
|
||||
|
||||
| EDM Type | Python | Notes |
|
||||
|----------|--------|-------|
|
||||
| `Edm.String` | str | Searchable text |
|
||||
| `Edm.Int32` | int | Integer |
|
||||
| `Edm.Int64` | int | Long integer |
|
||||
| `Edm.Double` | float | Floating point |
|
||||
| `Edm.Boolean` | bool | True/False |
|
||||
| `Edm.DateTimeOffset` | datetime | ISO 8601 |
|
||||
| `Collection(Edm.Single)` | List[float] | Vector embeddings |
|
||||
| `Collection(Edm.String)` | List[str] | String arrays |
|
||||
|
||||
## Error Handling
|
||||
|
||||
```python
|
||||
from azure.core.exceptions import (
|
||||
HttpResponseError,
|
||||
ResourceNotFoundError,
|
||||
ResourceExistsError
|
||||
)
|
||||
|
||||
try:
|
||||
result = search_client.get_document(key="123")
|
||||
except ResourceNotFoundError:
|
||||
print("Document not found")
|
||||
except HttpResponseError as e:
|
||||
print(f"Search error: {e.message}")
|
||||
```
|
||||
@@ -0,0 +1,372 @@
|
||||
---
|
||||
name: azure-speech-to-text-rest-py
|
||||
description: |
|
||||
Azure Speech to Text REST API for short audio (Python). Use for simple speech recognition of audio files up to 60 seconds without the Speech SDK.
|
||||
Triggers: "speech to text REST", "short audio transcription", "speech recognition REST API", "STT REST", "recognize speech REST".
|
||||
DO NOT USE FOR: Long audio (>60 seconds), real-time streaming, batch transcription, custom speech models, speech translation. Use Speech SDK or Batch Transcription API instead.
|
||||
---
|
||||
|
||||
# Azure Speech to Text REST API for Short Audio
|
||||
|
||||
Simple REST API for speech-to-text transcription of short audio files (up to 60 seconds). No SDK required - just HTTP requests.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. **Azure subscription** - [Create one free](https://azure.microsoft.com/free/)
|
||||
2. **Speech resource** - Create in [Azure Portal](https://portal.azure.com/#create/Microsoft.CognitiveServicesSpeechServices)
|
||||
3. **Get credentials** - After deployment, go to resource > Keys and Endpoint
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
# Required
|
||||
AZURE_SPEECH_KEY=<your-speech-resource-key>
|
||||
AZURE_SPEECH_REGION=<region> # e.g., eastus, westus2, westeurope
|
||||
|
||||
# Alternative: Use endpoint directly
|
||||
AZURE_SPEECH_ENDPOINT=https://<region>.stt.speech.microsoft.com
|
||||
```
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install requests
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
import os
|
||||
import requests
|
||||
|
||||
def transcribe_audio(audio_file_path: str, language: str = "en-US") -> dict:
|
||||
"""Transcribe short audio file (max 60 seconds) using REST API."""
|
||||
region = os.environ["AZURE_SPEECH_REGION"]
|
||||
api_key = os.environ["AZURE_SPEECH_KEY"]
|
||||
|
||||
url = f"https://{region}.stt.speech.microsoft.com/speech/recognition/conversation/cognitiveservices/v1"
|
||||
|
||||
headers = {
|
||||
"Ocp-Apim-Subscription-Key": api_key,
|
||||
"Content-Type": "audio/wav; codecs=audio/pcm; samplerate=16000",
|
||||
"Accept": "application/json"
|
||||
}
|
||||
|
||||
params = {
|
||||
"language": language,
|
||||
"format": "detailed" # or "simple"
|
||||
}
|
||||
|
||||
with open(audio_file_path, "rb") as audio_file:
|
||||
response = requests.post(url, headers=headers, params=params, data=audio_file)
|
||||
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
# Usage
|
||||
result = transcribe_audio("audio.wav", "en-US")
|
||||
print(result["DisplayText"])
|
||||
```
|
||||
|
||||
## Audio Requirements
|
||||
|
||||
| Format | Codec | Sample Rate | Notes |
|
||||
|--------|-------|-------------|-------|
|
||||
| WAV | PCM | 16 kHz, mono | **Recommended** |
|
||||
| OGG | OPUS | 16 kHz, mono | Smaller file size |
|
||||
|
||||
**Limitations:**
|
||||
- Maximum 60 seconds of audio
|
||||
- For pronunciation assessment: maximum 30 seconds
|
||||
- No partial/interim results (final only)
|
||||
|
||||
## Content-Type Headers
|
||||
|
||||
```python
|
||||
# WAV PCM 16kHz
|
||||
"Content-Type": "audio/wav; codecs=audio/pcm; samplerate=16000"
|
||||
|
||||
# OGG OPUS
|
||||
"Content-Type": "audio/ogg; codecs=opus"
|
||||
```
|
||||
|
||||
## Response Formats
|
||||
|
||||
### Simple Format (default)
|
||||
|
||||
```python
|
||||
params = {"language": "en-US", "format": "simple"}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"RecognitionStatus": "Success",
|
||||
"DisplayText": "Remind me to buy 5 pencils.",
|
||||
"Offset": "1236645672289",
|
||||
"Duration": "1236645672289"
|
||||
}
|
||||
```
|
||||
|
||||
### Detailed Format
|
||||
|
||||
```python
|
||||
params = {"language": "en-US", "format": "detailed"}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"RecognitionStatus": "Success",
|
||||
"Offset": "1236645672289",
|
||||
"Duration": "1236645672289",
|
||||
"NBest": [
|
||||
{
|
||||
"Confidence": 0.9052885,
|
||||
"Display": "What's the weather like?",
|
||||
"ITN": "what's the weather like",
|
||||
"Lexical": "what's the weather like",
|
||||
"MaskedITN": "what's the weather like"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Chunked Transfer (Recommended)
|
||||
|
||||
For lower latency, stream audio in chunks:
|
||||
|
||||
```python
|
||||
import os
|
||||
import requests
|
||||
|
||||
def transcribe_chunked(audio_file_path: str, language: str = "en-US") -> dict:
|
||||
"""Stream audio in chunks for lower latency."""
|
||||
region = os.environ["AZURE_SPEECH_REGION"]
|
||||
api_key = os.environ["AZURE_SPEECH_KEY"]
|
||||
|
||||
url = f"https://{region}.stt.speech.microsoft.com/speech/recognition/conversation/cognitiveservices/v1"
|
||||
|
||||
headers = {
|
||||
"Ocp-Apim-Subscription-Key": api_key,
|
||||
"Content-Type": "audio/wav; codecs=audio/pcm; samplerate=16000",
|
||||
"Accept": "application/json",
|
||||
"Transfer-Encoding": "chunked",
|
||||
"Expect": "100-continue"
|
||||
}
|
||||
|
||||
params = {"language": language, "format": "detailed"}
|
||||
|
||||
def generate_chunks(file_path: str, chunk_size: int = 1024):
|
||||
with open(file_path, "rb") as f:
|
||||
while chunk := f.read(chunk_size):
|
||||
yield chunk
|
||||
|
||||
response = requests.post(
|
||||
url,
|
||||
headers=headers,
|
||||
params=params,
|
||||
data=generate_chunks(audio_file_path)
|
||||
)
|
||||
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
```
|
||||
|
||||
## Authentication Options
|
||||
|
||||
### Option 1: Subscription Key (Simple)
|
||||
|
||||
```python
|
||||
headers = {
|
||||
"Ocp-Apim-Subscription-Key": os.environ["AZURE_SPEECH_KEY"]
|
||||
}
|
||||
```
|
||||
|
||||
### Option 2: Bearer Token
|
||||
|
||||
```python
|
||||
import requests
|
||||
import os
|
||||
|
||||
def get_access_token() -> str:
|
||||
"""Get access token from the token endpoint."""
|
||||
region = os.environ["AZURE_SPEECH_REGION"]
|
||||
api_key = os.environ["AZURE_SPEECH_KEY"]
|
||||
|
||||
token_url = f"https://{region}.api.cognitive.microsoft.com/sts/v1.0/issueToken"
|
||||
|
||||
response = requests.post(
|
||||
token_url,
|
||||
headers={
|
||||
"Ocp-Apim-Subscription-Key": api_key,
|
||||
"Content-Type": "application/x-www-form-urlencoded",
|
||||
"Content-Length": "0"
|
||||
}
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.text
|
||||
|
||||
# Use token in requests (valid for 10 minutes)
|
||||
token = get_access_token()
|
||||
headers = {
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Content-Type": "audio/wav; codecs=audio/pcm; samplerate=16000",
|
||||
"Accept": "application/json"
|
||||
}
|
||||
```
|
||||
|
||||
## Query Parameters
|
||||
|
||||
| Parameter | Required | Values | Description |
|
||||
|-----------|----------|--------|-------------|
|
||||
| `language` | **Yes** | `en-US`, `de-DE`, etc. | Language of speech |
|
||||
| `format` | No | `simple`, `detailed` | Result format (default: simple) |
|
||||
| `profanity` | No | `masked`, `removed`, `raw` | Profanity handling (default: masked) |
|
||||
|
||||
## Recognition Status Values
|
||||
|
||||
| Status | Description |
|
||||
|--------|-------------|
|
||||
| `Success` | Recognition succeeded |
|
||||
| `NoMatch` | Speech detected but no words matched |
|
||||
| `InitialSilenceTimeout` | Only silence detected |
|
||||
| `BabbleTimeout` | Only noise detected |
|
||||
| `Error` | Internal service error |
|
||||
|
||||
## Profanity Handling
|
||||
|
||||
```python
|
||||
# Mask profanity with asterisks (default)
|
||||
params = {"language": "en-US", "profanity": "masked"}
|
||||
|
||||
# Remove profanity entirely
|
||||
params = {"language": "en-US", "profanity": "removed"}
|
||||
|
||||
# Include profanity as-is
|
||||
params = {"language": "en-US", "profanity": "raw"}
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
```python
|
||||
import requests
|
||||
|
||||
def transcribe_with_error_handling(audio_path: str, language: str = "en-US") -> dict | None:
|
||||
"""Transcribe with proper error handling."""
|
||||
region = os.environ["AZURE_SPEECH_REGION"]
|
||||
api_key = os.environ["AZURE_SPEECH_KEY"]
|
||||
|
||||
url = f"https://{region}.stt.speech.microsoft.com/speech/recognition/conversation/cognitiveservices/v1"
|
||||
|
||||
try:
|
||||
with open(audio_path, "rb") as audio_file:
|
||||
response = requests.post(
|
||||
url,
|
||||
headers={
|
||||
"Ocp-Apim-Subscription-Key": api_key,
|
||||
"Content-Type": "audio/wav; codecs=audio/pcm; samplerate=16000",
|
||||
"Accept": "application/json"
|
||||
},
|
||||
params={"language": language, "format": "detailed"},
|
||||
data=audio_file
|
||||
)
|
||||
|
||||
if response.status_code == 200:
|
||||
result = response.json()
|
||||
if result.get("RecognitionStatus") == "Success":
|
||||
return result
|
||||
else:
|
||||
print(f"Recognition failed: {result.get('RecognitionStatus')}")
|
||||
return None
|
||||
elif response.status_code == 400:
|
||||
print(f"Bad request: Check language code or audio format")
|
||||
elif response.status_code == 401:
|
||||
print(f"Unauthorized: Check API key or token")
|
||||
elif response.status_code == 403:
|
||||
print(f"Forbidden: Missing authorization header")
|
||||
else:
|
||||
print(f"Error {response.status_code}: {response.text}")
|
||||
|
||||
return None
|
||||
|
||||
except requests.exceptions.RequestException as e:
|
||||
print(f"Request failed: {e}")
|
||||
return None
|
||||
```
|
||||
|
||||
## Async Version
|
||||
|
||||
```python
|
||||
import os
|
||||
import aiohttp
|
||||
import asyncio
|
||||
|
||||
async def transcribe_async(audio_file_path: str, language: str = "en-US") -> dict:
|
||||
"""Async version using aiohttp."""
|
||||
region = os.environ["AZURE_SPEECH_REGION"]
|
||||
api_key = os.environ["AZURE_SPEECH_KEY"]
|
||||
|
||||
url = f"https://{region}.stt.speech.microsoft.com/speech/recognition/conversation/cognitiveservices/v1"
|
||||
|
||||
headers = {
|
||||
"Ocp-Apim-Subscription-Key": api_key,
|
||||
"Content-Type": "audio/wav; codecs=audio/pcm; samplerate=16000",
|
||||
"Accept": "application/json"
|
||||
}
|
||||
|
||||
params = {"language": language, "format": "detailed"}
|
||||
|
||||
async with aiohttp.ClientSession() as session:
|
||||
with open(audio_file_path, "rb") as f:
|
||||
audio_data = f.read()
|
||||
|
||||
async with session.post(url, headers=headers, params=params, data=audio_data) as response:
|
||||
response.raise_for_status()
|
||||
return await response.json()
|
||||
|
||||
# Usage
|
||||
result = asyncio.run(transcribe_async("audio.wav", "en-US"))
|
||||
print(result["DisplayText"])
|
||||
```
|
||||
|
||||
## Supported Languages
|
||||
|
||||
Common language codes (see [full list](https://learn.microsoft.com/azure/ai-services/speech-service/language-support)):
|
||||
|
||||
| Code | Language |
|
||||
|------|----------|
|
||||
| `en-US` | English (US) |
|
||||
| `en-GB` | English (UK) |
|
||||
| `de-DE` | German |
|
||||
| `fr-FR` | French |
|
||||
| `es-ES` | Spanish (Spain) |
|
||||
| `es-MX` | Spanish (Mexico) |
|
||||
| `zh-CN` | Chinese (Mandarin) |
|
||||
| `ja-JP` | Japanese |
|
||||
| `ko-KR` | Korean |
|
||||
| `pt-BR` | Portuguese (Brazil) |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use WAV PCM 16kHz mono** for best compatibility
|
||||
2. **Enable chunked transfer** for lower latency
|
||||
3. **Cache access tokens** for 9 minutes (valid for 10)
|
||||
4. **Specify the correct language** for accurate recognition
|
||||
5. **Use detailed format** when you need confidence scores
|
||||
6. **Handle all RecognitionStatus values** in production code
|
||||
|
||||
## When NOT to Use This API
|
||||
|
||||
Use the Speech SDK or Batch Transcription API instead when you need:
|
||||
|
||||
- Audio longer than 60 seconds
|
||||
- Real-time streaming transcription
|
||||
- Partial/interim results
|
||||
- Speech translation
|
||||
- Custom speech models
|
||||
- Batch transcription of many files
|
||||
|
||||
## Reference Files
|
||||
|
||||
| File | Contents |
|
||||
|------|----------|
|
||||
| [references/pronunciation-assessment.md](references/pronunciation-assessment.md) | Pronunciation assessment parameters and scoring |
|
||||
227
skills/official/microsoft/python/foundry/textanalytics/SKILL.md
Normal file
227
skills/official/microsoft/python/foundry/textanalytics/SKILL.md
Normal file
@@ -0,0 +1,227 @@
|
||||
---
|
||||
name: azure-ai-textanalytics-py
|
||||
description: |
|
||||
Azure AI Text Analytics SDK for sentiment analysis, entity recognition, key phrases, language detection, PII, and healthcare NLP. Use for natural language processing on text.
|
||||
Triggers: "text analytics", "sentiment analysis", "entity recognition", "key phrase", "PII detection", "TextAnalyticsClient".
|
||||
package: azure-ai-textanalytics
|
||||
---
|
||||
|
||||
# Azure AI Text Analytics SDK for Python
|
||||
|
||||
Client library for Azure AI Language service NLP capabilities including sentiment, entities, key phrases, and more.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-ai-textanalytics
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_LANGUAGE_ENDPOINT=https://<resource>.cognitiveservices.azure.com
|
||||
AZURE_LANGUAGE_KEY=<your-api-key> # If using API key
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
### API Key
|
||||
|
||||
```python
|
||||
import os
|
||||
from azure.core.credentials import AzureKeyCredential
|
||||
from azure.ai.textanalytics import TextAnalyticsClient
|
||||
|
||||
endpoint = os.environ["AZURE_LANGUAGE_ENDPOINT"]
|
||||
key = os.environ["AZURE_LANGUAGE_KEY"]
|
||||
|
||||
client = TextAnalyticsClient(endpoint, AzureKeyCredential(key))
|
||||
```
|
||||
|
||||
### Entra ID (Recommended)
|
||||
|
||||
```python
|
||||
from azure.ai.textanalytics import TextAnalyticsClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
client = TextAnalyticsClient(
|
||||
endpoint=os.environ["AZURE_LANGUAGE_ENDPOINT"],
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
```
|
||||
|
||||
## Sentiment Analysis
|
||||
|
||||
```python
|
||||
documents = [
|
||||
"I had a wonderful trip to Seattle last week!",
|
||||
"The food was terrible and the service was slow."
|
||||
]
|
||||
|
||||
result = client.analyze_sentiment(documents, show_opinion_mining=True)
|
||||
|
||||
for doc in result:
|
||||
if not doc.is_error:
|
||||
print(f"Sentiment: {doc.sentiment}")
|
||||
print(f"Scores: pos={doc.confidence_scores.positive:.2f}, "
|
||||
f"neg={doc.confidence_scores.negative:.2f}, "
|
||||
f"neu={doc.confidence_scores.neutral:.2f}")
|
||||
|
||||
# Opinion mining (aspect-based sentiment)
|
||||
for sentence in doc.sentences:
|
||||
for opinion in sentence.mined_opinions:
|
||||
target = opinion.target
|
||||
print(f" Target: '{target.text}' - {target.sentiment}")
|
||||
for assessment in opinion.assessments:
|
||||
print(f" Assessment: '{assessment.text}' - {assessment.sentiment}")
|
||||
```
|
||||
|
||||
## Entity Recognition
|
||||
|
||||
```python
|
||||
documents = ["Microsoft was founded by Bill Gates and Paul Allen in Albuquerque."]
|
||||
|
||||
result = client.recognize_entities(documents)
|
||||
|
||||
for doc in result:
|
||||
if not doc.is_error:
|
||||
for entity in doc.entities:
|
||||
print(f"Entity: {entity.text}")
|
||||
print(f" Category: {entity.category}")
|
||||
print(f" Subcategory: {entity.subcategory}")
|
||||
print(f" Confidence: {entity.confidence_score:.2f}")
|
||||
```
|
||||
|
||||
## PII Detection
|
||||
|
||||
```python
|
||||
documents = ["My SSN is 123-45-6789 and my email is john@example.com"]
|
||||
|
||||
result = client.recognize_pii_entities(documents)
|
||||
|
||||
for doc in result:
|
||||
if not doc.is_error:
|
||||
print(f"Redacted: {doc.redacted_text}")
|
||||
for entity in doc.entities:
|
||||
print(f"PII: {entity.text} ({entity.category})")
|
||||
```
|
||||
|
||||
## Key Phrase Extraction
|
||||
|
||||
```python
|
||||
documents = ["Azure AI provides powerful machine learning capabilities for developers."]
|
||||
|
||||
result = client.extract_key_phrases(documents)
|
||||
|
||||
for doc in result:
|
||||
if not doc.is_error:
|
||||
print(f"Key phrases: {doc.key_phrases}")
|
||||
```
|
||||
|
||||
## Language Detection
|
||||
|
||||
```python
|
||||
documents = ["Ce document est en francais.", "This is written in English."]
|
||||
|
||||
result = client.detect_language(documents)
|
||||
|
||||
for doc in result:
|
||||
if not doc.is_error:
|
||||
print(f"Language: {doc.primary_language.name} ({doc.primary_language.iso6391_name})")
|
||||
print(f"Confidence: {doc.primary_language.confidence_score:.2f}")
|
||||
```
|
||||
|
||||
## Healthcare Text Analytics
|
||||
|
||||
```python
|
||||
documents = ["Patient has diabetes and was prescribed metformin 500mg twice daily."]
|
||||
|
||||
poller = client.begin_analyze_healthcare_entities(documents)
|
||||
result = poller.result()
|
||||
|
||||
for doc in result:
|
||||
if not doc.is_error:
|
||||
for entity in doc.entities:
|
||||
print(f"Entity: {entity.text}")
|
||||
print(f" Category: {entity.category}")
|
||||
print(f" Normalized: {entity.normalized_text}")
|
||||
|
||||
# Entity links (UMLS, etc.)
|
||||
for link in entity.data_sources:
|
||||
print(f" Link: {link.name} - {link.entity_id}")
|
||||
```
|
||||
|
||||
## Multiple Analysis (Batch)
|
||||
|
||||
```python
|
||||
from azure.ai.textanalytics import (
|
||||
RecognizeEntitiesAction,
|
||||
ExtractKeyPhrasesAction,
|
||||
AnalyzeSentimentAction
|
||||
)
|
||||
|
||||
documents = ["Microsoft announced new Azure AI features at Build conference."]
|
||||
|
||||
poller = client.begin_analyze_actions(
|
||||
documents,
|
||||
actions=[
|
||||
RecognizeEntitiesAction(),
|
||||
ExtractKeyPhrasesAction(),
|
||||
AnalyzeSentimentAction()
|
||||
]
|
||||
)
|
||||
|
||||
results = poller.result()
|
||||
for doc_results in results:
|
||||
for result in doc_results:
|
||||
if result.kind == "EntityRecognition":
|
||||
print(f"Entities: {[e.text for e in result.entities]}")
|
||||
elif result.kind == "KeyPhraseExtraction":
|
||||
print(f"Key phrases: {result.key_phrases}")
|
||||
elif result.kind == "SentimentAnalysis":
|
||||
print(f"Sentiment: {result.sentiment}")
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.ai.textanalytics.aio import TextAnalyticsClient
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def analyze():
|
||||
async with TextAnalyticsClient(
|
||||
endpoint=endpoint,
|
||||
credential=DefaultAzureCredential()
|
||||
) as client:
|
||||
result = await client.analyze_sentiment(documents)
|
||||
# Process results...
|
||||
```
|
||||
|
||||
## Client Types
|
||||
|
||||
| Client | Purpose |
|
||||
|--------|---------|
|
||||
| `TextAnalyticsClient` | All text analytics operations |
|
||||
| `TextAnalyticsClient` (aio) | Async version |
|
||||
|
||||
## Available Operations
|
||||
|
||||
| Method | Description |
|
||||
|--------|-------------|
|
||||
| `analyze_sentiment` | Sentiment analysis with opinion mining |
|
||||
| `recognize_entities` | Named entity recognition |
|
||||
| `recognize_pii_entities` | PII detection and redaction |
|
||||
| `recognize_linked_entities` | Entity linking to Wikipedia |
|
||||
| `extract_key_phrases` | Key phrase extraction |
|
||||
| `detect_language` | Language detection |
|
||||
| `begin_analyze_healthcare_entities` | Healthcare NLP (long-running) |
|
||||
| `begin_analyze_actions` | Multiple analyses in batch |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use batch operations** for multiple documents (up to 10 per request)
|
||||
2. **Enable opinion mining** for detailed aspect-based sentiment
|
||||
3. **Use async client** for high-throughput scenarios
|
||||
4. **Handle document errors** — results list may contain errors for some docs
|
||||
5. **Specify language** when known to improve accuracy
|
||||
6. **Use context manager** or close client explicitly
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
name: azure-ai-transcription-py
|
||||
description: |
|
||||
Azure AI Transcription SDK for Python. Use for real-time and batch speech-to-text transcription with timestamps and diarization.
|
||||
Triggers: "transcription", "speech to text", "Azure AI Transcription", "TranscriptionClient".
|
||||
package: azure-ai-transcription
|
||||
---
|
||||
|
||||
# Azure AI Transcription SDK for Python
|
||||
|
||||
Client library for Azure AI Transcription (speech-to-text) with real-time and batch transcription.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-ai-transcription
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
TRANSCRIPTION_ENDPOINT=https://<resource>.cognitiveservices.azure.com
|
||||
TRANSCRIPTION_KEY=<your-key>
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
Use subscription key authentication (DefaultAzureCredential is not supported for this client):
|
||||
|
||||
```python
|
||||
import os
|
||||
from azure.ai.transcription import TranscriptionClient
|
||||
|
||||
client = TranscriptionClient(
|
||||
endpoint=os.environ["TRANSCRIPTION_ENDPOINT"],
|
||||
credential=os.environ["TRANSCRIPTION_KEY"]
|
||||
)
|
||||
```
|
||||
|
||||
## Transcription (Batch)
|
||||
|
||||
```python
|
||||
job = client.begin_transcription(
|
||||
name="meeting-transcription",
|
||||
locale="en-US",
|
||||
content_urls=["https://<storage>/audio.wav"],
|
||||
diarization_enabled=True
|
||||
)
|
||||
result = job.result()
|
||||
print(result.status)
|
||||
```
|
||||
|
||||
## Transcription (Real-time)
|
||||
|
||||
```python
|
||||
stream = client.begin_stream_transcription(locale="en-US")
|
||||
stream.send_audio_file("audio.wav")
|
||||
for event in stream:
|
||||
print(event.text)
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Enable diarization** when multiple speakers are present
|
||||
2. **Use batch transcription** for long files stored in blob storage
|
||||
3. **Capture timestamps** for subtitle generation
|
||||
4. **Specify language** to improve recognition accuracy
|
||||
5. **Handle streaming backpressure** for real-time transcription
|
||||
6. **Close transcription sessions** when complete
|
||||
@@ -0,0 +1,249 @@
|
||||
---
|
||||
name: azure-ai-translation-document-py
|
||||
description: |
|
||||
Azure AI Document Translation SDK for batch translation of documents with format preservation. Use for translating Word, PDF, Excel, PowerPoint, and other document formats at scale.
|
||||
Triggers: "document translation", "batch translation", "translate documents", "DocumentTranslationClient".
|
||||
package: azure-ai-translation-document
|
||||
---
|
||||
|
||||
# Azure AI Document Translation SDK for Python
|
||||
|
||||
Client library for Azure AI Translator document translation service for batch document translation with format preservation.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-ai-translation-document
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_DOCUMENT_TRANSLATION_ENDPOINT=https://<resource>.cognitiveservices.azure.com
|
||||
AZURE_DOCUMENT_TRANSLATION_KEY=<your-api-key> # If using API key
|
||||
|
||||
# Storage for source and target documents
|
||||
AZURE_SOURCE_CONTAINER_URL=https://<storage>.blob.core.windows.net/<container>?<sas>
|
||||
AZURE_TARGET_CONTAINER_URL=https://<storage>.blob.core.windows.net/<container>?<sas>
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
### API Key
|
||||
|
||||
```python
|
||||
import os
|
||||
from azure.ai.translation.document import DocumentTranslationClient
|
||||
from azure.core.credentials import AzureKeyCredential
|
||||
|
||||
endpoint = os.environ["AZURE_DOCUMENT_TRANSLATION_ENDPOINT"]
|
||||
key = os.environ["AZURE_DOCUMENT_TRANSLATION_KEY"]
|
||||
|
||||
client = DocumentTranslationClient(endpoint, AzureKeyCredential(key))
|
||||
```
|
||||
|
||||
### Entra ID (Recommended)
|
||||
|
||||
```python
|
||||
from azure.ai.translation.document import DocumentTranslationClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
client = DocumentTranslationClient(
|
||||
endpoint=os.environ["AZURE_DOCUMENT_TRANSLATION_ENDPOINT"],
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
```
|
||||
|
||||
## Basic Document Translation
|
||||
|
||||
```python
|
||||
from azure.ai.translation.document import DocumentTranslationInput, TranslationTarget
|
||||
|
||||
source_url = os.environ["AZURE_SOURCE_CONTAINER_URL"]
|
||||
target_url = os.environ["AZURE_TARGET_CONTAINER_URL"]
|
||||
|
||||
# Start translation job
|
||||
poller = client.begin_translation(
|
||||
inputs=[
|
||||
DocumentTranslationInput(
|
||||
source_url=source_url,
|
||||
targets=[
|
||||
TranslationTarget(
|
||||
target_url=target_url,
|
||||
language="es" # Translate to Spanish
|
||||
)
|
||||
]
|
||||
)
|
||||
]
|
||||
)
|
||||
|
||||
# Wait for completion
|
||||
result = poller.result()
|
||||
|
||||
print(f"Status: {poller.status()}")
|
||||
print(f"Documents translated: {poller.details.documents_succeeded_count}")
|
||||
print(f"Documents failed: {poller.details.documents_failed_count}")
|
||||
```
|
||||
|
||||
## Multiple Target Languages
|
||||
|
||||
```python
|
||||
poller = client.begin_translation(
|
||||
inputs=[
|
||||
DocumentTranslationInput(
|
||||
source_url=source_url,
|
||||
targets=[
|
||||
TranslationTarget(target_url=target_url_es, language="es"),
|
||||
TranslationTarget(target_url=target_url_fr, language="fr"),
|
||||
TranslationTarget(target_url=target_url_de, language="de")
|
||||
]
|
||||
)
|
||||
]
|
||||
)
|
||||
```
|
||||
|
||||
## Translate Single Document
|
||||
|
||||
```python
|
||||
from azure.ai.translation.document import SingleDocumentTranslationClient
|
||||
|
||||
single_client = SingleDocumentTranslationClient(endpoint, AzureKeyCredential(key))
|
||||
|
||||
with open("document.docx", "rb") as f:
|
||||
document_content = f.read()
|
||||
|
||||
result = single_client.translate(
|
||||
body=document_content,
|
||||
target_language="es",
|
||||
content_type="application/vnd.openxmlformats-officedocument.wordprocessingml.document"
|
||||
)
|
||||
|
||||
# Save translated document
|
||||
with open("document_es.docx", "wb") as f:
|
||||
f.write(result)
|
||||
```
|
||||
|
||||
## Check Translation Status
|
||||
|
||||
```python
|
||||
# Get all translation operations
|
||||
operations = client.list_translation_statuses()
|
||||
|
||||
for op in operations:
|
||||
print(f"Operation ID: {op.id}")
|
||||
print(f"Status: {op.status}")
|
||||
print(f"Created: {op.created_on}")
|
||||
print(f"Total documents: {op.documents_total_count}")
|
||||
print(f"Succeeded: {op.documents_succeeded_count}")
|
||||
print(f"Failed: {op.documents_failed_count}")
|
||||
```
|
||||
|
||||
## List Document Statuses
|
||||
|
||||
```python
|
||||
# Get status of individual documents in a job
|
||||
operation_id = poller.id
|
||||
document_statuses = client.list_document_statuses(operation_id)
|
||||
|
||||
for doc in document_statuses:
|
||||
print(f"Document: {doc.source_document_url}")
|
||||
print(f" Status: {doc.status}")
|
||||
print(f" Translated to: {doc.translated_to}")
|
||||
if doc.error:
|
||||
print(f" Error: {doc.error.message}")
|
||||
```
|
||||
|
||||
## Cancel Translation
|
||||
|
||||
```python
|
||||
# Cancel a running translation
|
||||
client.cancel_translation(operation_id)
|
||||
```
|
||||
|
||||
## Using Glossary
|
||||
|
||||
```python
|
||||
from azure.ai.translation.document import TranslationGlossary
|
||||
|
||||
poller = client.begin_translation(
|
||||
inputs=[
|
||||
DocumentTranslationInput(
|
||||
source_url=source_url,
|
||||
targets=[
|
||||
TranslationTarget(
|
||||
target_url=target_url,
|
||||
language="es",
|
||||
glossaries=[
|
||||
TranslationGlossary(
|
||||
glossary_url="https://<storage>.blob.core.windows.net/glossary/terms.csv?<sas>",
|
||||
file_format="csv"
|
||||
)
|
||||
]
|
||||
)
|
||||
]
|
||||
)
|
||||
]
|
||||
)
|
||||
```
|
||||
|
||||
## Supported Document Formats
|
||||
|
||||
```python
|
||||
# Get supported formats
|
||||
formats = client.get_supported_document_formats()
|
||||
|
||||
for fmt in formats:
|
||||
print(f"Format: {fmt.format}")
|
||||
print(f" Extensions: {fmt.file_extensions}")
|
||||
print(f" Content types: {fmt.content_types}")
|
||||
```
|
||||
|
||||
## Supported Languages
|
||||
|
||||
```python
|
||||
# Get supported languages
|
||||
languages = client.get_supported_languages()
|
||||
|
||||
for lang in languages:
|
||||
print(f"Language: {lang.name} ({lang.code})")
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.ai.translation.document.aio import DocumentTranslationClient
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def translate_documents():
|
||||
async with DocumentTranslationClient(
|
||||
endpoint=endpoint,
|
||||
credential=DefaultAzureCredential()
|
||||
) as client:
|
||||
poller = await client.begin_translation(inputs=[...])
|
||||
result = await poller.result()
|
||||
```
|
||||
|
||||
## Supported Formats
|
||||
|
||||
| Category | Formats |
|
||||
|----------|---------|
|
||||
| Documents | DOCX, PDF, PPTX, XLSX, HTML, TXT, RTF |
|
||||
| Structured | CSV, TSV, JSON, XML |
|
||||
| Localization | XLIFF, XLF, MHTML |
|
||||
|
||||
## Storage Requirements
|
||||
|
||||
- Source and target containers must be Azure Blob Storage
|
||||
- Use SAS tokens with appropriate permissions:
|
||||
- Source: Read, List
|
||||
- Target: Write, List
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use SAS tokens** with minimal required permissions
|
||||
2. **Monitor long-running operations** with `poller.status()`
|
||||
3. **Handle document-level errors** by iterating document statuses
|
||||
4. **Use glossaries** for domain-specific terminology
|
||||
5. **Separate target containers** for each language
|
||||
6. **Use async client** for multiple concurrent jobs
|
||||
7. **Check supported formats** before submitting documents
|
||||
@@ -0,0 +1,274 @@
|
||||
---
|
||||
name: azure-ai-translation-text-py
|
||||
description: |
|
||||
Azure AI Text Translation SDK for real-time text translation, transliteration, language detection, and dictionary lookup. Use for translating text content in applications.
|
||||
Triggers: "text translation", "translator", "translate text", "transliterate", "TextTranslationClient".
|
||||
package: azure-ai-translation-text
|
||||
---
|
||||
|
||||
# Azure AI Text Translation SDK for Python
|
||||
|
||||
Client library for Azure AI Translator text translation service for real-time text translation, transliteration, and language operations.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-ai-translation-text
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_TRANSLATOR_KEY=<your-api-key>
|
||||
AZURE_TRANSLATOR_REGION=<your-region> # e.g., eastus, westus2
|
||||
# Or use custom endpoint
|
||||
AZURE_TRANSLATOR_ENDPOINT=https://<resource>.cognitiveservices.azure.com
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
### API Key with Region
|
||||
|
||||
```python
|
||||
import os
|
||||
from azure.ai.translation.text import TextTranslationClient
|
||||
from azure.core.credentials import AzureKeyCredential
|
||||
|
||||
key = os.environ["AZURE_TRANSLATOR_KEY"]
|
||||
region = os.environ["AZURE_TRANSLATOR_REGION"]
|
||||
|
||||
# Create credential with region
|
||||
credential = AzureKeyCredential(key)
|
||||
client = TextTranslationClient(credential=credential, region=region)
|
||||
```
|
||||
|
||||
### API Key with Custom Endpoint
|
||||
|
||||
```python
|
||||
endpoint = os.environ["AZURE_TRANSLATOR_ENDPOINT"]
|
||||
|
||||
client = TextTranslationClient(
|
||||
credential=AzureKeyCredential(key),
|
||||
endpoint=endpoint
|
||||
)
|
||||
```
|
||||
|
||||
### Entra ID (Recommended)
|
||||
|
||||
```python
|
||||
from azure.ai.translation.text import TextTranslationClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
client = TextTranslationClient(
|
||||
credential=DefaultAzureCredential(),
|
||||
endpoint=os.environ["AZURE_TRANSLATOR_ENDPOINT"]
|
||||
)
|
||||
```
|
||||
|
||||
## Basic Translation
|
||||
|
||||
```python
|
||||
# Translate to a single language
|
||||
result = client.translate(
|
||||
body=["Hello, how are you?", "Welcome to Azure!"],
|
||||
to=["es"] # Spanish
|
||||
)
|
||||
|
||||
for item in result:
|
||||
for translation in item.translations:
|
||||
print(f"Translated: {translation.text}")
|
||||
print(f"Target language: {translation.to}")
|
||||
```
|
||||
|
||||
## Translate to Multiple Languages
|
||||
|
||||
```python
|
||||
result = client.translate(
|
||||
body=["Hello, world!"],
|
||||
to=["es", "fr", "de", "ja"] # Spanish, French, German, Japanese
|
||||
)
|
||||
|
||||
for item in result:
|
||||
print(f"Source: {item.detected_language.language if item.detected_language else 'unknown'}")
|
||||
for translation in item.translations:
|
||||
print(f" {translation.to}: {translation.text}")
|
||||
```
|
||||
|
||||
## Specify Source Language
|
||||
|
||||
```python
|
||||
result = client.translate(
|
||||
body=["Bonjour le monde"],
|
||||
from_parameter="fr", # Source is French
|
||||
to=["en", "es"]
|
||||
)
|
||||
```
|
||||
|
||||
## Language Detection
|
||||
|
||||
```python
|
||||
result = client.translate(
|
||||
body=["Hola, como estas?"],
|
||||
to=["en"]
|
||||
)
|
||||
|
||||
for item in result:
|
||||
if item.detected_language:
|
||||
print(f"Detected language: {item.detected_language.language}")
|
||||
print(f"Confidence: {item.detected_language.score:.2f}")
|
||||
```
|
||||
|
||||
## Transliteration
|
||||
|
||||
Convert text from one script to another:
|
||||
|
||||
```python
|
||||
result = client.transliterate(
|
||||
body=["konnichiwa"],
|
||||
language="ja",
|
||||
from_script="Latn", # From Latin script
|
||||
to_script="Jpan" # To Japanese script
|
||||
)
|
||||
|
||||
for item in result:
|
||||
print(f"Transliterated: {item.text}")
|
||||
print(f"Script: {item.script}")
|
||||
```
|
||||
|
||||
## Dictionary Lookup
|
||||
|
||||
Find alternate translations and definitions:
|
||||
|
||||
```python
|
||||
result = client.lookup_dictionary_entries(
|
||||
body=["fly"],
|
||||
from_parameter="en",
|
||||
to="es"
|
||||
)
|
||||
|
||||
for item in result:
|
||||
print(f"Source: {item.normalized_source} ({item.display_source})")
|
||||
for translation in item.translations:
|
||||
print(f" Translation: {translation.normalized_target}")
|
||||
print(f" Part of speech: {translation.pos_tag}")
|
||||
print(f" Confidence: {translation.confidence:.2f}")
|
||||
```
|
||||
|
||||
## Dictionary Examples
|
||||
|
||||
Get usage examples for translations:
|
||||
|
||||
```python
|
||||
from azure.ai.translation.text.models import DictionaryExampleTextItem
|
||||
|
||||
result = client.lookup_dictionary_examples(
|
||||
body=[DictionaryExampleTextItem(text="fly", translation="volar")],
|
||||
from_parameter="en",
|
||||
to="es"
|
||||
)
|
||||
|
||||
for item in result:
|
||||
for example in item.examples:
|
||||
print(f"Source: {example.source_prefix}{example.source_term}{example.source_suffix}")
|
||||
print(f"Target: {example.target_prefix}{example.target_term}{example.target_suffix}")
|
||||
```
|
||||
|
||||
## Get Supported Languages
|
||||
|
||||
```python
|
||||
# Get all supported languages
|
||||
languages = client.get_supported_languages()
|
||||
|
||||
# Translation languages
|
||||
print("Translation languages:")
|
||||
for code, lang in languages.translation.items():
|
||||
print(f" {code}: {lang.name} ({lang.native_name})")
|
||||
|
||||
# Transliteration languages
|
||||
print("\nTransliteration languages:")
|
||||
for code, lang in languages.transliteration.items():
|
||||
print(f" {code}: {lang.name}")
|
||||
for script in lang.scripts:
|
||||
print(f" {script.code} -> {[t.code for t in script.to_scripts]}")
|
||||
|
||||
# Dictionary languages
|
||||
print("\nDictionary languages:")
|
||||
for code, lang in languages.dictionary.items():
|
||||
print(f" {code}: {lang.name}")
|
||||
```
|
||||
|
||||
## Break Sentence
|
||||
|
||||
Identify sentence boundaries:
|
||||
|
||||
```python
|
||||
result = client.find_sentence_boundaries(
|
||||
body=["Hello! How are you? I hope you are well."],
|
||||
language="en"
|
||||
)
|
||||
|
||||
for item in result:
|
||||
print(f"Sentence lengths: {item.sent_len}")
|
||||
```
|
||||
|
||||
## Translation Options
|
||||
|
||||
```python
|
||||
result = client.translate(
|
||||
body=["Hello, world!"],
|
||||
to=["de"],
|
||||
text_type="html", # "plain" or "html"
|
||||
profanity_action="Marked", # "NoAction", "Deleted", "Marked"
|
||||
profanity_marker="Asterisk", # "Asterisk", "Tag"
|
||||
include_alignment=True, # Include word alignment
|
||||
include_sentence_length=True # Include sentence boundaries
|
||||
)
|
||||
|
||||
for item in result:
|
||||
translation = item.translations[0]
|
||||
print(f"Translated: {translation.text}")
|
||||
if translation.alignment:
|
||||
print(f"Alignment: {translation.alignment.proj}")
|
||||
if translation.sent_len:
|
||||
print(f"Sentence lengths: {translation.sent_len.src_sent_len}")
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.ai.translation.text.aio import TextTranslationClient
|
||||
from azure.core.credentials import AzureKeyCredential
|
||||
|
||||
async def translate_text():
|
||||
async with TextTranslationClient(
|
||||
credential=AzureKeyCredential(key),
|
||||
region=region
|
||||
) as client:
|
||||
result = await client.translate(
|
||||
body=["Hello, world!"],
|
||||
to=["es"]
|
||||
)
|
||||
print(result[0].translations[0].text)
|
||||
```
|
||||
|
||||
## Client Methods
|
||||
|
||||
| Method | Description |
|
||||
|--------|-------------|
|
||||
| `translate` | Translate text to one or more languages |
|
||||
| `transliterate` | Convert text between scripts |
|
||||
| `detect` | Detect language of text |
|
||||
| `find_sentence_boundaries` | Identify sentence boundaries |
|
||||
| `lookup_dictionary_entries` | Dictionary lookup for translations |
|
||||
| `lookup_dictionary_examples` | Get usage examples |
|
||||
| `get_supported_languages` | List supported languages |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Batch translations** — Send multiple texts in one request (up to 100)
|
||||
2. **Specify source language** when known to improve accuracy
|
||||
3. **Use async client** for high-throughput scenarios
|
||||
4. **Cache language list** — Supported languages don't change frequently
|
||||
5. **Handle profanity** appropriately for your application
|
||||
6. **Use html text_type** when translating HTML content
|
||||
7. **Include alignment** for applications needing word mapping
|
||||
@@ -0,0 +1,260 @@
|
||||
---
|
||||
name: azure-ai-vision-imageanalysis-py
|
||||
description: |
|
||||
Azure AI Vision Image Analysis SDK for captions, tags, objects, OCR, people detection, and smart cropping. Use for computer vision and image understanding tasks.
|
||||
Triggers: "image analysis", "computer vision", "OCR", "object detection", "ImageAnalysisClient", "image caption".
|
||||
package: azure-ai-vision-imageanalysis
|
||||
---
|
||||
|
||||
# Azure AI Vision Image Analysis SDK for Python
|
||||
|
||||
Client library for Azure AI Vision 4.0 image analysis including captions, tags, objects, OCR, and more.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-ai-vision-imageanalysis
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
VISION_ENDPOINT=https://<resource>.cognitiveservices.azure.com
|
||||
VISION_KEY=<your-api-key> # If using API key
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
### API Key
|
||||
|
||||
```python
|
||||
import os
|
||||
from azure.ai.vision.imageanalysis import ImageAnalysisClient
|
||||
from azure.core.credentials import AzureKeyCredential
|
||||
|
||||
endpoint = os.environ["VISION_ENDPOINT"]
|
||||
key = os.environ["VISION_KEY"]
|
||||
|
||||
client = ImageAnalysisClient(
|
||||
endpoint=endpoint,
|
||||
credential=AzureKeyCredential(key)
|
||||
)
|
||||
```
|
||||
|
||||
### Entra ID (Recommended)
|
||||
|
||||
```python
|
||||
from azure.ai.vision.imageanalysis import ImageAnalysisClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
client = ImageAnalysisClient(
|
||||
endpoint=os.environ["VISION_ENDPOINT"],
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
```
|
||||
|
||||
## Analyze Image from URL
|
||||
|
||||
```python
|
||||
from azure.ai.vision.imageanalysis.models import VisualFeatures
|
||||
|
||||
image_url = "https://example.com/image.jpg"
|
||||
|
||||
result = client.analyze_from_url(
|
||||
image_url=image_url,
|
||||
visual_features=[
|
||||
VisualFeatures.CAPTION,
|
||||
VisualFeatures.TAGS,
|
||||
VisualFeatures.OBJECTS,
|
||||
VisualFeatures.READ,
|
||||
VisualFeatures.PEOPLE,
|
||||
VisualFeatures.SMART_CROPS,
|
||||
VisualFeatures.DENSE_CAPTIONS
|
||||
],
|
||||
gender_neutral_caption=True,
|
||||
language="en"
|
||||
)
|
||||
```
|
||||
|
||||
## Analyze Image from File
|
||||
|
||||
```python
|
||||
with open("image.jpg", "rb") as f:
|
||||
image_data = f.read()
|
||||
|
||||
result = client.analyze(
|
||||
image_data=image_data,
|
||||
visual_features=[VisualFeatures.CAPTION, VisualFeatures.TAGS]
|
||||
)
|
||||
```
|
||||
|
||||
## Image Caption
|
||||
|
||||
```python
|
||||
result = client.analyze_from_url(
|
||||
image_url=image_url,
|
||||
visual_features=[VisualFeatures.CAPTION],
|
||||
gender_neutral_caption=True
|
||||
)
|
||||
|
||||
if result.caption:
|
||||
print(f"Caption: {result.caption.text}")
|
||||
print(f"Confidence: {result.caption.confidence:.2f}")
|
||||
```
|
||||
|
||||
## Dense Captions (Multiple Regions)
|
||||
|
||||
```python
|
||||
result = client.analyze_from_url(
|
||||
image_url=image_url,
|
||||
visual_features=[VisualFeatures.DENSE_CAPTIONS]
|
||||
)
|
||||
|
||||
if result.dense_captions:
|
||||
for caption in result.dense_captions.list:
|
||||
print(f"Caption: {caption.text}")
|
||||
print(f" Confidence: {caption.confidence:.2f}")
|
||||
print(f" Bounding box: {caption.bounding_box}")
|
||||
```
|
||||
|
||||
## Tags
|
||||
|
||||
```python
|
||||
result = client.analyze_from_url(
|
||||
image_url=image_url,
|
||||
visual_features=[VisualFeatures.TAGS]
|
||||
)
|
||||
|
||||
if result.tags:
|
||||
for tag in result.tags.list:
|
||||
print(f"Tag: {tag.name} (confidence: {tag.confidence:.2f})")
|
||||
```
|
||||
|
||||
## Object Detection
|
||||
|
||||
```python
|
||||
result = client.analyze_from_url(
|
||||
image_url=image_url,
|
||||
visual_features=[VisualFeatures.OBJECTS]
|
||||
)
|
||||
|
||||
if result.objects:
|
||||
for obj in result.objects.list:
|
||||
print(f"Object: {obj.tags[0].name}")
|
||||
print(f" Confidence: {obj.tags[0].confidence:.2f}")
|
||||
box = obj.bounding_box
|
||||
print(f" Bounding box: x={box.x}, y={box.y}, w={box.width}, h={box.height}")
|
||||
```
|
||||
|
||||
## OCR (Text Extraction)
|
||||
|
||||
```python
|
||||
result = client.analyze_from_url(
|
||||
image_url=image_url,
|
||||
visual_features=[VisualFeatures.READ]
|
||||
)
|
||||
|
||||
if result.read:
|
||||
for block in result.read.blocks:
|
||||
for line in block.lines:
|
||||
print(f"Line: {line.text}")
|
||||
print(f" Bounding polygon: {line.bounding_polygon}")
|
||||
|
||||
# Word-level details
|
||||
for word in line.words:
|
||||
print(f" Word: {word.text} (confidence: {word.confidence:.2f})")
|
||||
```
|
||||
|
||||
## People Detection
|
||||
|
||||
```python
|
||||
result = client.analyze_from_url(
|
||||
image_url=image_url,
|
||||
visual_features=[VisualFeatures.PEOPLE]
|
||||
)
|
||||
|
||||
if result.people:
|
||||
for person in result.people.list:
|
||||
print(f"Person detected:")
|
||||
print(f" Confidence: {person.confidence:.2f}")
|
||||
box = person.bounding_box
|
||||
print(f" Bounding box: x={box.x}, y={box.y}, w={box.width}, h={box.height}")
|
||||
```
|
||||
|
||||
## Smart Cropping
|
||||
|
||||
```python
|
||||
result = client.analyze_from_url(
|
||||
image_url=image_url,
|
||||
visual_features=[VisualFeatures.SMART_CROPS],
|
||||
smart_crops_aspect_ratios=[0.9, 1.33, 1.78] # Portrait, 4:3, 16:9
|
||||
)
|
||||
|
||||
if result.smart_crops:
|
||||
for crop in result.smart_crops.list:
|
||||
print(f"Aspect ratio: {crop.aspect_ratio}")
|
||||
box = crop.bounding_box
|
||||
print(f" Crop region: x={box.x}, y={box.y}, w={box.width}, h={box.height}")
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.ai.vision.imageanalysis.aio import ImageAnalysisClient
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def analyze_image():
|
||||
async with ImageAnalysisClient(
|
||||
endpoint=endpoint,
|
||||
credential=DefaultAzureCredential()
|
||||
) as client:
|
||||
result = await client.analyze_from_url(
|
||||
image_url=image_url,
|
||||
visual_features=[VisualFeatures.CAPTION]
|
||||
)
|
||||
print(result.caption.text)
|
||||
```
|
||||
|
||||
## Visual Features
|
||||
|
||||
| Feature | Description |
|
||||
|---------|-------------|
|
||||
| `CAPTION` | Single sentence describing the image |
|
||||
| `DENSE_CAPTIONS` | Captions for multiple regions |
|
||||
| `TAGS` | Content tags (objects, scenes, actions) |
|
||||
| `OBJECTS` | Object detection with bounding boxes |
|
||||
| `READ` | OCR text extraction |
|
||||
| `PEOPLE` | People detection with bounding boxes |
|
||||
| `SMART_CROPS` | Suggested crop regions for thumbnails |
|
||||
|
||||
## Error Handling
|
||||
|
||||
```python
|
||||
from azure.core.exceptions import HttpResponseError
|
||||
|
||||
try:
|
||||
result = client.analyze_from_url(
|
||||
image_url=image_url,
|
||||
visual_features=[VisualFeatures.CAPTION]
|
||||
)
|
||||
except HttpResponseError as e:
|
||||
print(f"Status code: {e.status_code}")
|
||||
print(f"Reason: {e.reason}")
|
||||
print(f"Message: {e.error.message}")
|
||||
```
|
||||
|
||||
## Image Requirements
|
||||
|
||||
- Formats: JPEG, PNG, GIF, BMP, WEBP, ICO, TIFF, MPO
|
||||
- Max size: 20 MB
|
||||
- Dimensions: 50x50 to 16000x16000 pixels
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Select only needed features** to optimize latency and cost
|
||||
2. **Use async client** for high-throughput scenarios
|
||||
3. **Handle HttpResponseError** for invalid images or auth issues
|
||||
4. **Enable gender_neutral_caption** for inclusive descriptions
|
||||
5. **Specify language** for localized captions
|
||||
6. **Use smart_crops_aspect_ratios** matching your thumbnail requirements
|
||||
7. **Cache results** when analyzing the same image multiple times
|
||||
309
skills/official/microsoft/python/foundry/voicelive/SKILL.md
Normal file
309
skills/official/microsoft/python/foundry/voicelive/SKILL.md
Normal file
@@ -0,0 +1,309 @@
|
||||
---
|
||||
name: azure-ai-voicelive-py
|
||||
description: Build real-time voice AI applications using Azure AI Voice Live SDK (azure-ai-voicelive). Use this skill when creating Python applications that need real-time bidirectional audio communication with Azure AI, including voice assistants, voice-enabled chatbots, real-time speech-to-speech translation, voice-driven avatars, or any WebSocket-based audio streaming with AI models. Supports Server VAD (Voice Activity Detection), turn-based conversation, function calling, MCP tools, avatar integration, and transcription.
|
||||
package: azure-ai-voicelive
|
||||
---
|
||||
|
||||
# Azure AI Voice Live SDK
|
||||
|
||||
Build real-time voice AI applications with bidirectional WebSocket communication.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-ai-voicelive aiohttp azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_COGNITIVE_SERVICES_ENDPOINT=https://<region>.api.cognitive.microsoft.com
|
||||
# For API key auth (not recommended for production)
|
||||
AZURE_COGNITIVE_SERVICES_KEY=<api-key>
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
**DefaultAzureCredential (preferred)**:
|
||||
```python
|
||||
from azure.ai.voicelive.aio import connect
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async with connect(
|
||||
endpoint=os.environ["AZURE_COGNITIVE_SERVICES_ENDPOINT"],
|
||||
credential=DefaultAzureCredential(),
|
||||
model="gpt-4o-realtime-preview",
|
||||
credential_scopes=["https://cognitiveservices.azure.com/.default"]
|
||||
) as conn:
|
||||
...
|
||||
```
|
||||
|
||||
**API Key**:
|
||||
```python
|
||||
from azure.ai.voicelive.aio import connect
|
||||
from azure.core.credentials import AzureKeyCredential
|
||||
|
||||
async with connect(
|
||||
endpoint=os.environ["AZURE_COGNITIVE_SERVICES_ENDPOINT"],
|
||||
credential=AzureKeyCredential(os.environ["AZURE_COGNITIVE_SERVICES_KEY"]),
|
||||
model="gpt-4o-realtime-preview"
|
||||
) as conn:
|
||||
...
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
import os
|
||||
from azure.ai.voicelive.aio import connect
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def main():
|
||||
async with connect(
|
||||
endpoint=os.environ["AZURE_COGNITIVE_SERVICES_ENDPOINT"],
|
||||
credential=DefaultAzureCredential(),
|
||||
model="gpt-4o-realtime-preview",
|
||||
credential_scopes=["https://cognitiveservices.azure.com/.default"]
|
||||
) as conn:
|
||||
# Update session with instructions
|
||||
await conn.session.update(session={
|
||||
"instructions": "You are a helpful assistant.",
|
||||
"modalities": ["text", "audio"],
|
||||
"voice": "alloy"
|
||||
})
|
||||
|
||||
# Listen for events
|
||||
async for event in conn:
|
||||
print(f"Event: {event.type}")
|
||||
if event.type == "response.audio_transcript.done":
|
||||
print(f"Transcript: {event.transcript}")
|
||||
elif event.type == "response.done":
|
||||
break
|
||||
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
## Core Architecture
|
||||
|
||||
### Connection Resources
|
||||
|
||||
The `VoiceLiveConnection` exposes these resources:
|
||||
|
||||
| Resource | Purpose | Key Methods |
|
||||
|----------|---------|-------------|
|
||||
| `conn.session` | Session configuration | `update(session=...)` |
|
||||
| `conn.response` | Model responses | `create()`, `cancel()` |
|
||||
| `conn.input_audio_buffer` | Audio input | `append()`, `commit()`, `clear()` |
|
||||
| `conn.output_audio_buffer` | Audio output | `clear()` |
|
||||
| `conn.conversation` | Conversation state | `item.create()`, `item.delete()`, `item.truncate()` |
|
||||
| `conn.transcription_session` | Transcription config | `update(session=...)` |
|
||||
|
||||
## Session Configuration
|
||||
|
||||
```python
|
||||
from azure.ai.voicelive.models import RequestSession, FunctionTool
|
||||
|
||||
await conn.session.update(session=RequestSession(
|
||||
instructions="You are a helpful voice assistant.",
|
||||
modalities=["text", "audio"],
|
||||
voice="alloy", # or "echo", "shimmer", "sage", etc.
|
||||
input_audio_format="pcm16",
|
||||
output_audio_format="pcm16",
|
||||
turn_detection={
|
||||
"type": "server_vad",
|
||||
"threshold": 0.5,
|
||||
"prefix_padding_ms": 300,
|
||||
"silence_duration_ms": 500
|
||||
},
|
||||
tools=[
|
||||
FunctionTool(
|
||||
type="function",
|
||||
name="get_weather",
|
||||
description="Get current weather",
|
||||
parameters={
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"location": {"type": "string"}
|
||||
},
|
||||
"required": ["location"]
|
||||
}
|
||||
)
|
||||
]
|
||||
))
|
||||
```
|
||||
|
||||
## Audio Streaming
|
||||
|
||||
### Send Audio (Base64 PCM16)
|
||||
|
||||
```python
|
||||
import base64
|
||||
|
||||
# Read audio chunk (16-bit PCM, 24kHz mono)
|
||||
audio_chunk = await read_audio_from_microphone()
|
||||
b64_audio = base64.b64encode(audio_chunk).decode()
|
||||
|
||||
await conn.input_audio_buffer.append(audio=b64_audio)
|
||||
```
|
||||
|
||||
### Receive Audio
|
||||
|
||||
```python
|
||||
async for event in conn:
|
||||
if event.type == "response.audio.delta":
|
||||
audio_bytes = base64.b64decode(event.delta)
|
||||
await play_audio(audio_bytes)
|
||||
elif event.type == "response.audio.done":
|
||||
print("Audio complete")
|
||||
```
|
||||
|
||||
## Event Handling
|
||||
|
||||
```python
|
||||
async for event in conn:
|
||||
match event.type:
|
||||
# Session events
|
||||
case "session.created":
|
||||
print(f"Session: {event.session}")
|
||||
case "session.updated":
|
||||
print("Session updated")
|
||||
|
||||
# Audio input events
|
||||
case "input_audio_buffer.speech_started":
|
||||
print(f"Speech started at {event.audio_start_ms}ms")
|
||||
case "input_audio_buffer.speech_stopped":
|
||||
print(f"Speech stopped at {event.audio_end_ms}ms")
|
||||
|
||||
# Transcription events
|
||||
case "conversation.item.input_audio_transcription.completed":
|
||||
print(f"User said: {event.transcript}")
|
||||
case "conversation.item.input_audio_transcription.delta":
|
||||
print(f"Partial: {event.delta}")
|
||||
|
||||
# Response events
|
||||
case "response.created":
|
||||
print(f"Response started: {event.response.id}")
|
||||
case "response.audio_transcript.delta":
|
||||
print(event.delta, end="", flush=True)
|
||||
case "response.audio.delta":
|
||||
audio = base64.b64decode(event.delta)
|
||||
case "response.done":
|
||||
print(f"Response complete: {event.response.status}")
|
||||
|
||||
# Function calls
|
||||
case "response.function_call_arguments.done":
|
||||
result = handle_function(event.name, event.arguments)
|
||||
await conn.conversation.item.create(item={
|
||||
"type": "function_call_output",
|
||||
"call_id": event.call_id,
|
||||
"output": json.dumps(result)
|
||||
})
|
||||
await conn.response.create()
|
||||
|
||||
# Errors
|
||||
case "error":
|
||||
print(f"Error: {event.error.message}")
|
||||
```
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Manual Turn Mode (No VAD)
|
||||
|
||||
```python
|
||||
await conn.session.update(session={"turn_detection": None})
|
||||
|
||||
# Manually control turns
|
||||
await conn.input_audio_buffer.append(audio=b64_audio)
|
||||
await conn.input_audio_buffer.commit() # End of user turn
|
||||
await conn.response.create() # Trigger response
|
||||
```
|
||||
|
||||
### Interrupt Handling
|
||||
|
||||
```python
|
||||
async for event in conn:
|
||||
if event.type == "input_audio_buffer.speech_started":
|
||||
# User interrupted - cancel current response
|
||||
await conn.response.cancel()
|
||||
await conn.output_audio_buffer.clear()
|
||||
```
|
||||
|
||||
### Conversation History
|
||||
|
||||
```python
|
||||
# Add system message
|
||||
await conn.conversation.item.create(item={
|
||||
"type": "message",
|
||||
"role": "system",
|
||||
"content": [{"type": "input_text", "text": "Be concise."}]
|
||||
})
|
||||
|
||||
# Add user message
|
||||
await conn.conversation.item.create(item={
|
||||
"type": "message",
|
||||
"role": "user",
|
||||
"content": [{"type": "input_text", "text": "Hello!"}]
|
||||
})
|
||||
|
||||
await conn.response.create()
|
||||
```
|
||||
|
||||
## Voice Options
|
||||
|
||||
| Voice | Description |
|
||||
|-------|-------------|
|
||||
| `alloy` | Neutral, balanced |
|
||||
| `echo` | Warm, conversational |
|
||||
| `shimmer` | Clear, professional |
|
||||
| `sage` | Calm, authoritative |
|
||||
| `coral` | Friendly, upbeat |
|
||||
| `ash` | Deep, measured |
|
||||
| `ballad` | Expressive |
|
||||
| `verse` | Storytelling |
|
||||
|
||||
Azure voices: Use `AzureStandardVoice`, `AzureCustomVoice`, or `AzurePersonalVoice` models.
|
||||
|
||||
## Audio Formats
|
||||
|
||||
| Format | Sample Rate | Use Case |
|
||||
|--------|-------------|----------|
|
||||
| `pcm16` | 24kHz | Default, high quality |
|
||||
| `pcm16-8000hz` | 8kHz | Telephony |
|
||||
| `pcm16-16000hz` | 16kHz | Voice assistants |
|
||||
| `g711_ulaw` | 8kHz | Telephony (US) |
|
||||
| `g711_alaw` | 8kHz | Telephony (EU) |
|
||||
|
||||
## Turn Detection Options
|
||||
|
||||
```python
|
||||
# Server VAD (default)
|
||||
{"type": "server_vad", "threshold": 0.5, "silence_duration_ms": 500}
|
||||
|
||||
# Azure Semantic VAD (smarter detection)
|
||||
{"type": "azure_semantic_vad"}
|
||||
{"type": "azure_semantic_vad_en"} # English optimized
|
||||
{"type": "azure_semantic_vad_multilingual"}
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
```python
|
||||
from azure.ai.voicelive.aio import ConnectionError, ConnectionClosed
|
||||
|
||||
try:
|
||||
async with connect(...) as conn:
|
||||
async for event in conn:
|
||||
if event.type == "error":
|
||||
print(f"API Error: {event.error.code} - {event.error.message}")
|
||||
except ConnectionClosed as e:
|
||||
print(f"Connection closed: {e.code} - {e.reason}")
|
||||
except ConnectionError as e:
|
||||
print(f"Connection error: {e}")
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
- **Detailed API Reference**: See [references/api-reference.md](references/api-reference.md)
|
||||
- **Complete Examples**: See [references/examples.md](references/examples.md)
|
||||
- **All Models & Types**: See [references/models.md](references/models.md)
|
||||
242
skills/official/microsoft/python/integration/apicenter/SKILL.md
Normal file
242
skills/official/microsoft/python/integration/apicenter/SKILL.md
Normal file
@@ -0,0 +1,242 @@
|
||||
---
|
||||
name: azure-mgmt-apicenter-py
|
||||
description: |
|
||||
Azure API Center Management SDK for Python. Use for managing API inventory, metadata, and governance across your organization.
|
||||
Triggers: "azure-mgmt-apicenter", "ApiCenterMgmtClient", "API Center", "API inventory", "API governance".
|
||||
package: azure-mgmt-apicenter
|
||||
---
|
||||
|
||||
# Azure API Center Management SDK for Python
|
||||
|
||||
Manage API inventory, metadata, and governance in Azure API Center.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-mgmt-apicenter
|
||||
pip install azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_SUBSCRIPTION_ID=your-subscription-id
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.mgmt.apicenter import ApiCenterMgmtClient
|
||||
import os
|
||||
|
||||
client = ApiCenterMgmtClient(
|
||||
credential=DefaultAzureCredential(),
|
||||
subscription_id=os.environ["AZURE_SUBSCRIPTION_ID"]
|
||||
)
|
||||
```
|
||||
|
||||
## Create API Center
|
||||
|
||||
```python
|
||||
from azure.mgmt.apicenter.models import Service
|
||||
|
||||
api_center = client.services.create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-api-center",
|
||||
resource=Service(
|
||||
location="eastus",
|
||||
tags={"environment": "production"}
|
||||
)
|
||||
)
|
||||
|
||||
print(f"Created API Center: {api_center.name}")
|
||||
```
|
||||
|
||||
## List API Centers
|
||||
|
||||
```python
|
||||
api_centers = client.services.list_by_subscription()
|
||||
|
||||
for api_center in api_centers:
|
||||
print(f"{api_center.name} - {api_center.location}")
|
||||
```
|
||||
|
||||
## Register an API
|
||||
|
||||
```python
|
||||
from azure.mgmt.apicenter.models import Api, ApiKind, LifecycleStage
|
||||
|
||||
api = client.apis.create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-api-center",
|
||||
workspace_name="default",
|
||||
api_name="my-api",
|
||||
resource=Api(
|
||||
title="My API",
|
||||
description="A sample API for demonstration",
|
||||
kind=ApiKind.REST,
|
||||
lifecycle_stage=LifecycleStage.PRODUCTION,
|
||||
terms_of_service={"url": "https://example.com/terms"},
|
||||
contacts=[{"name": "API Team", "email": "api-team@example.com"}]
|
||||
)
|
||||
)
|
||||
|
||||
print(f"Registered API: {api.title}")
|
||||
```
|
||||
|
||||
## Create API Version
|
||||
|
||||
```python
|
||||
from azure.mgmt.apicenter.models import ApiVersion, LifecycleStage
|
||||
|
||||
version = client.api_versions.create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-api-center",
|
||||
workspace_name="default",
|
||||
api_name="my-api",
|
||||
version_name="v1",
|
||||
resource=ApiVersion(
|
||||
title="Version 1.0",
|
||||
lifecycle_stage=LifecycleStage.PRODUCTION
|
||||
)
|
||||
)
|
||||
|
||||
print(f"Created version: {version.title}")
|
||||
```
|
||||
|
||||
## Add API Definition
|
||||
|
||||
```python
|
||||
from azure.mgmt.apicenter.models import ApiDefinition
|
||||
|
||||
definition = client.api_definitions.create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-api-center",
|
||||
workspace_name="default",
|
||||
api_name="my-api",
|
||||
version_name="v1",
|
||||
definition_name="openapi",
|
||||
resource=ApiDefinition(
|
||||
title="OpenAPI Definition",
|
||||
description="OpenAPI 3.0 specification"
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
## Import API Specification
|
||||
|
||||
```python
|
||||
from azure.mgmt.apicenter.models import ApiSpecImportRequest, ApiSpecImportSourceFormat
|
||||
|
||||
# Import from inline content
|
||||
client.api_definitions.import_specification(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-api-center",
|
||||
workspace_name="default",
|
||||
api_name="my-api",
|
||||
version_name="v1",
|
||||
definition_name="openapi",
|
||||
body=ApiSpecImportRequest(
|
||||
format=ApiSpecImportSourceFormat.INLINE,
|
||||
value='{"openapi": "3.0.0", "info": {"title": "My API", "version": "1.0"}, "paths": {}}'
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
## List APIs
|
||||
|
||||
```python
|
||||
apis = client.apis.list(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-api-center",
|
||||
workspace_name="default"
|
||||
)
|
||||
|
||||
for api in apis:
|
||||
print(f"{api.name}: {api.title} ({api.kind})")
|
||||
```
|
||||
|
||||
## Create Environment
|
||||
|
||||
```python
|
||||
from azure.mgmt.apicenter.models import Environment, EnvironmentKind
|
||||
|
||||
environment = client.environments.create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-api-center",
|
||||
workspace_name="default",
|
||||
environment_name="production",
|
||||
resource=Environment(
|
||||
title="Production",
|
||||
description="Production environment",
|
||||
kind=EnvironmentKind.PRODUCTION,
|
||||
server={"type": "Azure API Management", "management_portal_uri": ["https://portal.azure.com"]}
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
## Create Deployment
|
||||
|
||||
```python
|
||||
from azure.mgmt.apicenter.models import Deployment, DeploymentState
|
||||
|
||||
deployment = client.deployments.create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-api-center",
|
||||
workspace_name="default",
|
||||
api_name="my-api",
|
||||
deployment_name="prod-deployment",
|
||||
resource=Deployment(
|
||||
title="Production Deployment",
|
||||
description="Deployed to production APIM",
|
||||
environment_id="/workspaces/default/environments/production",
|
||||
definition_id="/workspaces/default/apis/my-api/versions/v1/definitions/openapi",
|
||||
state=DeploymentState.ACTIVE,
|
||||
server={"runtime_uri": ["https://api.example.com"]}
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
## Define Custom Metadata
|
||||
|
||||
```python
|
||||
from azure.mgmt.apicenter.models import MetadataSchema
|
||||
|
||||
metadata = client.metadata_schemas.create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-api-center",
|
||||
metadata_schema_name="data-classification",
|
||||
resource=MetadataSchema(
|
||||
schema='{"type": "string", "title": "Data Classification", "enum": ["public", "internal", "confidential"]}'
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
## Client Types
|
||||
|
||||
| Client | Purpose |
|
||||
|--------|---------|
|
||||
| `ApiCenterMgmtClient` | Main client for all operations |
|
||||
|
||||
## Operations
|
||||
|
||||
| Operation Group | Purpose |
|
||||
|----------------|---------|
|
||||
| `services` | API Center service management |
|
||||
| `workspaces` | Workspace management |
|
||||
| `apis` | API registration and management |
|
||||
| `api_versions` | API version management |
|
||||
| `api_definitions` | API definition management |
|
||||
| `deployments` | Deployment tracking |
|
||||
| `environments` | Environment management |
|
||||
| `metadata_schemas` | Custom metadata definitions |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use workspaces** to organize APIs by team or domain
|
||||
2. **Define metadata schemas** for consistent governance
|
||||
3. **Track deployments** to understand where APIs are running
|
||||
4. **Import specifications** to enable API analysis and linting
|
||||
5. **Use lifecycle stages** to track API maturity
|
||||
6. **Add contacts** for API ownership and support
|
||||
@@ -0,0 +1,278 @@
|
||||
---
|
||||
name: azure-mgmt-apimanagement-py
|
||||
description: |
|
||||
Azure API Management SDK for Python. Use for managing APIM services, APIs, products, subscriptions, and policies.
|
||||
Triggers: "azure-mgmt-apimanagement", "ApiManagementClient", "APIM", "API gateway", "API Management".
|
||||
package: azure-mgmt-apimanagement
|
||||
---
|
||||
|
||||
# Azure API Management SDK for Python
|
||||
|
||||
Manage Azure API Management services, APIs, products, and policies.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-mgmt-apimanagement
|
||||
pip install azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_SUBSCRIPTION_ID=your-subscription-id
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.mgmt.apimanagement import ApiManagementClient
|
||||
import os
|
||||
|
||||
client = ApiManagementClient(
|
||||
credential=DefaultAzureCredential(),
|
||||
subscription_id=os.environ["AZURE_SUBSCRIPTION_ID"]
|
||||
)
|
||||
```
|
||||
|
||||
## Create APIM Service
|
||||
|
||||
```python
|
||||
from azure.mgmt.apimanagement.models import (
|
||||
ApiManagementServiceResource,
|
||||
ApiManagementServiceSkuProperties,
|
||||
SkuType
|
||||
)
|
||||
|
||||
service = client.api_management_service.begin_create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-apim",
|
||||
parameters=ApiManagementServiceResource(
|
||||
location="eastus",
|
||||
publisher_email="admin@example.com",
|
||||
publisher_name="My Organization",
|
||||
sku=ApiManagementServiceSkuProperties(
|
||||
name=SkuType.DEVELOPER,
|
||||
capacity=1
|
||||
)
|
||||
)
|
||||
).result()
|
||||
|
||||
print(f"Created APIM: {service.name}")
|
||||
```
|
||||
|
||||
## Import API from OpenAPI
|
||||
|
||||
```python
|
||||
from azure.mgmt.apimanagement.models import (
|
||||
ApiCreateOrUpdateParameter,
|
||||
ContentFormat,
|
||||
Protocol
|
||||
)
|
||||
|
||||
api = client.api.begin_create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-apim",
|
||||
api_id="my-api",
|
||||
parameters=ApiCreateOrUpdateParameter(
|
||||
display_name="My API",
|
||||
path="myapi",
|
||||
protocols=[Protocol.HTTPS],
|
||||
format=ContentFormat.OPENAPI_JSON,
|
||||
value='{"openapi": "3.0.0", "info": {"title": "My API", "version": "1.0"}, "paths": {"/health": {"get": {"responses": {"200": {"description": "OK"}}}}}}'
|
||||
)
|
||||
).result()
|
||||
|
||||
print(f"Imported API: {api.display_name}")
|
||||
```
|
||||
|
||||
## Import API from URL
|
||||
|
||||
```python
|
||||
api = client.api.begin_create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-apim",
|
||||
api_id="petstore",
|
||||
parameters=ApiCreateOrUpdateParameter(
|
||||
display_name="Petstore API",
|
||||
path="petstore",
|
||||
protocols=[Protocol.HTTPS],
|
||||
format=ContentFormat.OPENAPI_LINK,
|
||||
value="https://petstore.swagger.io/v2/swagger.json"
|
||||
)
|
||||
).result()
|
||||
```
|
||||
|
||||
## List APIs
|
||||
|
||||
```python
|
||||
apis = client.api.list_by_service(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-apim"
|
||||
)
|
||||
|
||||
for api in apis:
|
||||
print(f"{api.name}: {api.display_name} - {api.path}")
|
||||
```
|
||||
|
||||
## Create Product
|
||||
|
||||
```python
|
||||
from azure.mgmt.apimanagement.models import ProductContract
|
||||
|
||||
product = client.product.create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-apim",
|
||||
product_id="premium",
|
||||
parameters=ProductContract(
|
||||
display_name="Premium",
|
||||
description="Premium tier with unlimited access",
|
||||
subscription_required=True,
|
||||
approval_required=False,
|
||||
state="published"
|
||||
)
|
||||
)
|
||||
|
||||
print(f"Created product: {product.display_name}")
|
||||
```
|
||||
|
||||
## Add API to Product
|
||||
|
||||
```python
|
||||
client.product_api.create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-apim",
|
||||
product_id="premium",
|
||||
api_id="my-api"
|
||||
)
|
||||
```
|
||||
|
||||
## Create Subscription
|
||||
|
||||
```python
|
||||
from azure.mgmt.apimanagement.models import SubscriptionCreateParameters
|
||||
|
||||
subscription = client.subscription.create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-apim",
|
||||
sid="my-subscription",
|
||||
parameters=SubscriptionCreateParameters(
|
||||
display_name="My Subscription",
|
||||
scope=f"/products/premium",
|
||||
state="active"
|
||||
)
|
||||
)
|
||||
|
||||
print(f"Subscription key: {subscription.primary_key}")
|
||||
```
|
||||
|
||||
## Set API Policy
|
||||
|
||||
```python
|
||||
from azure.mgmt.apimanagement.models import PolicyContract
|
||||
|
||||
policy_xml = """
|
||||
<policies>
|
||||
<inbound>
|
||||
<rate-limit calls="100" renewal-period="60" />
|
||||
<set-header name="X-Custom-Header" exists-action="override">
|
||||
<value>CustomValue</value>
|
||||
</set-header>
|
||||
</inbound>
|
||||
<backend>
|
||||
<forward-request />
|
||||
</backend>
|
||||
<outbound />
|
||||
<on-error />
|
||||
</policies>
|
||||
"""
|
||||
|
||||
client.api_policy.create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-apim",
|
||||
api_id="my-api",
|
||||
policy_id="policy",
|
||||
parameters=PolicyContract(
|
||||
value=policy_xml,
|
||||
format="xml"
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
## Create Named Value (Secret)
|
||||
|
||||
```python
|
||||
from azure.mgmt.apimanagement.models import NamedValueCreateContract
|
||||
|
||||
named_value = client.named_value.begin_create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-apim",
|
||||
named_value_id="backend-api-key",
|
||||
parameters=NamedValueCreateContract(
|
||||
display_name="Backend API Key",
|
||||
value="secret-key-value",
|
||||
secret=True
|
||||
)
|
||||
).result()
|
||||
```
|
||||
|
||||
## Create Backend
|
||||
|
||||
```python
|
||||
from azure.mgmt.apimanagement.models import BackendContract
|
||||
|
||||
backend = client.backend.create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-apim",
|
||||
backend_id="my-backend",
|
||||
parameters=BackendContract(
|
||||
url="https://api.backend.example.com",
|
||||
protocol="http",
|
||||
description="My backend service"
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
## Create User
|
||||
|
||||
```python
|
||||
from azure.mgmt.apimanagement.models import UserCreateParameters
|
||||
|
||||
user = client.user.create_or_update(
|
||||
resource_group_name="my-resource-group",
|
||||
service_name="my-apim",
|
||||
user_id="newuser",
|
||||
parameters=UserCreateParameters(
|
||||
email="user@example.com",
|
||||
first_name="John",
|
||||
last_name="Doe"
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
## Operation Groups
|
||||
|
||||
| Group | Purpose |
|
||||
|-------|---------|
|
||||
| `api_management_service` | APIM instance management |
|
||||
| `api` | API operations |
|
||||
| `api_operation` | API operation details |
|
||||
| `api_policy` | API-level policies |
|
||||
| `product` | Product management |
|
||||
| `product_api` | Product-API associations |
|
||||
| `subscription` | Subscription management |
|
||||
| `user` | User management |
|
||||
| `named_value` | Named values/secrets |
|
||||
| `backend` | Backend services |
|
||||
| `certificate` | Certificates |
|
||||
| `gateway` | Self-hosted gateways |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use named values** for secrets and configuration
|
||||
2. **Apply policies** at appropriate scopes (global, product, API, operation)
|
||||
3. **Use products** to bundle APIs and manage access
|
||||
4. **Enable Application Insights** for monitoring
|
||||
5. **Use backends** to abstract backend services
|
||||
6. **Version your APIs** using APIM's versioning features
|
||||
@@ -0,0 +1,249 @@
|
||||
---
|
||||
name: azure-appconfiguration-py
|
||||
description: |
|
||||
Azure App Configuration SDK for Python. Use for centralized configuration management, feature flags, and dynamic settings.
|
||||
Triggers: "azure-appconfiguration", "AzureAppConfigurationClient", "feature flags", "configuration", "key-value settings".
|
||||
package: azure-appconfiguration
|
||||
---
|
||||
|
||||
# Azure App Configuration SDK for Python
|
||||
|
||||
Centralized configuration management with feature flags and dynamic settings.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-appconfiguration
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_APPCONFIGURATION_CONNECTION_STRING=Endpoint=https://<name>.azconfig.io;Id=...;Secret=...
|
||||
# Or for Entra ID:
|
||||
AZURE_APPCONFIGURATION_ENDPOINT=https://<name>.azconfig.io
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
### Connection String
|
||||
|
||||
```python
|
||||
from azure.appconfiguration import AzureAppConfigurationClient
|
||||
|
||||
client = AzureAppConfigurationClient.from_connection_string(
|
||||
os.environ["AZURE_APPCONFIGURATION_CONNECTION_STRING"]
|
||||
)
|
||||
```
|
||||
|
||||
### Entra ID
|
||||
|
||||
```python
|
||||
from azure.appconfiguration import AzureAppConfigurationClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
client = AzureAppConfigurationClient(
|
||||
base_url=os.environ["AZURE_APPCONFIGURATION_ENDPOINT"],
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
```
|
||||
|
||||
## Configuration Settings
|
||||
|
||||
### Get Setting
|
||||
|
||||
```python
|
||||
setting = client.get_configuration_setting(key="app:settings:message")
|
||||
print(f"{setting.key} = {setting.value}")
|
||||
```
|
||||
|
||||
### Get with Label
|
||||
|
||||
```python
|
||||
# Labels allow environment-specific values
|
||||
setting = client.get_configuration_setting(
|
||||
key="app:settings:message",
|
||||
label="production"
|
||||
)
|
||||
```
|
||||
|
||||
### Set Setting
|
||||
|
||||
```python
|
||||
from azure.appconfiguration import ConfigurationSetting
|
||||
|
||||
setting = ConfigurationSetting(
|
||||
key="app:settings:message",
|
||||
value="Hello, World!",
|
||||
label="development",
|
||||
content_type="text/plain",
|
||||
tags={"environment": "dev"}
|
||||
)
|
||||
|
||||
client.set_configuration_setting(setting)
|
||||
```
|
||||
|
||||
### Delete Setting
|
||||
|
||||
```python
|
||||
client.delete_configuration_setting(
|
||||
key="app:settings:message",
|
||||
label="development"
|
||||
)
|
||||
```
|
||||
|
||||
## List Settings
|
||||
|
||||
### All Settings
|
||||
|
||||
```python
|
||||
settings = client.list_configuration_settings()
|
||||
for setting in settings:
|
||||
print(f"{setting.key} [{setting.label}] = {setting.value}")
|
||||
```
|
||||
|
||||
### Filter by Key Prefix
|
||||
|
||||
```python
|
||||
settings = client.list_configuration_settings(
|
||||
key_filter="app:settings:*"
|
||||
)
|
||||
```
|
||||
|
||||
### Filter by Label
|
||||
|
||||
```python
|
||||
settings = client.list_configuration_settings(
|
||||
label_filter="production"
|
||||
)
|
||||
```
|
||||
|
||||
## Feature Flags
|
||||
|
||||
### Set Feature Flag
|
||||
|
||||
```python
|
||||
from azure.appconfiguration import ConfigurationSetting
|
||||
import json
|
||||
|
||||
feature_flag = ConfigurationSetting(
|
||||
key=".appconfig.featureflag/beta-feature",
|
||||
value=json.dumps({
|
||||
"id": "beta-feature",
|
||||
"enabled": True,
|
||||
"conditions": {
|
||||
"client_filters": []
|
||||
}
|
||||
}),
|
||||
content_type="application/vnd.microsoft.appconfig.ff+json;charset=utf-8"
|
||||
)
|
||||
|
||||
client.set_configuration_setting(feature_flag)
|
||||
```
|
||||
|
||||
### Get Feature Flag
|
||||
|
||||
```python
|
||||
setting = client.get_configuration_setting(
|
||||
key=".appconfig.featureflag/beta-feature"
|
||||
)
|
||||
flag_data = json.loads(setting.value)
|
||||
print(f"Feature enabled: {flag_data['enabled']}")
|
||||
```
|
||||
|
||||
### List Feature Flags
|
||||
|
||||
```python
|
||||
flags = client.list_configuration_settings(
|
||||
key_filter=".appconfig.featureflag/*"
|
||||
)
|
||||
for flag in flags:
|
||||
data = json.loads(flag.value)
|
||||
print(f"{data['id']}: {'enabled' if data['enabled'] else 'disabled'}")
|
||||
```
|
||||
|
||||
## Read-Only Settings
|
||||
|
||||
```python
|
||||
# Make setting read-only
|
||||
client.set_read_only(
|
||||
configuration_setting=setting,
|
||||
read_only=True
|
||||
)
|
||||
|
||||
# Remove read-only
|
||||
client.set_read_only(
|
||||
configuration_setting=setting,
|
||||
read_only=False
|
||||
)
|
||||
```
|
||||
|
||||
## Snapshots
|
||||
|
||||
### Create Snapshot
|
||||
|
||||
```python
|
||||
from azure.appconfiguration import ConfigurationSnapshot, ConfigurationSettingFilter
|
||||
|
||||
snapshot = ConfigurationSnapshot(
|
||||
name="v1-snapshot",
|
||||
filters=[
|
||||
ConfigurationSettingFilter(key="app:*", label="production")
|
||||
]
|
||||
)
|
||||
|
||||
created = client.begin_create_snapshot(
|
||||
name="v1-snapshot",
|
||||
snapshot=snapshot
|
||||
).result()
|
||||
```
|
||||
|
||||
### List Snapshot Settings
|
||||
|
||||
```python
|
||||
settings = client.list_configuration_settings(
|
||||
snapshot_name="v1-snapshot"
|
||||
)
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.appconfiguration.aio import AzureAppConfigurationClient
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def main():
|
||||
credential = DefaultAzureCredential()
|
||||
client = AzureAppConfigurationClient(
|
||||
base_url=endpoint,
|
||||
credential=credential
|
||||
)
|
||||
|
||||
setting = await client.get_configuration_setting(key="app:message")
|
||||
print(setting.value)
|
||||
|
||||
await client.close()
|
||||
await credential.close()
|
||||
```
|
||||
|
||||
## Client Operations
|
||||
|
||||
| Operation | Description |
|
||||
|-----------|-------------|
|
||||
| `get_configuration_setting` | Get single setting |
|
||||
| `set_configuration_setting` | Create or update setting |
|
||||
| `delete_configuration_setting` | Delete setting |
|
||||
| `list_configuration_settings` | List with filters |
|
||||
| `set_read_only` | Lock/unlock setting |
|
||||
| `begin_create_snapshot` | Create point-in-time snapshot |
|
||||
| `list_snapshots` | List all snapshots |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use labels** for environment separation (dev, staging, prod)
|
||||
2. **Use key prefixes** for logical grouping (app:database:*, app:cache:*)
|
||||
3. **Make production settings read-only** to prevent accidental changes
|
||||
4. **Create snapshots** before deployments for rollback capability
|
||||
5. **Use Entra ID** instead of connection strings in production
|
||||
6. **Refresh settings periodically** in long-running applications
|
||||
7. **Use feature flags** for gradual rollouts and A/B testing
|
||||
345
skills/official/microsoft/python/m365/m365-agents/SKILL.md
Normal file
345
skills/official/microsoft/python/m365/m365-agents/SKILL.md
Normal file
@@ -0,0 +1,345 @@
|
||||
---
|
||||
name: m365-agents-py
|
||||
description: |
|
||||
Microsoft 365 Agents SDK for Python. Build multichannel agents for Teams/M365/Copilot Studio with aiohttp hosting, AgentApplication routing, streaming responses, and MSAL-based auth. Triggers: "Microsoft 365 Agents SDK", "microsoft_agents", "AgentApplication", "start_agent_process", "TurnContext", "Copilot Studio client", "CloudAdapter".
|
||||
package: microsoft-agents-hosting-core, microsoft-agents-hosting-aiohttp, microsoft-agents-activity, microsoft-agents-authentication-msal, microsoft-agents-copilotstudio-client
|
||||
---
|
||||
|
||||
# Microsoft 365 Agents SDK (Python)
|
||||
|
||||
Build enterprise agents for Microsoft 365, Teams, and Copilot Studio using the Microsoft Agents SDK with aiohttp hosting, AgentApplication routing, streaming responses, and MSAL-based authentication.
|
||||
|
||||
## Before implementation
|
||||
- Use the microsoft-docs MCP to verify the latest API signatures for AgentApplication, start_agent_process, and authentication options.
|
||||
- Confirm package versions on PyPI for the microsoft-agents-* packages you plan to use.
|
||||
|
||||
## Important Notice - Import Changes
|
||||
|
||||
> **⚠️ Breaking Change**: Recent updates have changed the Python import structure from `microsoft.agents` to `microsoft_agents` (using underscores instead of dots).
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install microsoft-agents-hosting-core
|
||||
pip install microsoft-agents-hosting-aiohttp
|
||||
pip install microsoft-agents-activity
|
||||
pip install microsoft-agents-authentication-msal
|
||||
pip install microsoft-agents-copilotstudio-client
|
||||
pip install python-dotenv aiohttp
|
||||
```
|
||||
|
||||
## Environment Variables (.env)
|
||||
|
||||
```bash
|
||||
CONNECTIONS__SERVICE_CONNECTION__SETTINGS__CLIENTID=<client-id>
|
||||
CONNECTIONS__SERVICE_CONNECTION__SETTINGS__CLIENTSECRET=<client-secret>
|
||||
CONNECTIONS__SERVICE_CONNECTION__SETTINGS__TENANTID=<tenant-id>
|
||||
|
||||
# Optional: OAuth handlers for auto sign-in
|
||||
AGENTAPPLICATION__USERAUTHORIZATION__HANDLERS__GRAPH__SETTINGS__AZUREBOTOAUTHCONNECTIONNAME=<connection-name>
|
||||
|
||||
# Optional: Azure OpenAI for streaming
|
||||
AZURE_OPENAI_ENDPOINT=<endpoint>
|
||||
AZURE_OPENAI_API_VERSION=<version>
|
||||
AZURE_OPENAI_API_KEY=<key>
|
||||
|
||||
# Optional: Copilot Studio client
|
||||
COPILOTSTUDIOAGENT__ENVIRONMENTID=<environment-id>
|
||||
COPILOTSTUDIOAGENT__SCHEMANAME=<schema-name>
|
||||
COPILOTSTUDIOAGENT__TENANTID=<tenant-id>
|
||||
COPILOTSTUDIOAGENT__AGENTAPPID=<app-id>
|
||||
```
|
||||
|
||||
## Core Workflow: aiohttp-hosted AgentApplication
|
||||
|
||||
```python
|
||||
import logging
|
||||
from os import environ
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from aiohttp.web import Request, Response, Application, run_app
|
||||
|
||||
from microsoft_agents.activity import load_configuration_from_env
|
||||
from microsoft_agents.hosting.core import (
|
||||
Authorization,
|
||||
AgentApplication,
|
||||
TurnState,
|
||||
TurnContext,
|
||||
MemoryStorage,
|
||||
)
|
||||
from microsoft_agents.hosting.aiohttp import (
|
||||
CloudAdapter,
|
||||
start_agent_process,
|
||||
jwt_authorization_middleware,
|
||||
)
|
||||
from microsoft_agents.authentication.msal import MsalConnectionManager
|
||||
|
||||
# Enable logging
|
||||
ms_agents_logger = logging.getLogger("microsoft_agents")
|
||||
ms_agents_logger.addHandler(logging.StreamHandler())
|
||||
ms_agents_logger.setLevel(logging.INFO)
|
||||
|
||||
# Load configuration
|
||||
load_dotenv()
|
||||
agents_sdk_config = load_configuration_from_env(environ)
|
||||
|
||||
# Create storage and connection manager
|
||||
STORAGE = MemoryStorage()
|
||||
CONNECTION_MANAGER = MsalConnectionManager(**agents_sdk_config)
|
||||
ADAPTER = CloudAdapter(connection_manager=CONNECTION_MANAGER)
|
||||
AUTHORIZATION = Authorization(STORAGE, CONNECTION_MANAGER, **agents_sdk_config)
|
||||
|
||||
# Create AgentApplication
|
||||
AGENT_APP = AgentApplication[TurnState](
|
||||
storage=STORAGE, adapter=ADAPTER, authorization=AUTHORIZATION, **agents_sdk_config
|
||||
)
|
||||
|
||||
|
||||
@AGENT_APP.conversation_update("membersAdded")
|
||||
async def on_members_added(context: TurnContext, _state: TurnState):
|
||||
await context.send_activity("Welcome to the agent!")
|
||||
|
||||
|
||||
@AGENT_APP.activity("message")
|
||||
async def on_message(context: TurnContext, _state: TurnState):
|
||||
await context.send_activity(f"You said: {context.activity.text}")
|
||||
|
||||
|
||||
@AGENT_APP.error
|
||||
async def on_error(context: TurnContext, error: Exception):
|
||||
await context.send_activity("The agent encountered an error.")
|
||||
|
||||
|
||||
# Server setup
|
||||
async def entry_point(req: Request) -> Response:
|
||||
agent: AgentApplication = req.app["agent_app"]
|
||||
adapter: CloudAdapter = req.app["adapter"]
|
||||
return await start_agent_process(req, agent, adapter)
|
||||
|
||||
|
||||
APP = Application(middlewares=[jwt_authorization_middleware])
|
||||
APP.router.add_post("/api/messages", entry_point)
|
||||
APP["agent_configuration"] = CONNECTION_MANAGER.get_default_connection_configuration()
|
||||
APP["agent_app"] = AGENT_APP
|
||||
APP["adapter"] = AGENT_APP.adapter
|
||||
|
||||
if __name__ == "__main__":
|
||||
run_app(APP, host="localhost", port=environ.get("PORT", 3978))
|
||||
```
|
||||
|
||||
## AgentApplication Routing
|
||||
|
||||
```python
|
||||
import re
|
||||
from microsoft_agents.hosting.core import (
|
||||
AgentApplication, TurnState, TurnContext, MessageFactory
|
||||
)
|
||||
from microsoft_agents.activity import ActivityTypes
|
||||
|
||||
AGENT_APP = AgentApplication[TurnState](
|
||||
storage=STORAGE, adapter=ADAPTER, authorization=AUTHORIZATION, **agents_sdk_config
|
||||
)
|
||||
|
||||
# Welcome handler
|
||||
@AGENT_APP.conversation_update("membersAdded")
|
||||
async def on_members_added(context: TurnContext, _state: TurnState):
|
||||
await context.send_activity("Welcome!")
|
||||
|
||||
# Regex-based message handler
|
||||
@AGENT_APP.message(re.compile(r"^hello$", re.IGNORECASE))
|
||||
async def on_hello(context: TurnContext, _state: TurnState):
|
||||
await context.send_activity("Hello!")
|
||||
|
||||
# Simple string message handler
|
||||
@AGENT_APP.message("/status")
|
||||
async def on_status(context: TurnContext, _state: TurnState):
|
||||
await context.send_activity("Status: OK")
|
||||
|
||||
# Auth-protected message handler
|
||||
@AGENT_APP.message("/me", auth_handlers=["GRAPH"])
|
||||
async def on_profile(context: TurnContext, state: TurnState):
|
||||
token_response = await AGENT_APP.auth.get_token(context, "GRAPH")
|
||||
if token_response and token_response.token:
|
||||
# Use token to call Graph API
|
||||
await context.send_activity("Profile retrieved")
|
||||
|
||||
# Invoke activity handler
|
||||
@AGENT_APP.activity(ActivityTypes.invoke)
|
||||
async def on_invoke(context: TurnContext, _state: TurnState):
|
||||
invoke_response = Activity(
|
||||
type=ActivityTypes.invoke_response, value={"status": 200}
|
||||
)
|
||||
await context.send_activity(invoke_response)
|
||||
|
||||
# Fallback message handler
|
||||
@AGENT_APP.activity("message")
|
||||
async def on_message(context: TurnContext, _state: TurnState):
|
||||
await context.send_activity(f"Echo: {context.activity.text}")
|
||||
|
||||
# Error handler
|
||||
@AGENT_APP.error
|
||||
async def on_error(context: TurnContext, error: Exception):
|
||||
await context.send_activity("An error occurred.")
|
||||
```
|
||||
|
||||
## Streaming Responses with Azure OpenAI
|
||||
|
||||
```python
|
||||
from openai import AsyncAzureOpenAI
|
||||
from microsoft_agents.activity import SensitivityUsageInfo
|
||||
|
||||
CLIENT = AsyncAzureOpenAI(
|
||||
api_version=environ["AZURE_OPENAI_API_VERSION"],
|
||||
azure_endpoint=environ["AZURE_OPENAI_ENDPOINT"],
|
||||
api_key=environ["AZURE_OPENAI_API_KEY"]
|
||||
)
|
||||
|
||||
@AGENT_APP.message("poem")
|
||||
async def on_poem_message(context: TurnContext, _state: TurnState):
|
||||
# Configure streaming response
|
||||
context.streaming_response.set_feedback_loop(True)
|
||||
context.streaming_response.set_generated_by_ai_label(True)
|
||||
context.streaming_response.set_sensitivity_label(
|
||||
SensitivityUsageInfo(
|
||||
type="https://schema.org/Message",
|
||||
schema_type="CreativeWork",
|
||||
name="Internal",
|
||||
)
|
||||
)
|
||||
context.streaming_response.queue_informative_update("Starting a poem...\n")
|
||||
|
||||
# Stream from Azure OpenAI
|
||||
streamed_response = await CLIENT.chat.completions.create(
|
||||
model="gpt-4o",
|
||||
messages=[
|
||||
{"role": "system", "content": "You are a creative assistant."},
|
||||
{"role": "user", "content": "Write a poem about Python."}
|
||||
],
|
||||
stream=True,
|
||||
)
|
||||
|
||||
try:
|
||||
async for chunk in streamed_response:
|
||||
if chunk.choices and chunk.choices[0].delta.content:
|
||||
context.streaming_response.queue_text_chunk(
|
||||
chunk.choices[0].delta.content
|
||||
)
|
||||
finally:
|
||||
await context.streaming_response.end_stream()
|
||||
```
|
||||
|
||||
## OAuth / Auto Sign-In
|
||||
|
||||
```python
|
||||
@AGENT_APP.message("/logout")
|
||||
async def logout(context: TurnContext, state: TurnState):
|
||||
await AGENT_APP.auth.sign_out(context, "GRAPH")
|
||||
await context.send_activity(MessageFactory.text("You have been logged out."))
|
||||
|
||||
|
||||
@AGENT_APP.message("/me", auth_handlers=["GRAPH"])
|
||||
async def profile_request(context: TurnContext, state: TurnState):
|
||||
user_token_response = await AGENT_APP.auth.get_token(context, "GRAPH")
|
||||
if user_token_response and user_token_response.token:
|
||||
# Use token to call Microsoft Graph
|
||||
async with aiohttp.ClientSession() as session:
|
||||
headers = {
|
||||
"Authorization": f"Bearer {user_token_response.token}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
async with session.get(
|
||||
"https://graph.microsoft.com/v1.0/me", headers=headers
|
||||
) as response:
|
||||
if response.status == 200:
|
||||
user_info = await response.json()
|
||||
await context.send_activity(f"Hello, {user_info['displayName']}!")
|
||||
```
|
||||
|
||||
## Copilot Studio Client (Direct to Engine)
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
from msal import PublicClientApplication
|
||||
from microsoft_agents.activity import ActivityTypes, load_configuration_from_env
|
||||
from microsoft_agents.copilotstudio.client import (
|
||||
ConnectionSettings,
|
||||
CopilotClient,
|
||||
)
|
||||
|
||||
# Token cache (local file for interactive flows)
|
||||
class LocalTokenCache:
|
||||
# See samples for full implementation
|
||||
pass
|
||||
|
||||
def acquire_token(settings, app_client_id, tenant_id):
|
||||
pca = PublicClientApplication(
|
||||
client_id=app_client_id,
|
||||
authority=f"https://login.microsoftonline.com/{tenant_id}",
|
||||
)
|
||||
|
||||
token_request = {"scopes": ["https://api.powerplatform.com/.default"]}
|
||||
accounts = pca.get_accounts()
|
||||
|
||||
if accounts:
|
||||
response = pca.acquire_token_silent(token_request["scopes"], account=accounts[0])
|
||||
return response.get("access_token")
|
||||
else:
|
||||
response = pca.acquire_token_interactive(**token_request)
|
||||
return response.get("access_token")
|
||||
|
||||
|
||||
async def main():
|
||||
settings = ConnectionSettings(
|
||||
environment_id=environ.get("COPILOTSTUDIOAGENT__ENVIRONMENTID"),
|
||||
agent_identifier=environ.get("COPILOTSTUDIOAGENT__SCHEMANAME"),
|
||||
)
|
||||
|
||||
token = acquire_token(
|
||||
settings,
|
||||
app_client_id=environ.get("COPILOTSTUDIOAGENT__AGENTAPPID"),
|
||||
tenant_id=environ.get("COPILOTSTUDIOAGENT__TENANTID"),
|
||||
)
|
||||
|
||||
copilot_client = CopilotClient(settings, token)
|
||||
|
||||
# Start conversation
|
||||
act = copilot_client.start_conversation(True)
|
||||
async for action in act:
|
||||
if action.text:
|
||||
print(action.text)
|
||||
|
||||
# Ask question
|
||||
replies = copilot_client.ask_question("Hello!", action.conversation.id)
|
||||
async for reply in replies:
|
||||
if reply.type == ActivityTypes.message:
|
||||
print(reply.text)
|
||||
|
||||
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. Use `microsoft_agents` import prefix (underscores, not dots).
|
||||
2. Use `MemoryStorage` only for development; use BlobStorage or CosmosDB in production.
|
||||
3. Always use `load_configuration_from_env(environ)` to load SDK configuration.
|
||||
4. Include `jwt_authorization_middleware` in aiohttp Application middlewares.
|
||||
5. Use `MsalConnectionManager` for MSAL-based authentication.
|
||||
6. Call `end_stream()` in finally blocks when using streaming responses.
|
||||
7. Use `auth_handlers` parameter on message decorators for OAuth-protected routes.
|
||||
8. Keep secrets in environment variables, not in source code.
|
||||
|
||||
## Reference Files
|
||||
|
||||
| File | Contents |
|
||||
| --- | --- |
|
||||
| [references/acceptance-criteria.md](references/acceptance-criteria.md) | Import paths, hosting pipeline, streaming, OAuth, and Copilot Studio patterns |
|
||||
|
||||
## Reference Links
|
||||
|
||||
| Resource | URL |
|
||||
| --- | --- |
|
||||
| Microsoft 365 Agents SDK | https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/ |
|
||||
| GitHub samples (Python) | https://github.com/microsoft/Agents-for-python |
|
||||
| PyPI packages | https://pypi.org/search/?q=microsoft-agents |
|
||||
| Integrate with Copilot Studio | https://learn.microsoft.com/en-us/microsoft-365/agents-sdk/integrate-with-mcs |
|
||||
168
skills/official/microsoft/python/messaging/eventgrid/SKILL.md
Normal file
168
skills/official/microsoft/python/messaging/eventgrid/SKILL.md
Normal file
@@ -0,0 +1,168 @@
|
||||
---
|
||||
name: azure-eventgrid-py
|
||||
description: |
|
||||
Azure Event Grid SDK for Python. Use for publishing events, handling CloudEvents, and event-driven architectures.
|
||||
Triggers: "event grid", "EventGridPublisherClient", "CloudEvent", "EventGridEvent", "publish events".
|
||||
package: azure-eventgrid
|
||||
---
|
||||
|
||||
# Azure Event Grid SDK for Python
|
||||
|
||||
Event routing service for building event-driven applications with pub/sub semantics.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-eventgrid azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
EVENTGRID_TOPIC_ENDPOINT=https://<topic-name>.<region>.eventgrid.azure.net/api/events
|
||||
EVENTGRID_NAMESPACE_ENDPOINT=https://<namespace>.<region>.eventgrid.azure.net
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.eventgrid import EventGridPublisherClient
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
endpoint = "https://<topic-name>.<region>.eventgrid.azure.net/api/events"
|
||||
|
||||
client = EventGridPublisherClient(endpoint, credential)
|
||||
```
|
||||
|
||||
## Event Types
|
||||
|
||||
| Format | Class | Use Case |
|
||||
|--------|-------|----------|
|
||||
| Cloud Events 1.0 | `CloudEvent` | Standard, interoperable (recommended) |
|
||||
| Event Grid Schema | `EventGridEvent` | Azure-native format |
|
||||
|
||||
## Publish CloudEvents
|
||||
|
||||
```python
|
||||
from azure.eventgrid import EventGridPublisherClient, CloudEvent
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
client = EventGridPublisherClient(endpoint, DefaultAzureCredential())
|
||||
|
||||
# Single event
|
||||
event = CloudEvent(
|
||||
type="MyApp.Events.OrderCreated",
|
||||
source="/myapp/orders",
|
||||
data={"order_id": "12345", "amount": 99.99}
|
||||
)
|
||||
client.send(event)
|
||||
|
||||
# Multiple events
|
||||
events = [
|
||||
CloudEvent(
|
||||
type="MyApp.Events.OrderCreated",
|
||||
source="/myapp/orders",
|
||||
data={"order_id": f"order-{i}"}
|
||||
)
|
||||
for i in range(10)
|
||||
]
|
||||
client.send(events)
|
||||
```
|
||||
|
||||
## Publish EventGridEvents
|
||||
|
||||
```python
|
||||
from azure.eventgrid import EventGridEvent
|
||||
from datetime import datetime, timezone
|
||||
|
||||
event = EventGridEvent(
|
||||
subject="/myapp/orders/12345",
|
||||
event_type="MyApp.Events.OrderCreated",
|
||||
data={"order_id": "12345", "amount": 99.99},
|
||||
data_version="1.0"
|
||||
)
|
||||
|
||||
client.send(event)
|
||||
```
|
||||
|
||||
## Event Properties
|
||||
|
||||
### CloudEvent Properties
|
||||
|
||||
```python
|
||||
event = CloudEvent(
|
||||
type="MyApp.Events.ItemCreated", # Required: event type
|
||||
source="/myapp/items", # Required: event source
|
||||
data={"key": "value"}, # Event payload
|
||||
subject="items/123", # Optional: subject/path
|
||||
datacontenttype="application/json", # Optional: content type
|
||||
dataschema="https://schema.example", # Optional: schema URL
|
||||
time=datetime.now(timezone.utc), # Optional: timestamp
|
||||
extensions={"custom": "value"} # Optional: custom attributes
|
||||
)
|
||||
```
|
||||
|
||||
### EventGridEvent Properties
|
||||
|
||||
```python
|
||||
event = EventGridEvent(
|
||||
subject="/myapp/items/123", # Required: subject
|
||||
event_type="MyApp.ItemCreated", # Required: event type
|
||||
data={"key": "value"}, # Required: event payload
|
||||
data_version="1.0", # Required: schema version
|
||||
topic="/subscriptions/.../topics/...", # Optional: auto-set
|
||||
event_time=datetime.now(timezone.utc) # Optional: timestamp
|
||||
)
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.eventgrid.aio import EventGridPublisherClient
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def publish_events():
|
||||
credential = DefaultAzureCredential()
|
||||
|
||||
async with EventGridPublisherClient(endpoint, credential) as client:
|
||||
event = CloudEvent(
|
||||
type="MyApp.Events.Test",
|
||||
source="/myapp",
|
||||
data={"message": "hello"}
|
||||
)
|
||||
await client.send(event)
|
||||
|
||||
import asyncio
|
||||
asyncio.run(publish_events())
|
||||
```
|
||||
|
||||
## Namespace Topics (Event Grid Namespaces)
|
||||
|
||||
For Event Grid Namespaces (pull delivery):
|
||||
|
||||
```python
|
||||
from azure.eventgrid.aio import EventGridPublisherClient
|
||||
|
||||
# Namespace endpoint (different from custom topic)
|
||||
namespace_endpoint = "https://<namespace>.<region>.eventgrid.azure.net"
|
||||
topic_name = "my-topic"
|
||||
|
||||
async with EventGridPublisherClient(
|
||||
endpoint=namespace_endpoint,
|
||||
credential=DefaultAzureCredential()
|
||||
) as client:
|
||||
await client.send(
|
||||
event,
|
||||
namespace_topic=topic_name
|
||||
)
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use CloudEvents** for new applications (industry standard)
|
||||
2. **Batch events** when publishing multiple events
|
||||
3. **Include meaningful subjects** for filtering
|
||||
4. **Use async client** for high-throughput scenarios
|
||||
5. **Handle retries** — Event Grid has built-in retry
|
||||
6. **Set appropriate event types** for routing and filtering
|
||||
240
skills/official/microsoft/python/messaging/eventhub/SKILL.md
Normal file
240
skills/official/microsoft/python/messaging/eventhub/SKILL.md
Normal file
@@ -0,0 +1,240 @@
|
||||
---
|
||||
name: azure-eventhub-py
|
||||
description: |
|
||||
Azure Event Hubs SDK for Python streaming. Use for high-throughput event ingestion, producers, consumers, and checkpointing.
|
||||
Triggers: "event hubs", "EventHubProducerClient", "EventHubConsumerClient", "streaming", "partitions".
|
||||
package: azure-eventhub
|
||||
---
|
||||
|
||||
# Azure Event Hubs SDK for Python
|
||||
|
||||
Big data streaming platform for high-throughput event ingestion.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-eventhub azure-identity
|
||||
# For checkpointing with blob storage
|
||||
pip install azure-eventhub-checkpointstoreblob-aio
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
EVENT_HUB_FULLY_QUALIFIED_NAMESPACE=<namespace>.servicebus.windows.net
|
||||
EVENT_HUB_NAME=my-eventhub
|
||||
STORAGE_ACCOUNT_URL=https://<account>.blob.core.windows.net
|
||||
CHECKPOINT_CONTAINER=checkpoints
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.eventhub import EventHubProducerClient, EventHubConsumerClient
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
namespace = "<namespace>.servicebus.windows.net"
|
||||
eventhub_name = "my-eventhub"
|
||||
|
||||
# Producer
|
||||
producer = EventHubProducerClient(
|
||||
fully_qualified_namespace=namespace,
|
||||
eventhub_name=eventhub_name,
|
||||
credential=credential
|
||||
)
|
||||
|
||||
# Consumer
|
||||
consumer = EventHubConsumerClient(
|
||||
fully_qualified_namespace=namespace,
|
||||
eventhub_name=eventhub_name,
|
||||
consumer_group="$Default",
|
||||
credential=credential
|
||||
)
|
||||
```
|
||||
|
||||
## Client Types
|
||||
|
||||
| Client | Purpose |
|
||||
|--------|---------|
|
||||
| `EventHubProducerClient` | Send events to Event Hub |
|
||||
| `EventHubConsumerClient` | Receive events from Event Hub |
|
||||
| `BlobCheckpointStore` | Track consumer progress |
|
||||
|
||||
## Send Events
|
||||
|
||||
```python
|
||||
from azure.eventhub import EventHubProducerClient, EventData
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
producer = EventHubProducerClient(
|
||||
fully_qualified_namespace="<namespace>.servicebus.windows.net",
|
||||
eventhub_name="my-eventhub",
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
|
||||
with producer:
|
||||
# Create batch (handles size limits)
|
||||
event_data_batch = producer.create_batch()
|
||||
|
||||
for i in range(10):
|
||||
try:
|
||||
event_data_batch.add(EventData(f"Event {i}"))
|
||||
except ValueError:
|
||||
# Batch is full, send and create new one
|
||||
producer.send_batch(event_data_batch)
|
||||
event_data_batch = producer.create_batch()
|
||||
event_data_batch.add(EventData(f"Event {i}"))
|
||||
|
||||
# Send remaining
|
||||
producer.send_batch(event_data_batch)
|
||||
```
|
||||
|
||||
### Send to Specific Partition
|
||||
|
||||
```python
|
||||
# By partition ID
|
||||
event_data_batch = producer.create_batch(partition_id="0")
|
||||
|
||||
# By partition key (consistent hashing)
|
||||
event_data_batch = producer.create_batch(partition_key="user-123")
|
||||
```
|
||||
|
||||
## Receive Events
|
||||
|
||||
### Simple Receive
|
||||
|
||||
```python
|
||||
from azure.eventhub import EventHubConsumerClient
|
||||
|
||||
def on_event(partition_context, event):
|
||||
print(f"Partition: {partition_context.partition_id}")
|
||||
print(f"Data: {event.body_as_str()}")
|
||||
partition_context.update_checkpoint(event)
|
||||
|
||||
consumer = EventHubConsumerClient(
|
||||
fully_qualified_namespace="<namespace>.servicebus.windows.net",
|
||||
eventhub_name="my-eventhub",
|
||||
consumer_group="$Default",
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
|
||||
with consumer:
|
||||
consumer.receive(
|
||||
on_event=on_event,
|
||||
starting_position="-1", # Beginning of stream
|
||||
)
|
||||
```
|
||||
|
||||
### With Blob Checkpoint Store (Production)
|
||||
|
||||
```python
|
||||
from azure.eventhub import EventHubConsumerClient
|
||||
from azure.eventhub.extensions.checkpointstoreblob import BlobCheckpointStore
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
checkpoint_store = BlobCheckpointStore(
|
||||
blob_account_url="https://<account>.blob.core.windows.net",
|
||||
container_name="checkpoints",
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
|
||||
consumer = EventHubConsumerClient(
|
||||
fully_qualified_namespace="<namespace>.servicebus.windows.net",
|
||||
eventhub_name="my-eventhub",
|
||||
consumer_group="$Default",
|
||||
credential=DefaultAzureCredential(),
|
||||
checkpoint_store=checkpoint_store
|
||||
)
|
||||
|
||||
def on_event(partition_context, event):
|
||||
print(f"Received: {event.body_as_str()}")
|
||||
# Checkpoint after processing
|
||||
partition_context.update_checkpoint(event)
|
||||
|
||||
with consumer:
|
||||
consumer.receive(on_event=on_event)
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
from azure.eventhub.aio import EventHubProducerClient, EventHubConsumerClient
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
import asyncio
|
||||
|
||||
async def send_events():
|
||||
credential = DefaultAzureCredential()
|
||||
|
||||
async with EventHubProducerClient(
|
||||
fully_qualified_namespace="<namespace>.servicebus.windows.net",
|
||||
eventhub_name="my-eventhub",
|
||||
credential=credential
|
||||
) as producer:
|
||||
batch = await producer.create_batch()
|
||||
batch.add(EventData("Async event"))
|
||||
await producer.send_batch(batch)
|
||||
|
||||
async def receive_events():
|
||||
async def on_event(partition_context, event):
|
||||
print(event.body_as_str())
|
||||
await partition_context.update_checkpoint(event)
|
||||
|
||||
async with EventHubConsumerClient(
|
||||
fully_qualified_namespace="<namespace>.servicebus.windows.net",
|
||||
eventhub_name="my-eventhub",
|
||||
consumer_group="$Default",
|
||||
credential=DefaultAzureCredential()
|
||||
) as consumer:
|
||||
await consumer.receive(on_event=on_event)
|
||||
|
||||
asyncio.run(send_events())
|
||||
```
|
||||
|
||||
## Event Properties
|
||||
|
||||
```python
|
||||
event = EventData("My event body")
|
||||
|
||||
# Set properties
|
||||
event.properties = {"custom_property": "value"}
|
||||
event.content_type = "application/json"
|
||||
|
||||
# Read properties (on receive)
|
||||
print(event.body_as_str())
|
||||
print(event.sequence_number)
|
||||
print(event.offset)
|
||||
print(event.enqueued_time)
|
||||
print(event.partition_key)
|
||||
```
|
||||
|
||||
## Get Event Hub Info
|
||||
|
||||
```python
|
||||
with producer:
|
||||
info = producer.get_eventhub_properties()
|
||||
print(f"Name: {info['name']}")
|
||||
print(f"Partitions: {info['partition_ids']}")
|
||||
|
||||
for partition_id in info['partition_ids']:
|
||||
partition_info = producer.get_partition_properties(partition_id)
|
||||
print(f"Partition {partition_id}: {partition_info['last_enqueued_sequence_number']}")
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use batches** for sending multiple events
|
||||
2. **Use checkpoint store** in production for reliable processing
|
||||
3. **Use async client** for high-throughput scenarios
|
||||
4. **Use partition keys** for ordered delivery within a partition
|
||||
5. **Handle batch size limits** — catch ValueError when batch is full
|
||||
6. **Use context managers** (`with`/`async with`) for proper cleanup
|
||||
7. **Set appropriate consumer groups** for different applications
|
||||
|
||||
## Reference Files
|
||||
|
||||
| File | Contents |
|
||||
|------|----------|
|
||||
| [references/checkpointing.md](references/checkpointing.md) | Checkpoint store patterns, blob checkpointing, checkpoint strategies |
|
||||
| [references/partitions.md](references/partitions.md) | Partition management, load balancing, starting positions |
|
||||
| [scripts/setup_consumer.py](scripts/setup_consumer.py) | CLI for Event Hub info, consumer setup, and event sending/receiving |
|
||||
267
skills/official/microsoft/python/messaging/servicebus/SKILL.md
Normal file
267
skills/official/microsoft/python/messaging/servicebus/SKILL.md
Normal file
@@ -0,0 +1,267 @@
|
||||
---
|
||||
name: azure-servicebus-py
|
||||
description: |
|
||||
Azure Service Bus SDK for Python messaging. Use for queues, topics, subscriptions, and enterprise messaging patterns.
|
||||
Triggers: "service bus", "ServiceBusClient", "queue", "topic", "subscription", "message broker".
|
||||
package: azure-servicebus
|
||||
---
|
||||
|
||||
# Azure Service Bus SDK for Python
|
||||
|
||||
Enterprise messaging for reliable cloud communication with queues and pub/sub topics.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-servicebus azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
SERVICEBUS_FULLY_QUALIFIED_NAMESPACE=<namespace>.servicebus.windows.net
|
||||
SERVICEBUS_QUEUE_NAME=myqueue
|
||||
SERVICEBUS_TOPIC_NAME=mytopic
|
||||
SERVICEBUS_SUBSCRIPTION_NAME=mysubscription
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.servicebus import ServiceBusClient
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
namespace = "<namespace>.servicebus.windows.net"
|
||||
|
||||
client = ServiceBusClient(
|
||||
fully_qualified_namespace=namespace,
|
||||
credential=credential
|
||||
)
|
||||
```
|
||||
|
||||
## Client Types
|
||||
|
||||
| Client | Purpose | Get From |
|
||||
|--------|---------|----------|
|
||||
| `ServiceBusClient` | Connection management | Direct instantiation |
|
||||
| `ServiceBusSender` | Send messages | `client.get_queue_sender()` / `get_topic_sender()` |
|
||||
| `ServiceBusReceiver` | Receive messages | `client.get_queue_receiver()` / `get_subscription_receiver()` |
|
||||
|
||||
## Send Messages (Async)
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
from azure.servicebus.aio import ServiceBusClient
|
||||
from azure.servicebus import ServiceBusMessage
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def send_messages():
|
||||
credential = DefaultAzureCredential()
|
||||
|
||||
async with ServiceBusClient(
|
||||
fully_qualified_namespace="<namespace>.servicebus.windows.net",
|
||||
credential=credential
|
||||
) as client:
|
||||
sender = client.get_queue_sender(queue_name="myqueue")
|
||||
|
||||
async with sender:
|
||||
# Single message
|
||||
message = ServiceBusMessage("Hello, Service Bus!")
|
||||
await sender.send_messages(message)
|
||||
|
||||
# Batch of messages
|
||||
messages = [ServiceBusMessage(f"Message {i}") for i in range(10)]
|
||||
await sender.send_messages(messages)
|
||||
|
||||
# Message batch (for size control)
|
||||
batch = await sender.create_message_batch()
|
||||
for i in range(100):
|
||||
try:
|
||||
batch.add_message(ServiceBusMessage(f"Batch message {i}"))
|
||||
except ValueError: # Batch full
|
||||
await sender.send_messages(batch)
|
||||
batch = await sender.create_message_batch()
|
||||
batch.add_message(ServiceBusMessage(f"Batch message {i}"))
|
||||
await sender.send_messages(batch)
|
||||
|
||||
asyncio.run(send_messages())
|
||||
```
|
||||
|
||||
## Receive Messages (Async)
|
||||
|
||||
```python
|
||||
async def receive_messages():
|
||||
credential = DefaultAzureCredential()
|
||||
|
||||
async with ServiceBusClient(
|
||||
fully_qualified_namespace="<namespace>.servicebus.windows.net",
|
||||
credential=credential
|
||||
) as client:
|
||||
receiver = client.get_queue_receiver(queue_name="myqueue")
|
||||
|
||||
async with receiver:
|
||||
# Receive batch
|
||||
messages = await receiver.receive_messages(
|
||||
max_message_count=10,
|
||||
max_wait_time=5 # seconds
|
||||
)
|
||||
|
||||
for msg in messages:
|
||||
print(f"Received: {str(msg)}")
|
||||
await receiver.complete_message(msg) # Remove from queue
|
||||
|
||||
asyncio.run(receive_messages())
|
||||
```
|
||||
|
||||
## Receive Modes
|
||||
|
||||
| Mode | Behavior | Use Case |
|
||||
|------|----------|----------|
|
||||
| `PEEK_LOCK` (default) | Message locked, must complete/abandon | Reliable processing |
|
||||
| `RECEIVE_AND_DELETE` | Removed immediately on receive | At-most-once delivery |
|
||||
|
||||
```python
|
||||
from azure.servicebus import ServiceBusReceiveMode
|
||||
|
||||
receiver = client.get_queue_receiver(
|
||||
queue_name="myqueue",
|
||||
receive_mode=ServiceBusReceiveMode.RECEIVE_AND_DELETE
|
||||
)
|
||||
```
|
||||
|
||||
## Message Settlement
|
||||
|
||||
```python
|
||||
async with receiver:
|
||||
messages = await receiver.receive_messages(max_message_count=1)
|
||||
|
||||
for msg in messages:
|
||||
try:
|
||||
# Process message...
|
||||
await receiver.complete_message(msg) # Success - remove from queue
|
||||
except ProcessingError:
|
||||
await receiver.abandon_message(msg) # Retry later
|
||||
except PermanentError:
|
||||
await receiver.dead_letter_message(
|
||||
msg,
|
||||
reason="ProcessingFailed",
|
||||
error_description="Could not process"
|
||||
)
|
||||
```
|
||||
|
||||
| Action | Effect |
|
||||
|--------|--------|
|
||||
| `complete_message()` | Remove from queue (success) |
|
||||
| `abandon_message()` | Release lock, retry immediately |
|
||||
| `dead_letter_message()` | Move to dead-letter queue |
|
||||
| `defer_message()` | Set aside, receive by sequence number |
|
||||
|
||||
## Topics and Subscriptions
|
||||
|
||||
```python
|
||||
# Send to topic
|
||||
sender = client.get_topic_sender(topic_name="mytopic")
|
||||
async with sender:
|
||||
await sender.send_messages(ServiceBusMessage("Topic message"))
|
||||
|
||||
# Receive from subscription
|
||||
receiver = client.get_subscription_receiver(
|
||||
topic_name="mytopic",
|
||||
subscription_name="mysubscription"
|
||||
)
|
||||
async with receiver:
|
||||
messages = await receiver.receive_messages(max_message_count=10)
|
||||
```
|
||||
|
||||
## Sessions (FIFO)
|
||||
|
||||
```python
|
||||
# Send with session
|
||||
message = ServiceBusMessage("Session message")
|
||||
message.session_id = "order-123"
|
||||
await sender.send_messages(message)
|
||||
|
||||
# Receive from specific session
|
||||
receiver = client.get_queue_receiver(
|
||||
queue_name="session-queue",
|
||||
session_id="order-123"
|
||||
)
|
||||
|
||||
# Receive from next available session
|
||||
from azure.servicebus import NEXT_AVAILABLE_SESSION
|
||||
receiver = client.get_queue_receiver(
|
||||
queue_name="session-queue",
|
||||
session_id=NEXT_AVAILABLE_SESSION
|
||||
)
|
||||
```
|
||||
|
||||
## Scheduled Messages
|
||||
|
||||
```python
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
message = ServiceBusMessage("Scheduled message")
|
||||
scheduled_time = datetime.now(timezone.utc) + timedelta(minutes=10)
|
||||
|
||||
# Schedule message
|
||||
sequence_number = await sender.schedule_messages(message, scheduled_time)
|
||||
|
||||
# Cancel scheduled message
|
||||
await sender.cancel_scheduled_messages(sequence_number)
|
||||
```
|
||||
|
||||
## Dead-Letter Queue
|
||||
|
||||
```python
|
||||
from azure.servicebus import ServiceBusSubQueue
|
||||
|
||||
# Receive from dead-letter queue
|
||||
dlq_receiver = client.get_queue_receiver(
|
||||
queue_name="myqueue",
|
||||
sub_queue=ServiceBusSubQueue.DEAD_LETTER
|
||||
)
|
||||
|
||||
async with dlq_receiver:
|
||||
messages = await dlq_receiver.receive_messages(max_message_count=10)
|
||||
for msg in messages:
|
||||
print(f"Dead-lettered: {msg.dead_letter_reason}")
|
||||
await dlq_receiver.complete_message(msg)
|
||||
```
|
||||
|
||||
## Sync Client (for simple scripts)
|
||||
|
||||
```python
|
||||
from azure.servicebus import ServiceBusClient, ServiceBusMessage
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
with ServiceBusClient(
|
||||
fully_qualified_namespace="<namespace>.servicebus.windows.net",
|
||||
credential=DefaultAzureCredential()
|
||||
) as client:
|
||||
with client.get_queue_sender("myqueue") as sender:
|
||||
sender.send_messages(ServiceBusMessage("Sync message"))
|
||||
|
||||
with client.get_queue_receiver("myqueue") as receiver:
|
||||
for msg in receiver:
|
||||
print(str(msg))
|
||||
receiver.complete_message(msg)
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use async client** for production workloads
|
||||
2. **Use context managers** (`async with`) for proper cleanup
|
||||
3. **Complete messages** after successful processing
|
||||
4. **Use dead-letter queue** for poison messages
|
||||
5. **Use sessions** for ordered, FIFO processing
|
||||
6. **Use message batches** for high-throughput scenarios
|
||||
7. **Set `max_wait_time`** to avoid infinite blocking
|
||||
|
||||
## Reference Files
|
||||
|
||||
| File | Contents |
|
||||
|------|----------|
|
||||
| [references/patterns.md](references/patterns.md) | Competing consumers, sessions, retry patterns, request-response, transactions |
|
||||
| [references/dead-letter.md](references/dead-letter.md) | DLQ handling, poison messages, reprocessing strategies |
|
||||
| [scripts/setup_servicebus.py](scripts/setup_servicebus.py) | CLI for queue/topic/subscription management and DLQ monitoring |
|
||||
@@ -0,0 +1,245 @@
|
||||
---
|
||||
name: azure-messaging-webpubsubservice-py
|
||||
description: |
|
||||
Azure Web PubSub Service SDK for Python. Use for real-time messaging, WebSocket connections, and pub/sub patterns.
|
||||
Triggers: "azure-messaging-webpubsubservice", "WebPubSubServiceClient", "real-time", "WebSocket", "pub/sub".
|
||||
package: azure-messaging-webpubsubservice
|
||||
---
|
||||
|
||||
# Azure Web PubSub Service SDK for Python
|
||||
|
||||
Real-time messaging with WebSocket connections at scale.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
# Service SDK (server-side)
|
||||
pip install azure-messaging-webpubsubservice
|
||||
|
||||
# Client SDK (for Python WebSocket clients)
|
||||
pip install azure-messaging-webpubsubclient
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
AZURE_WEBPUBSUB_CONNECTION_STRING=Endpoint=https://<name>.webpubsub.azure.com;AccessKey=...
|
||||
AZURE_WEBPUBSUB_HUB=my-hub
|
||||
```
|
||||
|
||||
## Service Client (Server-Side)
|
||||
|
||||
### Authentication
|
||||
|
||||
```python
|
||||
from azure.messaging.webpubsubservice import WebPubSubServiceClient
|
||||
|
||||
# Connection string
|
||||
client = WebPubSubServiceClient.from_connection_string(
|
||||
connection_string=os.environ["AZURE_WEBPUBSUB_CONNECTION_STRING"],
|
||||
hub="my-hub"
|
||||
)
|
||||
|
||||
# Entra ID
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
client = WebPubSubServiceClient(
|
||||
endpoint="https://<name>.webpubsub.azure.com",
|
||||
hub="my-hub",
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
```
|
||||
|
||||
### Generate Client Access Token
|
||||
|
||||
```python
|
||||
# Token for anonymous user
|
||||
token = client.get_client_access_token()
|
||||
print(f"URL: {token['url']}")
|
||||
|
||||
# Token with user ID
|
||||
token = client.get_client_access_token(
|
||||
user_id="user123",
|
||||
roles=["webpubsub.sendToGroup", "webpubsub.joinLeaveGroup"]
|
||||
)
|
||||
|
||||
# Token with groups
|
||||
token = client.get_client_access_token(
|
||||
user_id="user123",
|
||||
groups=["group1", "group2"]
|
||||
)
|
||||
```
|
||||
|
||||
### Send to All Clients
|
||||
|
||||
```python
|
||||
# Send text
|
||||
client.send_to_all(message="Hello everyone!", content_type="text/plain")
|
||||
|
||||
# Send JSON
|
||||
client.send_to_all(
|
||||
message={"type": "notification", "data": "Hello"},
|
||||
content_type="application/json"
|
||||
)
|
||||
```
|
||||
|
||||
### Send to User
|
||||
|
||||
```python
|
||||
client.send_to_user(
|
||||
user_id="user123",
|
||||
message="Hello user!",
|
||||
content_type="text/plain"
|
||||
)
|
||||
```
|
||||
|
||||
### Send to Group
|
||||
|
||||
```python
|
||||
client.send_to_group(
|
||||
group="my-group",
|
||||
message="Hello group!",
|
||||
content_type="text/plain"
|
||||
)
|
||||
```
|
||||
|
||||
### Send to Connection
|
||||
|
||||
```python
|
||||
client.send_to_connection(
|
||||
connection_id="abc123",
|
||||
message="Hello connection!",
|
||||
content_type="text/plain"
|
||||
)
|
||||
```
|
||||
|
||||
### Group Management
|
||||
|
||||
```python
|
||||
# Add user to group
|
||||
client.add_user_to_group(group="my-group", user_id="user123")
|
||||
|
||||
# Remove user from group
|
||||
client.remove_user_from_group(group="my-group", user_id="user123")
|
||||
|
||||
# Add connection to group
|
||||
client.add_connection_to_group(group="my-group", connection_id="abc123")
|
||||
|
||||
# Remove connection from group
|
||||
client.remove_connection_from_group(group="my-group", connection_id="abc123")
|
||||
```
|
||||
|
||||
### Connection Management
|
||||
|
||||
```python
|
||||
# Check if connection exists
|
||||
exists = client.connection_exists(connection_id="abc123")
|
||||
|
||||
# Check if user has connections
|
||||
exists = client.user_exists(user_id="user123")
|
||||
|
||||
# Check if group has connections
|
||||
exists = client.group_exists(group="my-group")
|
||||
|
||||
# Close connection
|
||||
client.close_connection(connection_id="abc123", reason="Session ended")
|
||||
|
||||
# Close all connections for user
|
||||
client.close_all_connections(user_id="user123")
|
||||
```
|
||||
|
||||
### Grant/Revoke Permissions
|
||||
|
||||
```python
|
||||
from azure.messaging.webpubsubservice import WebPubSubServiceClient
|
||||
|
||||
# Grant permission
|
||||
client.grant_permission(
|
||||
permission="joinLeaveGroup",
|
||||
connection_id="abc123",
|
||||
target_name="my-group"
|
||||
)
|
||||
|
||||
# Revoke permission
|
||||
client.revoke_permission(
|
||||
permission="joinLeaveGroup",
|
||||
connection_id="abc123",
|
||||
target_name="my-group"
|
||||
)
|
||||
|
||||
# Check permission
|
||||
has_permission = client.check_permission(
|
||||
permission="joinLeaveGroup",
|
||||
connection_id="abc123",
|
||||
target_name="my-group"
|
||||
)
|
||||
```
|
||||
|
||||
## Client SDK (Python WebSocket Client)
|
||||
|
||||
```python
|
||||
from azure.messaging.webpubsubclient import WebPubSubClient
|
||||
|
||||
client = WebPubSubClient(credential=token["url"])
|
||||
|
||||
# Event handlers
|
||||
@client.on("connected")
|
||||
def on_connected(e):
|
||||
print(f"Connected: {e.connection_id}")
|
||||
|
||||
@client.on("server-message")
|
||||
def on_message(e):
|
||||
print(f"Message: {e.data}")
|
||||
|
||||
@client.on("group-message")
|
||||
def on_group_message(e):
|
||||
print(f"Group {e.group}: {e.data}")
|
||||
|
||||
# Connect and send
|
||||
client.open()
|
||||
client.send_to_group("my-group", "Hello from Python!")
|
||||
```
|
||||
|
||||
## Async Service Client
|
||||
|
||||
```python
|
||||
from azure.messaging.webpubsubservice.aio import WebPubSubServiceClient
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def broadcast():
|
||||
credential = DefaultAzureCredential()
|
||||
client = WebPubSubServiceClient(
|
||||
endpoint="https://<name>.webpubsub.azure.com",
|
||||
hub="my-hub",
|
||||
credential=credential
|
||||
)
|
||||
|
||||
await client.send_to_all("Hello async!", content_type="text/plain")
|
||||
|
||||
await client.close()
|
||||
await credential.close()
|
||||
```
|
||||
|
||||
## Client Operations
|
||||
|
||||
| Operation | Description |
|
||||
|-----------|-------------|
|
||||
| `get_client_access_token` | Generate WebSocket connection URL |
|
||||
| `send_to_all` | Broadcast to all connections |
|
||||
| `send_to_user` | Send to specific user |
|
||||
| `send_to_group` | Send to group members |
|
||||
| `send_to_connection` | Send to specific connection |
|
||||
| `add_user_to_group` | Add user to group |
|
||||
| `remove_user_from_group` | Remove user from group |
|
||||
| `close_connection` | Disconnect client |
|
||||
| `connection_exists` | Check connection status |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use roles** to limit client permissions
|
||||
2. **Use groups** for targeted messaging
|
||||
3. **Generate short-lived tokens** for security
|
||||
4. **Use user IDs** to send to users across connections
|
||||
5. **Handle reconnection** in client applications
|
||||
6. **Use JSON** content type for structured data
|
||||
7. **Close connections** gracefully with reasons
|
||||
204
skills/official/microsoft/python/monitoring/ingestion/SKILL.md
Normal file
204
skills/official/microsoft/python/monitoring/ingestion/SKILL.md
Normal file
@@ -0,0 +1,204 @@
|
||||
---
|
||||
name: azure-monitor-ingestion-py
|
||||
description: |
|
||||
Azure Monitor Ingestion SDK for Python. Use for sending custom logs to Log Analytics workspace via Logs Ingestion API.
|
||||
Triggers: "azure-monitor-ingestion", "LogsIngestionClient", "custom logs", "DCR", "data collection rule", "Log Analytics".
|
||||
package: azure-monitor-ingestion
|
||||
---
|
||||
|
||||
# Azure Monitor Ingestion SDK for Python
|
||||
|
||||
Send custom logs to Azure Monitor Log Analytics workspace using the Logs Ingestion API.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-monitor-ingestion
|
||||
pip install azure-identity
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
# Data Collection Endpoint (DCE)
|
||||
AZURE_DCE_ENDPOINT=https://<dce-name>.<region>.ingest.monitor.azure.com
|
||||
|
||||
# Data Collection Rule (DCR) immutable ID
|
||||
AZURE_DCR_RULE_ID=dcr-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||
|
||||
# Stream name from DCR
|
||||
AZURE_DCR_STREAM_NAME=Custom-MyTable_CL
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before using this SDK, you need:
|
||||
|
||||
1. **Log Analytics Workspace** — Target for your logs
|
||||
2. **Data Collection Endpoint (DCE)** — Ingestion endpoint
|
||||
3. **Data Collection Rule (DCR)** — Defines schema and destination
|
||||
4. **Custom Table** — In Log Analytics (created via DCR or manually)
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.monitor.ingestion import LogsIngestionClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
import os
|
||||
|
||||
client = LogsIngestionClient(
|
||||
endpoint=os.environ["AZURE_DCE_ENDPOINT"],
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
```
|
||||
|
||||
## Upload Custom Logs
|
||||
|
||||
```python
|
||||
from azure.monitor.ingestion import LogsIngestionClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
import os
|
||||
|
||||
client = LogsIngestionClient(
|
||||
endpoint=os.environ["AZURE_DCE_ENDPOINT"],
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
|
||||
rule_id = os.environ["AZURE_DCR_RULE_ID"]
|
||||
stream_name = os.environ["AZURE_DCR_STREAM_NAME"]
|
||||
|
||||
logs = [
|
||||
{"TimeGenerated": "2024-01-15T10:00:00Z", "Computer": "server1", "Message": "Application started"},
|
||||
{"TimeGenerated": "2024-01-15T10:01:00Z", "Computer": "server1", "Message": "Processing request"},
|
||||
{"TimeGenerated": "2024-01-15T10:02:00Z", "Computer": "server2", "Message": "Connection established"}
|
||||
]
|
||||
|
||||
client.upload(rule_id=rule_id, stream_name=stream_name, logs=logs)
|
||||
```
|
||||
|
||||
## Upload from JSON File
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
with open("logs.json", "r") as f:
|
||||
logs = json.load(f)
|
||||
|
||||
client.upload(rule_id=rule_id, stream_name=stream_name, logs=logs)
|
||||
```
|
||||
|
||||
## Custom Error Handling
|
||||
|
||||
Handle partial failures with a callback:
|
||||
|
||||
```python
|
||||
failed_logs = []
|
||||
|
||||
def on_error(error):
|
||||
print(f"Upload failed: {error.error}")
|
||||
failed_logs.extend(error.failed_logs)
|
||||
|
||||
client.upload(
|
||||
rule_id=rule_id,
|
||||
stream_name=stream_name,
|
||||
logs=logs,
|
||||
on_error=on_error
|
||||
)
|
||||
|
||||
# Retry failed logs
|
||||
if failed_logs:
|
||||
print(f"Retrying {len(failed_logs)} failed logs...")
|
||||
client.upload(rule_id=rule_id, stream_name=stream_name, logs=failed_logs)
|
||||
```
|
||||
|
||||
## Ignore Errors
|
||||
|
||||
```python
|
||||
def ignore_errors(error):
|
||||
pass # Silently ignore upload failures
|
||||
|
||||
client.upload(
|
||||
rule_id=rule_id,
|
||||
stream_name=stream_name,
|
||||
logs=logs,
|
||||
on_error=ignore_errors
|
||||
)
|
||||
```
|
||||
|
||||
## Async Client
|
||||
|
||||
```python
|
||||
import asyncio
|
||||
from azure.monitor.ingestion.aio import LogsIngestionClient
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def upload_logs():
|
||||
async with LogsIngestionClient(
|
||||
endpoint=endpoint,
|
||||
credential=DefaultAzureCredential()
|
||||
) as client:
|
||||
await client.upload(
|
||||
rule_id=rule_id,
|
||||
stream_name=stream_name,
|
||||
logs=logs
|
||||
)
|
||||
|
||||
asyncio.run(upload_logs())
|
||||
```
|
||||
|
||||
## Sovereign Clouds
|
||||
|
||||
```python
|
||||
from azure.identity import AzureAuthorityHosts, DefaultAzureCredential
|
||||
from azure.monitor.ingestion import LogsIngestionClient
|
||||
|
||||
# Azure Government
|
||||
credential = DefaultAzureCredential(authority=AzureAuthorityHosts.AZURE_GOVERNMENT)
|
||||
client = LogsIngestionClient(
|
||||
endpoint="https://example.ingest.monitor.azure.us",
|
||||
credential=credential,
|
||||
credential_scopes=["https://monitor.azure.us/.default"]
|
||||
)
|
||||
```
|
||||
|
||||
## Batching Behavior
|
||||
|
||||
The SDK automatically:
|
||||
- Splits logs into chunks of 1MB or less
|
||||
- Compresses each chunk with gzip
|
||||
- Uploads chunks in parallel
|
||||
|
||||
No manual batching needed for large log sets.
|
||||
|
||||
## Client Types
|
||||
|
||||
| Client | Purpose |
|
||||
|--------|---------|
|
||||
| `LogsIngestionClient` | Sync client for uploading logs |
|
||||
| `LogsIngestionClient` (aio) | Async client for uploading logs |
|
||||
|
||||
## Key Concepts
|
||||
|
||||
| Concept | Description |
|
||||
|---------|-------------|
|
||||
| **DCE** | Data Collection Endpoint — ingestion URL |
|
||||
| **DCR** | Data Collection Rule — defines schema, transformations, destination |
|
||||
| **Stream** | Named data flow within a DCR |
|
||||
| **Custom Table** | Target table in Log Analytics (ends with `_CL`) |
|
||||
|
||||
## DCR Stream Name Format
|
||||
|
||||
Stream names follow patterns:
|
||||
- `Custom-<TableName>_CL` — For custom tables
|
||||
- `Microsoft-<TableName>` — For built-in tables
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use DefaultAzureCredential** for authentication
|
||||
2. **Handle errors gracefully** — use `on_error` callback for partial failures
|
||||
3. **Include TimeGenerated** — Required field for all logs
|
||||
4. **Match DCR schema** — Log fields must match DCR column definitions
|
||||
5. **Use async client** for high-throughput scenarios
|
||||
6. **Batch uploads** — SDK handles batching, but send reasonable chunks
|
||||
7. **Monitor ingestion** — Check Log Analytics for ingestion status
|
||||
8. **Use context manager** — Ensures proper client cleanup
|
||||
@@ -0,0 +1,207 @@
|
||||
---
|
||||
name: azure-monitor-opentelemetry-exporter-py
|
||||
description: |
|
||||
Azure Monitor OpenTelemetry Exporter for Python. Use for low-level OpenTelemetry export to Application Insights.
|
||||
Triggers: "azure-monitor-opentelemetry-exporter", "AzureMonitorTraceExporter", "AzureMonitorMetricExporter", "AzureMonitorLogExporter".
|
||||
package: azure-monitor-opentelemetry-exporter
|
||||
---
|
||||
|
||||
# Azure Monitor OpenTelemetry Exporter for Python
|
||||
|
||||
Low-level exporter for sending OpenTelemetry traces, metrics, and logs to Application Insights.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-monitor-opentelemetry-exporter
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
APPLICATIONINSIGHTS_CONNECTION_STRING=InstrumentationKey=xxx;IngestionEndpoint=https://xxx.in.applicationinsights.azure.com/
|
||||
```
|
||||
|
||||
## When to Use
|
||||
|
||||
| Scenario | Use |
|
||||
|----------|-----|
|
||||
| Quick setup, auto-instrumentation | `azure-monitor-opentelemetry` (distro) |
|
||||
| Custom OpenTelemetry pipeline | `azure-monitor-opentelemetry-exporter` (this) |
|
||||
| Fine-grained control over telemetry | `azure-monitor-opentelemetry-exporter` (this) |
|
||||
|
||||
## Trace Exporter
|
||||
|
||||
```python
|
||||
from opentelemetry import trace
|
||||
from opentelemetry.sdk.trace import TracerProvider
|
||||
from opentelemetry.sdk.trace.export import BatchSpanProcessor
|
||||
from azure.monitor.opentelemetry.exporter import AzureMonitorTraceExporter
|
||||
|
||||
# Create exporter
|
||||
exporter = AzureMonitorTraceExporter(
|
||||
connection_string="InstrumentationKey=xxx;..."
|
||||
)
|
||||
|
||||
# Configure tracer provider
|
||||
trace.set_tracer_provider(TracerProvider())
|
||||
trace.get_tracer_provider().add_span_processor(
|
||||
BatchSpanProcessor(exporter)
|
||||
)
|
||||
|
||||
# Use tracer
|
||||
tracer = trace.get_tracer(__name__)
|
||||
with tracer.start_as_current_span("my-span"):
|
||||
print("Hello, World!")
|
||||
```
|
||||
|
||||
## Metric Exporter
|
||||
|
||||
```python
|
||||
from opentelemetry import metrics
|
||||
from opentelemetry.sdk.metrics import MeterProvider
|
||||
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
|
||||
from azure.monitor.opentelemetry.exporter import AzureMonitorMetricExporter
|
||||
|
||||
# Create exporter
|
||||
exporter = AzureMonitorMetricExporter(
|
||||
connection_string="InstrumentationKey=xxx;..."
|
||||
)
|
||||
|
||||
# Configure meter provider
|
||||
reader = PeriodicExportingMetricReader(exporter, export_interval_millis=60000)
|
||||
metrics.set_meter_provider(MeterProvider(metric_readers=[reader]))
|
||||
|
||||
# Use meter
|
||||
meter = metrics.get_meter(__name__)
|
||||
counter = meter.create_counter("requests_total")
|
||||
counter.add(1, {"route": "/api/users"})
|
||||
```
|
||||
|
||||
## Log Exporter
|
||||
|
||||
```python
|
||||
import logging
|
||||
from opentelemetry._logs import set_logger_provider
|
||||
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
|
||||
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
|
||||
from azure.monitor.opentelemetry.exporter import AzureMonitorLogExporter
|
||||
|
||||
# Create exporter
|
||||
exporter = AzureMonitorLogExporter(
|
||||
connection_string="InstrumentationKey=xxx;..."
|
||||
)
|
||||
|
||||
# Configure logger provider
|
||||
logger_provider = LoggerProvider()
|
||||
logger_provider.add_log_record_processor(BatchLogRecordProcessor(exporter))
|
||||
set_logger_provider(logger_provider)
|
||||
|
||||
# Add handler to Python logging
|
||||
handler = LoggingHandler(level=logging.INFO, logger_provider=logger_provider)
|
||||
logging.getLogger().addHandler(handler)
|
||||
|
||||
# Use logging
|
||||
logger = logging.getLogger(__name__)
|
||||
logger.info("This will be sent to Application Insights")
|
||||
```
|
||||
|
||||
## From Environment Variable
|
||||
|
||||
Exporters read `APPLICATIONINSIGHTS_CONNECTION_STRING` automatically:
|
||||
|
||||
```python
|
||||
from azure.monitor.opentelemetry.exporter import AzureMonitorTraceExporter
|
||||
|
||||
# Connection string from environment
|
||||
exporter = AzureMonitorTraceExporter()
|
||||
```
|
||||
|
||||
## Azure AD Authentication
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.monitor.opentelemetry.exporter import AzureMonitorTraceExporter
|
||||
|
||||
exporter = AzureMonitorTraceExporter(
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
```
|
||||
|
||||
## Sampling
|
||||
|
||||
Use `ApplicationInsightsSampler` for consistent sampling:
|
||||
|
||||
```python
|
||||
from opentelemetry.sdk.trace import TracerProvider
|
||||
from opentelemetry.sdk.trace.sampling import ParentBasedTraceIdRatio
|
||||
from azure.monitor.opentelemetry.exporter import ApplicationInsightsSampler
|
||||
|
||||
# Sample 10% of traces
|
||||
sampler = ApplicationInsightsSampler(sampling_ratio=0.1)
|
||||
|
||||
trace.set_tracer_provider(TracerProvider(sampler=sampler))
|
||||
```
|
||||
|
||||
## Offline Storage
|
||||
|
||||
Configure offline storage for retry:
|
||||
|
||||
```python
|
||||
from azure.monitor.opentelemetry.exporter import AzureMonitorTraceExporter
|
||||
|
||||
exporter = AzureMonitorTraceExporter(
|
||||
connection_string="...",
|
||||
storage_directory="/path/to/storage", # Custom storage path
|
||||
disable_offline_storage=False # Enable retry (default)
|
||||
)
|
||||
```
|
||||
|
||||
## Disable Offline Storage
|
||||
|
||||
```python
|
||||
exporter = AzureMonitorTraceExporter(
|
||||
connection_string="...",
|
||||
disable_offline_storage=True # No retry on failure
|
||||
)
|
||||
```
|
||||
|
||||
## Sovereign Clouds
|
||||
|
||||
```python
|
||||
from azure.identity import AzureAuthorityHosts, DefaultAzureCredential
|
||||
from azure.monitor.opentelemetry.exporter import AzureMonitorTraceExporter
|
||||
|
||||
# Azure Government
|
||||
credential = DefaultAzureCredential(authority=AzureAuthorityHosts.AZURE_GOVERNMENT)
|
||||
exporter = AzureMonitorTraceExporter(
|
||||
connection_string="InstrumentationKey=xxx;IngestionEndpoint=https://xxx.in.applicationinsights.azure.us/",
|
||||
credential=credential
|
||||
)
|
||||
```
|
||||
|
||||
## Exporter Types
|
||||
|
||||
| Exporter | Telemetry Type | Application Insights Table |
|
||||
|----------|---------------|---------------------------|
|
||||
| `AzureMonitorTraceExporter` | Traces/Spans | requests, dependencies, exceptions |
|
||||
| `AzureMonitorMetricExporter` | Metrics | customMetrics, performanceCounters |
|
||||
| `AzureMonitorLogExporter` | Logs | traces, customEvents |
|
||||
|
||||
## Configuration Options
|
||||
|
||||
| Parameter | Description | Default |
|
||||
|-----------|-------------|---------|
|
||||
| `connection_string` | Application Insights connection string | From env var |
|
||||
| `credential` | Azure credential for AAD auth | None |
|
||||
| `disable_offline_storage` | Disable retry storage | False |
|
||||
| `storage_directory` | Custom storage path | Temp directory |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use BatchSpanProcessor** for production (not SimpleSpanProcessor)
|
||||
2. **Use ApplicationInsightsSampler** for consistent sampling across services
|
||||
3. **Enable offline storage** for reliability in production
|
||||
4. **Use AAD authentication** instead of instrumentation keys
|
||||
5. **Set export intervals** appropriate for your workload
|
||||
6. **Use the distro** (`azure-monitor-opentelemetry`) unless you need custom pipelines
|
||||
@@ -0,0 +1,224 @@
|
||||
---
|
||||
name: azure-monitor-opentelemetry-py
|
||||
description: |
|
||||
Azure Monitor OpenTelemetry Distro for Python. Use for one-line Application Insights setup with auto-instrumentation.
|
||||
Triggers: "azure-monitor-opentelemetry", "configure_azure_monitor", "Application Insights", "OpenTelemetry distro", "auto-instrumentation".
|
||||
package: azure-monitor-opentelemetry
|
||||
---
|
||||
|
||||
# Azure Monitor OpenTelemetry Distro for Python
|
||||
|
||||
One-line setup for Application Insights with OpenTelemetry auto-instrumentation.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-monitor-opentelemetry
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
APPLICATIONINSIGHTS_CONNECTION_STRING=InstrumentationKey=xxx;IngestionEndpoint=https://xxx.in.applicationinsights.azure.com/
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
|
||||
# One-line setup - reads connection string from environment
|
||||
configure_azure_monitor()
|
||||
|
||||
# Your application code...
|
||||
```
|
||||
|
||||
## Explicit Configuration
|
||||
|
||||
```python
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
|
||||
configure_azure_monitor(
|
||||
connection_string="InstrumentationKey=xxx;IngestionEndpoint=https://xxx.in.applicationinsights.azure.com/"
|
||||
)
|
||||
```
|
||||
|
||||
## With Flask
|
||||
|
||||
```python
|
||||
from flask import Flask
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
|
||||
configure_azure_monitor()
|
||||
|
||||
app = Flask(__name__)
|
||||
|
||||
@app.route("/")
|
||||
def hello():
|
||||
return "Hello, World!"
|
||||
|
||||
if __name__ == "__main__":
|
||||
app.run()
|
||||
```
|
||||
|
||||
## With Django
|
||||
|
||||
```python
|
||||
# settings.py
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
|
||||
configure_azure_monitor()
|
||||
|
||||
# Django settings...
|
||||
```
|
||||
|
||||
## With FastAPI
|
||||
|
||||
```python
|
||||
from fastapi import FastAPI
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
|
||||
configure_azure_monitor()
|
||||
|
||||
app = FastAPI()
|
||||
|
||||
@app.get("/")
|
||||
async def root():
|
||||
return {"message": "Hello World"}
|
||||
```
|
||||
|
||||
## Custom Traces
|
||||
|
||||
```python
|
||||
from opentelemetry import trace
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
|
||||
configure_azure_monitor()
|
||||
|
||||
tracer = trace.get_tracer(__name__)
|
||||
|
||||
with tracer.start_as_current_span("my-operation") as span:
|
||||
span.set_attribute("custom.attribute", "value")
|
||||
# Do work...
|
||||
```
|
||||
|
||||
## Custom Metrics
|
||||
|
||||
```python
|
||||
from opentelemetry import metrics
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
|
||||
configure_azure_monitor()
|
||||
|
||||
meter = metrics.get_meter(__name__)
|
||||
counter = meter.create_counter("my_counter")
|
||||
|
||||
counter.add(1, {"dimension": "value"})
|
||||
```
|
||||
|
||||
## Custom Logs
|
||||
|
||||
```python
|
||||
import logging
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
|
||||
configure_azure_monitor()
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
logger.setLevel(logging.INFO)
|
||||
|
||||
logger.info("This will appear in Application Insights")
|
||||
logger.error("Errors are captured too", exc_info=True)
|
||||
```
|
||||
|
||||
## Sampling
|
||||
|
||||
```python
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
|
||||
# Sample 10% of requests
|
||||
configure_azure_monitor(
|
||||
sampling_ratio=0.1
|
||||
)
|
||||
```
|
||||
|
||||
## Cloud Role Name
|
||||
|
||||
Set cloud role name for Application Map:
|
||||
|
||||
```python
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
from opentelemetry.sdk.resources import Resource, SERVICE_NAME
|
||||
|
||||
configure_azure_monitor(
|
||||
resource=Resource.create({SERVICE_NAME: "my-service-name"})
|
||||
)
|
||||
```
|
||||
|
||||
## Disable Specific Instrumentations
|
||||
|
||||
```python
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
|
||||
configure_azure_monitor(
|
||||
instrumentations=["flask", "requests"] # Only enable these
|
||||
)
|
||||
```
|
||||
|
||||
## Enable Live Metrics
|
||||
|
||||
```python
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
|
||||
configure_azure_monitor(
|
||||
enable_live_metrics=True
|
||||
)
|
||||
```
|
||||
|
||||
## Azure AD Authentication
|
||||
|
||||
```python
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
configure_azure_monitor(
|
||||
credential=DefaultAzureCredential()
|
||||
)
|
||||
```
|
||||
|
||||
## Auto-Instrumentations Included
|
||||
|
||||
| Library | Telemetry Type |
|
||||
|---------|---------------|
|
||||
| Flask | Traces |
|
||||
| Django | Traces |
|
||||
| FastAPI | Traces |
|
||||
| Requests | Traces |
|
||||
| urllib3 | Traces |
|
||||
| httpx | Traces |
|
||||
| aiohttp | Traces |
|
||||
| psycopg2 | Traces |
|
||||
| pymysql | Traces |
|
||||
| pymongo | Traces |
|
||||
| redis | Traces |
|
||||
|
||||
## Configuration Options
|
||||
|
||||
| Parameter | Description | Default |
|
||||
|-----------|-------------|---------|
|
||||
| `connection_string` | Application Insights connection string | From env var |
|
||||
| `credential` | Azure credential for AAD auth | None |
|
||||
| `sampling_ratio` | Sampling rate (0.0 to 1.0) | 1.0 |
|
||||
| `resource` | OpenTelemetry Resource | Auto-detected |
|
||||
| `instrumentations` | List of instrumentations to enable | All |
|
||||
| `enable_live_metrics` | Enable Live Metrics stream | False |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Call configure_azure_monitor() early** — Before importing instrumented libraries
|
||||
2. **Use environment variables** for connection string in production
|
||||
3. **Set cloud role name** for multi-service applications
|
||||
4. **Enable sampling** in high-traffic applications
|
||||
5. **Use structured logging** for better log analytics queries
|
||||
6. **Add custom attributes** to spans for better debugging
|
||||
7. **Use AAD authentication** for production workloads
|
||||
252
skills/official/microsoft/python/monitoring/query/SKILL.md
Normal file
252
skills/official/microsoft/python/monitoring/query/SKILL.md
Normal file
@@ -0,0 +1,252 @@
|
||||
---
|
||||
name: azure-monitor-query-py
|
||||
description: |
|
||||
Azure Monitor Query SDK for Python. Use for querying Log Analytics workspaces and Azure Monitor metrics.
|
||||
Triggers: "azure-monitor-query", "LogsQueryClient", "MetricsQueryClient", "Log Analytics", "Kusto queries", "Azure metrics".
|
||||
package: azure-monitor-query
|
||||
---
|
||||
|
||||
# Azure Monitor Query SDK for Python
|
||||
|
||||
Query logs and metrics from Azure Monitor and Log Analytics workspaces.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install azure-monitor-query
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
# Log Analytics
|
||||
AZURE_LOG_ANALYTICS_WORKSPACE_ID=<workspace-id>
|
||||
|
||||
# Metrics
|
||||
AZURE_METRICS_RESOURCE_URI=/subscriptions/<sub>/resourceGroups/<rg>/providers/<provider>/<type>/<name>
|
||||
```
|
||||
|
||||
## Authentication
|
||||
|
||||
```python
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
credential = DefaultAzureCredential()
|
||||
```
|
||||
|
||||
## Logs Query Client
|
||||
|
||||
### Basic Query
|
||||
|
||||
```python
|
||||
from azure.monitor.query import LogsQueryClient
|
||||
from datetime import timedelta
|
||||
|
||||
client = LogsQueryClient(credential)
|
||||
|
||||
query = """
|
||||
AppRequests
|
||||
| where TimeGenerated > ago(1h)
|
||||
| summarize count() by bin(TimeGenerated, 5m), ResultCode
|
||||
| order by TimeGenerated desc
|
||||
"""
|
||||
|
||||
response = client.query_workspace(
|
||||
workspace_id=os.environ["AZURE_LOG_ANALYTICS_WORKSPACE_ID"],
|
||||
query=query,
|
||||
timespan=timedelta(hours=1)
|
||||
)
|
||||
|
||||
for table in response.tables:
|
||||
for row in table.rows:
|
||||
print(row)
|
||||
```
|
||||
|
||||
### Query with Time Range
|
||||
|
||||
```python
|
||||
from datetime import datetime, timezone
|
||||
|
||||
response = client.query_workspace(
|
||||
workspace_id=workspace_id,
|
||||
query="AppRequests | take 10",
|
||||
timespan=(
|
||||
datetime(2024, 1, 1, tzinfo=timezone.utc),
|
||||
datetime(2024, 1, 2, tzinfo=timezone.utc)
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
### Convert to DataFrame
|
||||
|
||||
```python
|
||||
import pandas as pd
|
||||
|
||||
response = client.query_workspace(workspace_id, query, timespan=timedelta(hours=1))
|
||||
|
||||
if response.tables:
|
||||
table = response.tables[0]
|
||||
df = pd.DataFrame(data=table.rows, columns=[col.name for col in table.columns])
|
||||
print(df.head())
|
||||
```
|
||||
|
||||
### Batch Query
|
||||
|
||||
```python
|
||||
from azure.monitor.query import LogsBatchQuery
|
||||
|
||||
queries = [
|
||||
LogsBatchQuery(workspace_id=workspace_id, query="AppRequests | take 5", timespan=timedelta(hours=1)),
|
||||
LogsBatchQuery(workspace_id=workspace_id, query="AppExceptions | take 5", timespan=timedelta(hours=1))
|
||||
]
|
||||
|
||||
responses = client.query_batch(queries)
|
||||
|
||||
for response in responses:
|
||||
if response.tables:
|
||||
print(f"Rows: {len(response.tables[0].rows)}")
|
||||
```
|
||||
|
||||
### Handle Partial Results
|
||||
|
||||
```python
|
||||
from azure.monitor.query import LogsQueryStatus
|
||||
|
||||
response = client.query_workspace(workspace_id, query, timespan=timedelta(hours=24))
|
||||
|
||||
if response.status == LogsQueryStatus.PARTIAL:
|
||||
print(f"Partial results: {response.partial_error}")
|
||||
elif response.status == LogsQueryStatus.FAILURE:
|
||||
print(f"Query failed: {response.partial_error}")
|
||||
```
|
||||
|
||||
## Metrics Query Client
|
||||
|
||||
### Query Resource Metrics
|
||||
|
||||
```python
|
||||
from azure.monitor.query import MetricsQueryClient
|
||||
from datetime import timedelta
|
||||
|
||||
metrics_client = MetricsQueryClient(credential)
|
||||
|
||||
response = metrics_client.query_resource(
|
||||
resource_uri=os.environ["AZURE_METRICS_RESOURCE_URI"],
|
||||
metric_names=["Percentage CPU", "Network In Total"],
|
||||
timespan=timedelta(hours=1),
|
||||
granularity=timedelta(minutes=5)
|
||||
)
|
||||
|
||||
for metric in response.metrics:
|
||||
print(f"{metric.name}:")
|
||||
for time_series in metric.timeseries:
|
||||
for data in time_series.data:
|
||||
print(f" {data.timestamp}: {data.average}")
|
||||
```
|
||||
|
||||
### Aggregations
|
||||
|
||||
```python
|
||||
from azure.monitor.query import MetricAggregationType
|
||||
|
||||
response = metrics_client.query_resource(
|
||||
resource_uri=resource_uri,
|
||||
metric_names=["Requests"],
|
||||
timespan=timedelta(hours=1),
|
||||
aggregations=[
|
||||
MetricAggregationType.AVERAGE,
|
||||
MetricAggregationType.MAXIMUM,
|
||||
MetricAggregationType.MINIMUM,
|
||||
MetricAggregationType.COUNT
|
||||
]
|
||||
)
|
||||
```
|
||||
|
||||
### Filter by Dimension
|
||||
|
||||
```python
|
||||
response = metrics_client.query_resource(
|
||||
resource_uri=resource_uri,
|
||||
metric_names=["Requests"],
|
||||
timespan=timedelta(hours=1),
|
||||
filter="ApiName eq 'GetBlob'"
|
||||
)
|
||||
```
|
||||
|
||||
### List Metric Definitions
|
||||
|
||||
```python
|
||||
definitions = metrics_client.list_metric_definitions(resource_uri)
|
||||
for definition in definitions:
|
||||
print(f"{definition.name}: {definition.unit}")
|
||||
```
|
||||
|
||||
### List Metric Namespaces
|
||||
|
||||
```python
|
||||
namespaces = metrics_client.list_metric_namespaces(resource_uri)
|
||||
for ns in namespaces:
|
||||
print(ns.fully_qualified_namespace)
|
||||
```
|
||||
|
||||
## Async Clients
|
||||
|
||||
```python
|
||||
from azure.monitor.query.aio import LogsQueryClient, MetricsQueryClient
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
|
||||
async def query_logs():
|
||||
credential = DefaultAzureCredential()
|
||||
client = LogsQueryClient(credential)
|
||||
|
||||
response = await client.query_workspace(
|
||||
workspace_id=workspace_id,
|
||||
query="AppRequests | take 10",
|
||||
timespan=timedelta(hours=1)
|
||||
)
|
||||
|
||||
await client.close()
|
||||
await credential.close()
|
||||
return response
|
||||
```
|
||||
|
||||
## Common Kusto Queries
|
||||
|
||||
```kusto
|
||||
// Requests by status code
|
||||
AppRequests
|
||||
| summarize count() by ResultCode
|
||||
| order by count_ desc
|
||||
|
||||
// Exceptions over time
|
||||
AppExceptions
|
||||
| summarize count() by bin(TimeGenerated, 1h)
|
||||
|
||||
// Slow requests
|
||||
AppRequests
|
||||
| where DurationMs > 1000
|
||||
| project TimeGenerated, Name, DurationMs
|
||||
| order by DurationMs desc
|
||||
|
||||
// Top errors
|
||||
AppExceptions
|
||||
| summarize count() by ExceptionType
|
||||
| top 10 by count_
|
||||
```
|
||||
|
||||
## Client Types
|
||||
|
||||
| Client | Purpose |
|
||||
|--------|---------|
|
||||
| `LogsQueryClient` | Query Log Analytics workspaces |
|
||||
| `MetricsQueryClient` | Query Azure Monitor metrics |
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use timedelta** for relative time ranges
|
||||
2. **Handle partial results** for large queries
|
||||
3. **Use batch queries** when running multiple queries
|
||||
4. **Set appropriate granularity** for metrics to reduce data points
|
||||
5. **Convert to DataFrame** for easier data analysis
|
||||
6. **Use aggregations** to summarize metric data
|
||||
7. **Filter by dimensions** to narrow metric results
|
||||
Reference in New Issue
Block a user