Azure OpenAI Document Extracts and Notes

Featured

OVERVIEW

  • Azure OpenAI is a service provided by Microsoft Azure that allows users to access OpenAI’s powerful language models, including the GPT-3, Codex, and Embeddings model series. Users can access the service through REST APIs, Python SDK, or a web-based interface in the Azure OpenAI Studio.
  • Azure OpenAI Service gives customers advanced language AI with OpenAI
    • GPT-4, GPT-3, Codex, and DALL-E
    • Models with the enterprise security and privacy of Azure.
  • Azure OpenAI co-develops the APIs with OpenAI, ensuring compatibility and a smooth transition from one to the other
  • Azure OpenAI Infographic

Comparing Azure OpenAI and OpenAI

  • Enterprise-grade security with role-based access control (RBAC) and private networks
  • Essentially Security, Privacy, and Trust
  • Microsoft values a customer’s privacy and security of data. When using Azure AI services, Microsoft may collect and store data to improve the session experience and supportability of models. However, customer data is anonymized and aggregated to protect individual privacy.
  • Microsoft does not use customer data for fine-tuning or customizing models for individual users.
  • Microsoft Responsible AI Standard (PDF Download)

Responsible AI

  • For building AI systems according to six principles:
    • Fairness and Inclusiveness
      • Make the same recommendations to everyone who has similar symptoms, financial circumstances, or professional qualifications.
    • Reliability and Safety
      • Operate as originally designed, respond safely to unanticipated conditions, and resist harmful manipulation.
    • Privacy and Security
      • Restrict access to resources and operations by user account or group.
      • Restrict incoming and outgoing network communications.
      • Encrypt data in transit and at rest.
      • Scan for vulnerabilities.
      • Apply and audit configuration policies.
      • Microsoft has also created two open-source packages that can enable further implementation of privacy and security principles: SmartNoise and Counterfit
    • Transparency and Accountability
      • The model interpretability component provides multiple or global, local, and model explanations/views into a model’s behavior.
      • The people who design and deploy AI systems must be accountable for how their systems operate.

SECURITY AND PRIVACY

  • Azure OpenAI Service automatically encrypts your data when it’s persisted to the cloud, using FIPS 140-2 compliant 256-bit AES encryption.
  • By default, Microsoft-managed encryption keys are used, but you also have the option to use customer-managed keys (CMK) for greater control over encryption key management.
  • The Files API allows customers to upload their training data stored in Azure Storage, within the same region as the resource and logically isolated with their Azure subscription and API Credentials. Uploaded files can be deleted by the user via the DELETE API operation.
  • With Azure OpenAI, customers get the security capabilities of Microsoft Azure while running the same models as OpenAI. Azure OpenAI offers private networking, regional availability, and responsible AI content filtering.
    • Azure OpenAI Service contains neural multi-class classification models aimed at detecting and filtering harmful content; the models cover
      • four categories: hate, sexual, violence, and self-harm across
      • four severity levels: safe, low, medium, and high.
    • The default content filtering is default to filter at the medium severity threshold for all four content harm categories for both prompts and completions. That means that content that is detected at severity level medium or high is filtered, while content detected at severity level low is not filtered by the content filters. The configurability feature is available in preview and allows customers to adjust the settings, separately for prompts and completions, to filter content for each content category at different severity levels.

AZURE OPENAI MODELS

Azure OpenAI provides access to models with various capabilities. The following is a list of the models and their descriptions:

  • GPT-4 (8k/32k): A set of models that improve on GPT-3.5 and can understand as well as generate natural language and code.
  • GPT-3 (4k/16k): A series of models that can understand and generate natural language. This includes the new ChatGPT model.
  • DALL-E: A series of models that can generate original images from natural language.
  • Codex: A series of models that can understand and generate code, including translating natural language to code.
  • Embeddings: A set of models that can understand and use embeddings. An embedding is a special format of data representation that can be easily utilized by machine learning models and algorithms. The embedding is an information dense representation of the semantic meaning of a piece of text. Currently, we offer three families of Embeddings models for different functionalities: similarity, text search, and code search.

AZURE OPENAI ON YOUR DATA

With Azure OpenAI GPT-35-Turbo and GPT-4 models, enable them to provide responses based on your data. You can access Azure OpenAI on your data using a REST API or the web-based interface in the Azure OpenAI Studio to create a solution that connects to your data to enable an enhanced chat experience.

Per the document, Azure OpenAI on your data, Azure OpenAI Service supports the following file types:

File typeExtension
Text.txt
Markdown.md
HTML.html
Word.docx
PowerPoint.pptx
PDF.pdf
CSV.csv
TSV.tsv
Excel.xlsx
JSON.json
JSONL.jsonl

QUICKSTART

Previous models were text-in and text-out, meaning they accepted a prompt string and returned a completion to append to the prompt. However, the GPT-35-Turbo and GPT-4 models are conversation-in and message-out.

TRAIN MODEL

TOKEN

  • Azure OpenAI processes text by breaking it down into tokens. Tokens can be words or just chunks of characters. For example, the word “hamburger” gets broken up into the tokens “ham”, “bur” and “ger”, while a short and common word like “pear” is a single token. Many tokens start with a whitespace, for example “ hello” and “ bye”.
  • The total number of tokens processed in a given request depends on
    • the length of your input,
    • output and
    • request parameters.

The quantity of tokens being processed will also affect your response latency and throughput for the models.

Azure OpenAI Pricing

  • Pricing will be based on the pay-as-you-go consumption model with a price per unit for each model, which is similar to other Azure AI Services pricing models.

Azure Service Availability

  • SLA: This describes Microsoft’s commitments for uptime and connectivity for Microsoft Online Services covering Azure, Dynamics 365, Office 365, and Intune.

Quota and Limits

PLAYGROUND

The system role also known as the system message is included at the beginning of the array. This message provides the initial instructions to the model. You can provide various information in the system role including:

  • A brief description of the assistant
  • Personality traits of the assistant
  • Instructions or rules you would like the assistant to follow
  • Data or information needed for the model, such as relevant questions from an FAQ

You can customize the system role for your use case or just include basic instructions. The system role/message is optional, but it’s recommended to at least include a basic one to get the best results.

Azure Saving Plan vs. Reserved Instances, Document Extract and Notes

Saving Plans and Reserved Instances

I have compiled the following information to help you better understand “Saving plan” and “Reserve instances” when using Pricing Calculator as shown on the left.

 Azure Saving PlanAzure Reserved Instance
Doc
  • Save with Azure savings plans
  • Save with Azure reservations
  • What
  • Up to 30%+ discount from pay-as-you-go pricing across Azure compute resource globally
  • Applicable to a selected scope
  • Scope changeable
  • Plan transferable between supported agreements
  • 1- or 3-year commitment with monthly or up-front payment
  • For Windows VMs and SQL Database, the saving plan discount doesn’t apply to the software costs. You may be able to cover the licensing costs with Azure Hybrid Benefit.
  • Self-service reservations trade-in for Azure savings plans
  • If your Azure VMsDedicated Hosts, or Azure App Service reservations don’t provide the necessary flexibility you need, you may trade them for a savings plan.
  • Up to 60%+ discount from pay-as-you-go prices
  • Reservation scoping and options
  • Discount auto-applies to the resource usage that matches the attributes you select when you buy the reservation including: SKU, Region (where applicable); and Reservation scope
  • 1- or 3-year commitment with monthly or up-front payment
  • Reservation benefits are “use-it-or-lose-it” at hourly grain.   
  • HowSearch the keyword, save and click Saving plans.
  • Who can buy a savings plan
  • Savings plan-eligible resources
  • Determine your savings plan commitment
  • Manage Azure savings plans – Microsoft Cost Management
  • Seach with the keyword, save or reservation, and click Reservations. Azure Reservation
  • Determine what to purchasePurchase reservations
  • Why
  • Consistent compute spending on disparate resources
  •  
  • Consistent resource usage
  •  

    Additional Information

    Use Spot Virtual Machines to buy unused compute capacity at significant cost savings for:

    • workloads that can handle interruptions and don’t need to be completed within a specific period of time.
    • workloads for dev/test/QA/Batch/ML/AI/image, etc. 

    Azure Network Topology Document Extracts and Notes

    Azure Network Topology

    • Two core approaches: traditional and Azure Virtual WAN
    • The above document has a topology diagram for each model.
    FeatureTraditional Azure Network TopologyAzure Virtual WAN Network Topology
    HighlightsCustomer-managed routing and security

    An Azure subscription can create up to 50 vnets across all regions.

    Vnet Peering links two vnets either in the same region or in different regions and enables you to route traffic between them using private IP addresses (carry a nominal charge).

    Inbound and outbound traffic is charged at both ends of the peered networks. Network appliances such as VPN Gateway and Application Gateway that are run inside a virtual network are also charged.

    Azure Virtual Network Pricing  
    A Microsoft-managed networking service providing optimized and automated branch to branch connectivity through Azure.

    Virtual WAN allows customers to connect branches to each other and Azure, centralizing their network and security needs with virtual appliances such as firewalls and Azure network and security services.

    Azure Virtual WAN Pricing
    DeploymentCustomized deployment with routing and security managed by the customer

    Virtual Network documentation

    Plan virtual networks

    Tutorial: Filter network traffic with a network security group using the Azure portal
    Microsoft-managed service

    Virtual WAN documentation

    Tutorial: Create an ExpressRoute association to Virtual WAN – Azure portal

    – Other tutorials include site-to-site and point-to-site connections
    InterconnectivityTraffic between two virtual networks across two different Azure regions is expected. Full mesh network across all Azure regions is not required.Global connectivity between vnets in these Azure regions and multiple on-premises locations.
    IPsec TunnelsFewer than 30 IPsec Site-to-Site tunnels are needed.More than 30 branch sites for native IPsec termination.
    Routing PolicyFull control and granularity for manually configuring your Azure network routing policy.Not applicable
    Data CollectionCollects data from servers and Kubernetes clusters.Collects data from servers and Kubernetes clusters.
    Data StorageStores data in Log Analytics workspace or customer’s own storage account.Stores data in Log Analytics workspace or customer’s own storage account.
    Data Analysis and VisualizationUses Log Analytics for analysis and visualization of collected data.Uses Azure Monitor for analysis and visualization of collected data.

    Additional Information

    WHERE TO START AZURE

    If you have basic understanding of cloud computing, while new to Azure, I recommend starting with the following:

    I believe less is more. The above should be sufficient to get you situated.

    Azure Landing Zone Document Extracts and Notes

    What is an Azure landing zone? – Cloud Adoption Framework

    Landing zone implementation options – Cloud Adoption Framework

    “A migration landing zone is an environment that’s been provisioned and prepared to host certain workloads. These workloads are being migrated from an on-premises environment into Azure.”

    Deploy a CAF Foundation blueprint in Azure

    “The CAF Foundation blueprint does not deploy a landing zone. Instead, it deploys the tools required to establish a governance MVP (minimum viable product) to begin developing your governance disciplines. This blueprint is designed to be additive to an existing landing zone and can be applied to the CAF Migration landing zone blueprint with a single action.”

    Get help building a landing zone – Cloud Adoption Framework

    “Getting your Azure landing zone (ALZ) done right and on time is important. Working with a certified Azure partner is a great way to get the support you need to build your ALZ.”

    • Option 1 – use Azure Migrate and Modernize.
    • Option 2 – find a partner offer for a landing zone in our marketplace.

    Azure landing zone FAQ

    Azure Storage/Files/File Sync/Defenses for Ransomware Attack Document Extracts and Notes

    Service Description Documentation/Note
    Azure Storage Account
    • An Azure storage account contains all of your Azure Storage data objects: blobs, files, queues, and tables.
    • The storage account provides a unique namespace for your Azure Storage data that is accessible from anywhere in the world over HTTP or HTTPS.
    Azure Storage Types
    • Azure Blobs: A massively scalable object store for text and binary data. Also includes support for big data analytics through Data Lake Storage Gen2.
    • Azure Files: Managed file shares for cloud or on-premises deployments.
    • Azure Elastic SAN (preview): A fully integrated solution that simplifies deploying, scaling, managing, and configuring a SAN in Azure.
    • Azure Queues: A messaging store for reliable messaging between application components.
    • Azure Tables: A NoSQL store for schemaless storage of structured data.
    • Azure managed Disks: Block-level storage volumes for Azure VMs.
    Access to Azure Storage
    Microsoft Defender for Cloud

     

    Microsoft Defender for Storage

    • Azure Defender for Cloud is a cloud-native application protection platform (CNAPP) with a set of security measures and practices designed to protect cloud-based applications from various cyber threats and vulnerabilities.
    • Defender for Cloud combines the capabilities of
      • Cloud security operations (DevSecOps),
      • Cloud security posture management (CSPM), and
      • Cloud workload protection platform (CWPP) with specific protections for servers, containers, storage, databases, and other workloads
    • Azure Defender for Storage is a cloud-native security solution that provides malware scanningthreat protection, security alerts, etc. for your data stored in Azure Blob Storage and Azure Files .
    Azure Defenses for Ransomware Attack
    Azure Storage Access Tiers Hot tier

    • An online tier optimized for storing data that is accessed or modified frequently. The hot tier has the highest storage costs but the lowest access costs.

    Cool tier

    • An online tier optimized for storing data that is infrequently accessed or modified. Data in the cool tier should be stored for a minimum of 30 days. The cool tier has lower storage costs and higher access costs compared to the hot tier.

    Cold tier

    • An online tier optimized for storing data that is infrequently accessed or modified. Data in the cold tier should be stored for a minimum of 90 days. The cold tier has lower storage costs and higher access costs compared to the cool tier.

    Archive tier

    • Azure Blob Storage documentation
    • Setting the access tier is only allowed on Block Blobs. They are not supported for Append and Page Blobs.
    • When your data is stored in an online access tier (either hot, cool or cold), users can access it immediately.
    • The hot, cool, and cold tiers support all redundancy configurations.

     

    Azure Files
    Azure File Sync
    • Most frequently accessed files are cached on your local server and your least frequently accessed files are tiered to the cloud.
    • With cloud tiering enabled, this feature stores only frequently accessed (hot) files on your local server. Infrequently accessed (cool) files are split into namespace (file and folder structure) and file content. The namespace is stored locally and the file content stored in an Azure file share in the cloud.
    • Azure File Sync is ideal for distributed access scenarios. For each of your offices, you can provision a local Windows Server as part of your Azure File Sync deployment. Changes made to a server in one office automatically sync to the servers in all other offices.
    • Azure File Sync is backed by Azure Files, which offers several redundancy options for highly available storage. Because Azure contains resilient copies of your data, your local server becomes a disposable caching device, and recovering from a failed server can be done by adding a new server to your Azure File Sync deployment.

     

     

    Azure NetApp Files
    • It is an Azure native, first-party, enterprise-class, high-performance file storage service.
    • It provides NAS volumes as a service for which you can create NetApp accounts, capacity pools, select service and performance levels, create volumes, and manage data protection.
    Azure Backup Policy
    Azure File Alert Settings

    Why Azure Arc

    For IT decision makers, here’s why it’s pertinent to consider Azure Arc:

    • An integrated management and governance solution that is centralized and unified, providing streamlined control and oversight.
    • Securely extending your on-prem and non-Azure resources into Azure Resource Manager (ARM), empowering you to:
      • Define, deploy, and manage resources in a declarative fashion using JSON template for dependencies, configuration settings, policies, etc.
      • Manage Azure Arc-enabled servers, Kubernetes clusters, and databases as if they were running in Azure with consistent user experience.
      • Harness your existing Windows and Azure sysadmin skills honed from on-premises deployment.
    • When connecting to Azure Arc-enabled servers, you may perform many operational functions, just as you would with native Azure VMs including these key supported actions:
      • Govern
      • Protect
        • Secure non-Azure servers with Microsoft Defender for Endpoint, included through Microsoft Defender for Cloud, for threat detection, vulnerability management, and proactive monitoring for potential security threats. Microsoft Defender for Cloud presents the alerts and remediation suggestions from the threats detected.
      • Configure
      • Monitor
        • Keep an eye on OS, processes, and dependencies along with other resources using VM insights. Additionally collect, store, and analyze OS as well as workload logs, performance data, and events. Which may be injected into Microsoft Sentinel real-time analysis, threat detection, and proactive security measures across the entire IT environment.
    October 10, 2023 is the date the support for Windows Server 2012 and 2012 R2 ends.
January 9, 2024 is the date the support for Windows Server 2012 and 2012 R2 ends.

    Extended Security Updates (ESUs) is enabled by Azure Arc. IT can seamlessly deploy ESUs through Azure Arc in on-premises or multi-cloud environments, right from the Azure portal. In addition to providing a centralized management of security patching, ESUs enabled by Azure Arc is flexible with a pay-as-you-go subscription model compared to the classic ESU offered through the Volume Licensing Center which are purchased in yearly increments.

    To test it out, follow Quickstart – Connect hybrid machine with Azure Arc-enabled servers.

    Azure AD has become Microsoft Entra ID

    Per Microsoft’s Announcement on July 11th, the net is

    “To simplify our product naming and unify our product family, we’re changing the name of Azure AD to Microsoft Entra ID. Capabilities and licensing plans, sign-in URLs, and APIs remain unchanged, and all existing deployments, configurations, and integrations will continue to work as before. Starting today, you’ll see notifications in the administrator portal, on our websites, in documentation, and in other places where you may interact with Azure AD. We’ll complete the name change from Azure AD to Microsoft Entra ID by the end of 2023. No action is needed from you.

    Chart outlining all the product name changes that come with the renaming of Azure AD to Microsoft Entra ID.

    Here are some key resources:

    Azure OpenAI Is Ready, Are You?

    Azure OpenAI can be utilized for a wide range of tasks that cater to both business and technical requirements. It offers various capabilities, including but not limited to:

    Content Generation: Azure OpenAI can generate high-quality and coherent text content for a variety of purposes, such as writing articles, product descriptions, marketing materials, and more. It can help automate content creation and save time and effort.

    Summarization: With Azure OpenAI, you can extract key information and generate concise summaries from large volumes of text. This can be particularly useful for processing lengthy documents, news articles, research papers, or any content that requires distilling important points.

    Semantic Search: Azure OpenAI enables semantic search capabilities, allowing you to perform more advanced and accurate searches based on the meaning and context of the query. This can improve search results by understanding the intent behind the search terms, resulting in more relevant and targeted information retrieval.

    Natural Language to Code Translation: Azure OpenAI can assist in translating natural language queries or instructions into executable code. This feature can be helpful for developers and non-technical users alike, allowing them to express their requirements in plain language and receive code snippets or solutions that align with their intentions.

    In summary, Azure OpenAI offers a powerful suite of tools for content generation, summarization, semantic search, and translating natural language to code. It empowers businesses and individuals to leverage advanced AI capabilities to automate tasks, enhance productivity, and unlock new possibilities in various domains.

    Here’s how to start using Azure OpenAI services:

    TaskDescription
    Accessing Azure OpenAITo access Azure OpenAI, you need to create an Azure subscription and apply for access to the Azure OpenAI service by completing the form at https://aka.ms/oai/access
    Azure OpenAI StudioAzure OpenAI provides a web-based interface in the Azure OpenAI Studio to access OpenAI’s powerful language models including the GPT-3, Codex and Embeddings model series.
    Python SDKAzure OpenAI provides a Python SDK to access the service.
    Quotas and LimitsAzure OpenAI has certain quotas and limits that apply to the service, such as the number of requests per second per deployment and the total number of training jobs per resource.
    Business Continuity and Disaster Recovery (BCDR)Azure OpenAI provides BCDR considerations for implementing BCDR with Azure OpenAI.

    References:

    1. Quickstart – Deploy a model and generate text using Azure OpenAI Service
    2. How to customize a model with Azure OpenAI Service
    3. What’s new in Azure OpenAI Service? – Azure Cognitive Services
    4. Business Continuity and Disaster Recovery (BCDR) with Azure OpenAI Service

    End-to-End BCDR with Azure ASR: From Setup to Failover and Back

    In my experience, many companies viewed implementing Business Continuity and Disaster Recovery (BCDR) as too technically complex and financially unfeasible, resulting in it becoming more of an academic exercise than an attainable, predictable, measurable, and verifiable business process.

    Sample Solution Architecture (Source: Microsoft Cloud Workshop)

    With Azure Recovery Services, I have found this perception no longer accurate.

    In a recent demonstration, I used the Microsoft Cloud Workshop to showcase Azure BCDR with step-by-step guidance to

    • Configure a DR plan for a database app in Azure West US 3 region
    • Drill/rehearse the plan to failover the app to Azure East US region is a DR scenario
    • Execute a failover to mimic conducting a DR episode
    • Commit the failover upon verifying the plan executed with expected results

    Later

    • Follow a series of steps for reversing and falling back the app to its original region, West US 3
    • Reenable the protection, i.e., DR pan, and ensure readiness for future DR needs

    The following slide deck includes screen captures of relevant processes and resource settings, serves as a reference for context and expected results. While the deck is not intended to replace the workshop instructions and despite inconsistent resource names in some sections, the process flows with expected resource states are accurately depicted. One may find it handy for realizing the how and what of executing the workshop exercises and tasks.

    References