Microsoft Cortana Intelligence Suite Workshop Video Tutorial Series (1/5): Introduction

This series, based on the content developed by Microsoft, offers a learning path with minimal time and effort to acquire the essential operation-level knowledge of Microsoft Cortana Intelligence Suite. The workshop steps through a process to construct and deploy a web application with predictive analytics, while along the way introducing key functional components. By specifying an origin and a destination airports, a future date and time and an airline carrier, this application predicts a flight delay with probability based on the weather forecast. The video tutorial series runs about 75 minutes and has captured exactly when and what you will see on the screen, where and how to respond based on the instruction of each exercise in the workshop.

I believe this series will most benefit those who function in a technical leadership capacity including: enterprise architect, solution architect, cloud architect, application architect, DevOps lead, etc. and are interested in the solution architecture of an application of predictive analytics. Going through the recordings will provide you an end-to-end view and clarity on how to constructing and deploying a predictive analytics solution, hence a better understanding on the processes and technologies, integration points, packaging and publishing, resource skill profiles, critical path, cost model, etc.

Cortana Intelligence Suite is a set of processes and tools. This workshop outlines an approach where analytic models, data, analysis, visualization, packaging, publishing and deployment are delivered in an integrated fashion. In my view, this is a productive and the right way to start learning how to architect a predictive analytics solution. The above video is the first of five to accelerate your learning of Cortana Intelligence Suite, and highlight a few important items before starting the workshop.

Content Repo

The content of this workshop made available by Todd Kitta is at http://aka.ms/CortanaManual in github. The readme file of the workshop details the scenario, architecture, prerequisites and a list of links to the instructions of all eight exercises.

image

The above architecture diagram of the workshop depicts the functional components for a web application with predictive analytics. Here the lab VM is also employed as an on-premises file server as the source of a data pipeline securely connected to a created Azure Data Factory service to automatically upload data to be scored by the Azure Machine Learning model. At the center is a Spark HDInsight cluster for data analysis, while the data are visualized by Power BI. The predictive analytics model is integrated and package as a web service consumed by a web application.

Introduction

Let’s first pay attention to a few important items before doing the workshop. There are eight exercises in this workshop and I have grouped them into five videos: an introduction and four learning units.

I recommend reading the instruction of an exercise in its entirety before doing the exercise, this will help set the context and gain clarity the objectives of each exercise. To do the workshop, one will need an active Azure subscription. Notice that a free trial account does provide sufficient credit for doing the entire workshop.

image

The workshop environment is a collection of resources deployed to Azure, as shown above, including:

  • A VM with Internet connectivity for a student to log in and work on all the exercises, such that there is no need to download or install anything locally for this workshop
  • A Machine Learning workspace accessed via Microsoft Azure Machine Learning studio to develop an experiment of predictive analytics
  • A Spark cluster for hosting and analyzing data including a scored dataset and a summary table
  • A number of storage accounts for storing workshop data

These resources do incur a cost. And to minimize the cost, try deploying the workshop environment only when you are ready to work on the exercises and delete it once completed the workshop. The deployment will take about fifteen minutes, if not more. And do deploy all resources and create services into the same resource group, so all can be later removed by simply deleting the resources group. Personally, when doing the workshop, I will set aside at least a four-hour block, find a quiet room and get a great cup of coffee. It is indeed a lot to consume.

Enjoy the workshop. Let’s get started.

Advertisements

Microsoft Azure Stack Technical Preview 1: Introduction & Feature Overview

This is something I had wanted to do for a while. Finally did welcome Charles Joy, a Principal Program Manager on the Azure Stack team, back to the show and we discussed the recent release of Microsoft Azure Stack Technical Preview 1. It’s a fun episode. Enjoy it.

image

A Memorandum to IT Pros on Imperative vs. Declarative Scripting Models

One noticeable difference of Azure Infrastructure Services (IaaS) V2 from Azure IaaS V1 (or classic Azure IaaS as I call it) is the employment of  Azure “Resource Group” templates. A resource group not only is a newly introduced artifact in Azure, but denotes a fundamental shift on automating, deploying, and managing IT resources. This change signifies the arrival of a declarative programming/scripting model for the better. I will walk through an application deployment with Azure resource group templates in an upcoming post. In this memo, the focus is on distinguishing these two programming/scripting models.

Imperative vs. Declarative

Traditionally, within a logical unit of work (or simply a transaction) the conventional wisdom is to define how to implement a business logic by programmatically referencing parameter values, verifying the dependencies, examining variables at runtime, and stepping through a predefined data flow accordingly. This is a so-called imperative programming model which uses assignments, conditions/branching and looping statements to serialize operations for establishing the state of a program at runtime, i.e. an instance. An imperative programming model is to describe virtually “how” to reach “what.” A vivid example is that C-family programming languages are based on an imperative model. An imperative model like the following pseudo code specifies the steps (i.e. how) to ensure the operability of attaching a database to a SQL server (in other words, what) by ensuring the SQL server is first up and running, i.e. ready, before attaching an intended database. The implementation logic is to repeated a routine of waiting for a specified period of time, checking the status of a target resource, until the target resource is ready for an intended operation.

Wait 30 seconds and check the SQL server status again, till it is up and running

Then attach the database

At the same time, a declarative programming model is to describe business logic based on ‘what it is and not how to do it.’ For instance, rather than programming a loop to periodically check the status of if a target SQL server is up and running like what an imperative model does as depicted by the above example, a declarative model will simply state the dependency on a target SQL server, i.e. what the state must meet, before attaching an intended database and let the system (here I use the system as an umbrella team of other components) to implement how to enforce this pre-requisite. The following illustrates a declarative approach.

This database has a dependency of the hosting SQL server

The above states the dependency, i.e. what it is, and delegates the implementations carried out later.

What vs. How

Notice that an imperative model is to specify both the what and the how of a deployment. At the same time, a declarative model implies a logical separation and focuses on the what and leave the how later.  In layman’s term, imperative vs. declarative is simply an approach of how vs. what, respectively.

Why Declarative

For simple operations, one may not be advantageous over the other. For large amount of operations or tasks with high concurrency and noticeable complexities, the orchestrations can be too overwhelming to productively implement with an imperative model. This is increasingly what IT pros are facing in a cloud setting where operations are intermittent, concurrent, and carried out on hundreds or thousands of various application instances with inter- and intra-dependencies among themselves at an application layer and a system level.

A declarative model states what a target state is and the system will make it so, i.e. enforce it as stated. Employing an declarative model will fundamentally simplify how an administrator carries out application deployment and automation with increased consistency, persistency, and predictability.

As IT is transitioning into cloud computing, the number of VMs will continue to increase while the deployment environment is likely becoming hybrid and complex, adopting a declarative programming model is, in my view, critical and inevitable.

Essentially, IT has become such a highly integrated and increasingly complex environment, which is particularly true in an emerging IT model where cloud computing combined with hybrid deployment scenarios. Programmatically describing how to establish a state in runtime can quickly overwhelm programming logic and make an implementation based on imperative model very costly to develop and maintain. Shifting to a declarative programming model is strategic and becoming “imperative” for IT.

Call to Action

Recognizing the presented opportunity, IT pros should make this shift from imperative to declarative scripting models sooner than later. Employ a declarative model as a vehicle to improve the capabilities and productivity of application deployments, to facilitate and maximize ROI of transitioning to cloud in an IT organization. To get started, there are already abundant information of Azure IaaS V2 available including:

In addition, those who are new to Azure IaaS may find the following resources helpful:

And for those who would like to review cloud computing concepts, I recommend:

Microsoft Azure Site Recovery (ASR) as Disaster Recovery (DR) Solutions for Businesses of All Sizes

No longer is DR a process too complex to implement and too expensive to afford. DR today is a realistic and achievable solution with manageable costs. With Microsoft Azure, IT can now implement relatively easily compared with what it was a few years ago a DR solution to restore IT capabilities at the same facility or across physical sites in a DR scenario.

Azure Site Recovery, or ASR, is a Microsoft solution to protect applications in DR scenarios by automating, monitoring and orchestrating the replication, failover and recovery of physical and virtual machines relevant to both Microsoft and VMware environments. As of February, 2015, there are five targeted ASR scenarios as listed below.

image

And each is with prescriptive guidance to step through the processes for building an associated DR solution. A replication coordinated by ASR can be directed to another on-premises datacenter, Azure or a hosting service provider without facing the financial commitment and implementing the technical complexities of building and managing a company-owned secondary location. The simplicity, automation, customizable recovery plans, health monitoring and orchestrated recovery and supported SLAs makes ASR a reliable DR solution that is readily available for businesses of all sizes.

To facilitate IT pros to assess ASR as a DR solution, this article highlights the following essentials.

  • First acquiring an Azure subscription
  • Starting with an ASR Recovery Vault
  • Familiarizing with the 5 Scenarios which ASR Manages, Automates and Orchestrates

First acquiring an Azure subscription

Microsoft Azure is a subscription-based public cloud offering and requires an active subscription to access her services. A free 30-day trial of an Azure subscription with $200 credit for deploying Azure services is available at http://aka.ms/Azure200.

image

MSDN subscribers

For those who are active MSDN subscribers, an associated Azure subscription is already put in place. However, a subscriber first needs to activate the associated Azure subscription from the Account page once authenticated into MSDN subscriber web site at http://msdn.microsoft.com.

image

Starting with an ASR Recovery Vault

Upon authenticated into Azure Management Portal, http://manage.windowsazure.com, either scroll down on the left navigation pane then click +NEW or simply click +NEW in the lower left corner of the black menu bar at the bottom to create a Recovery Vault. 

image

image image image image

A name and a region are the required two pieces of information. And upon creating a Recovery Vault, a Quick Start should appear. However if the Quick Start page does not automatically display itself instead it shows the name list, as shown in the last screen capture, click the arrow next to the vault name to bring up the dashboard. And as needed, click the little-blue-cloud-with-a-lightening icon which is the first one and next to DASHBOARD to bring up the Quick Start page, as illustrated below.

Quick Start Page

There is a Quick Start page on most, if not all, Azure services. This is an important reference with official guidance to implement/configure an associated service. The Quick Start page is readily available  by clicking the indicated icon i.e. the associated page tab.

image

Familiarizing with the 5 Scenarios which ASR Manages, Automates and Orchestrates

Notice on the Quick Start page, there are 5 site recovery scenarios available as shown above. To employ ASR as a DR solution, one should first get familiar with these scenarios and their associated processes and tasks. One direct and quick way to master ASR is to create various recovery vaults and follow the guidance of each scenario to experiment and assess the automation, replication and failover relevant to particular business requirements. Here I am including the Quick Start page of each of the five scenarios.

image

image

image

image

image

Closing Thoughts

Over the years, DR has remained a critical part in protecting business facing unplanned events overwhelming an entire business application or an IT facility. What has dramatically changed is the availability, readiness and affordability of a DR solution by employing cloud computing as an integral part. Businesses in all sizes can now implement a DR solution that is not only technically feasible and verifiable, but financially affordable and manageable as a subscribed service like ASR.

While recognizing this opportunity and the availability, reliability and readiness of a DR infrastructure is not longer an issue, IT now must validate business plans, develop ASR DR solutions and incorporate the ASR processes and operations into standard operating procedures. On a regular basis, simulate failovers to DR sites based on business scenarios to ensure DR readiness.

The good news is that Azure has done the heavy lifting by providing a DR infrastructure in cloud with on-demand accessibility and availability and wizard-driven processes to implement DR solutions based on business requirements. What IT must do now is to translate business DR requirements into configurations and processes by following what is presented in the ASR Quick Start page.

Automating and Managing Hybrid Cloud Environment

In part 5 of our “Modernizing Your Infrastructure with Hybrid Cloud”  series, Keith Mayer and I got a chance to discuss and demonstrate ways to manage and automate a hybrid cloud environment. System Center, Microsoft Azure and Windows Azure Pack combined with PowerShell are great solutions for hybrid cloud scenarios. Keith is a great guy and we always have much fun working together.

image

  • [1:15] When architecting a Hybrid Cloud infrastructure, what are some of the important considerations relating to management and automation?
  • [4:09] You mentioned PowerShell for automation … how can PowerShell be leveraged for automation in a Hybrid Cloud?
  • [7:54]  Is PowerShell my ONLY choice? Are there other automation and configuration management solutions available for a Hybrid Cloud?
  • [11:12] DEMO: Let’s see some of this in action
      • Brief tour of System Center and Azure / Azure Pack management portal interfaces
      • Getting started with PowerShell for Azure, Azure Pack automation
      • Intro to PowerShell DSC for configuration management
      • Intro to Azure Automation for automated runbooks

Additional resources: