A Memorandum to IT Pros on Imperative vs. Declarative Scripting Models

One noticeable difference of Azure Infrastructure Services (IaaS) V2 from Azure IaaS V1 (or classic Azure IaaS as I call it) is the employment of  Azure “Resource Group” templates. A resource group not only is a newly introduced artifact in Azure, but denotes a fundamental shift on automating, deploying, and managing IT resources. This change signifies the arrival of a declarative programming/scripting model for the better. I will walk through an application deployment with Azure resource group templates in an upcoming post. In this memo, the focus is on distinguishing these two programming/scripting models.

Imperative vs. Declarative

Traditionally, within a logical unit of work (or simply a transaction) the conventional wisdom is to define how to implement a business logic by programmatically referencing parameter values, verifying the dependencies, examining variables at runtime, and stepping through a predefined data flow accordingly. This is a so-called imperative programming model which uses assignments, conditions/branching and looping statements to serialize operations for establishing the state of a program at runtime, i.e. an instance. An imperative programming model is to describe virtually “how” to reach “what.” A vivid example is that C-family programming languages are based on an imperative model. An imperative model like the following pseudo code specifies the steps (i.e. how) to ensure the operability of attaching a database to a SQL server (in other words, what) by ensuring the SQL server is first up and running, i.e. ready, before attaching an intended database. The implementation logic is to repeated a routine of waiting for a specified period of time, checking the status of a target resource, until the target resource is ready for an intended operation.

Wait 30 seconds and check the SQL server status again, till it is up and running

Then attach the database

At the same time, a declarative programming model is to describe business logic based on ‘what it is and not how to do it.’ For instance, rather than programming a loop to periodically check the status of if a target SQL server is up and running like what an imperative model does as depicted by the above example, a declarative model will simply state the dependency on a target SQL server, i.e. what the state must meet, before attaching an intended database and let the system (here I use the system as an umbrella team of other components) to implement how to enforce this pre-requisite. The following illustrates a declarative approach.

This database has a dependency of the hosting SQL server

The above states the dependency, i.e. what it is, and delegates the implementations carried out later.

What vs. How

Notice that an imperative model is to specify both the what and the how of a deployment. At the same time, a declarative model implies a logical separation and focuses on the what and leave the how later.  In layman’s term, imperative vs. declarative is simply an approach of how vs. what, respectively.

Why Declarative

For simple operations, one may not be advantageous over the other. For large amount of operations or tasks with high concurrency and noticeable complexities, the orchestrations can be too overwhelming to productively implement with an imperative model. This is increasingly what IT pros are facing in a cloud setting where operations are intermittent, concurrent, and carried out on hundreds or thousands of various application instances with inter- and intra-dependencies among themselves at an application layer and a system level.

A declarative model states what a target state is and the system will make it so, i.e. enforce it as stated. Employing an declarative model will fundamentally simplify how an administrator carries out application deployment and automation with increased consistency, persistency, and predictability.

As IT is transitioning into cloud computing, the number of VMs will continue to increase while the deployment environment is likely becoming hybrid and complex, adopting a declarative programming model is, in my view, critical and inevitable.

Essentially, IT has become such a highly integrated and increasingly complex environment, which is particularly true in an emerging IT model where cloud computing combined with hybrid deployment scenarios. Programmatically describing how to establish a state in runtime can quickly overwhelm programming logic and make an implementation based on imperative model very costly to develop and maintain. Shifting to a declarative programming model is strategic and becoming “imperative” for IT.

Call to Action

Recognizing the presented opportunity, IT pros should make this shift from imperative to declarative scripting models sooner than later. Employ a declarative model as a vehicle to improve the capabilities and productivity of application deployments, to facilitate and maximize ROI of transitioning to cloud in an IT organization. To get started, there are already abundant information of Azure IaaS V2 available including:

In addition, those who are new to Azure IaaS may find the following resources helpful:

And for those who would like to review cloud computing concepts, I recommend:

Application Deployment as a Service, A Sample Implementation of with Microsoft Azure PowerShell

This is a lab delivered in the spring of 2015 for Microsoft US IT Camps, Extend Your Datacenter to Azure, which is a whole day event with hands-on experience on deploying and migrating workloads to Azure. This lab is specifically for IT pros to experience an automatic deployment of a business function/application, instead of deploying just VMs. The ability to deploy VMs are important and essential. Deploying VMs are however not the ultimate goal of moving to cloud. As I have addressed elsewhere, cloud goes way beyond virtualization and deploying VMs, instead it is about the anytime readiness and on-demand abilities to grow and shrink resource capacities based on demands, i.e. being elastic. And this lab does just that to prove this concept using Azure and PowerShell.

The script is published in github and one can run the script as it is and without making changes. In the recording below, I walked through the steps to acquire and run the script. The intent is to run it as a service, i.e. on demand, to deploy application instances from zero to running instances. Notice this script is for learning and testing Microsoft Azure and PowerShell. It does hard-code and not encrypt employed password, has very limited error handling, is not intended for production use.


For those who are not familiar with the essentials of Microsoft Azure Infrastructure Services, compliance, pricing, and cost structure, here are additional resources:

Windows Azure Pack Express Installation

This is a project for Microsoft Virtual Academy that I had the pleasure to work with Shri (Shriram Natarajan, a Program Manager in Windows Azure Pack team) and had a wonderful time and learned much from him.

Windows Azure Pack, one of my favorite subjects on transforming your private cloud into a customer-centric IT as a service hub. The idea is to offer customers a solution platform such that they can self-serve on consuming, establishing, and managing IT capabilities including network, storage, and compute on demand regardless if resources are on-premises, deployed in Azure, or hosted in a 3rd party facility. The enabler, Windows Azure Pack, places an abstraction to present VMM-based private cloud with a Azure-like interface and experience, while integrating and consolidating at the middleware layer to enable on-premises, Azure, and 3rd-party resources to be managed with a consistent experience.

The first step in this strategic approach is to experience and asses Windows Azure Pack relevant to your unique IT environment. Which is what this project is about.


Essentials for Realizing Azure Baseline Costs

Subscription and Service Limits, Quotas, and Constraints (http://aka.ms/Limits)

This is a must-bookmarked page to find out, for instance:

  • Total cpu cores a subscription can consume
  • The number of endpoints a VM can have.
  • The bandwidth an Azure website may provide.
  • Total VMs a virtual network may host.
  • Total TB per storage account
  • Maximum of objects in Azure Active Directory

Azure Pricing Model (http://aka.ms/AzurePricing)

This is where to find detailed Azure pricing information of Azure services with options like pay-as-you-go, Microsoft resellers, and enterprise agreements. A good reference this is. Keep it handy.

Azure Pricing Calculator (http://aka.ms/Calculator)

This is a what-if analysis tool. How will the costs be different when changing your deployment plan from 3 small Linux VMs to 2 Large ones, deploying Oracle Weblogic server with A5 instead of A8, and replacing locally-redundant storage with geo-redundant storage? Just drag the slide bar on each category and one can realizing the combined baseline costs form the number shown under full calculator.


There is also a VM calculator (http://aka.ms/AzureVMCalculator) to better understand the cost implementations on VM, bandwidth, support, etc. with Azure Infrastructure Services.


Azure SLAs (http://aka.ms/AzureSLAs)

imageDo examine the SLAs from a cloud vendor including those of Microsoft’s. Cloud computing is emerging and the business model continues evolving. SLAs are not all created equal. Pay attention to the fine print. Understand what does and what does not count as an outage and when there is one how a subscriber is compensated. Following your flow of data, the overall SLA is the service with the lowest SLA along the path.

Azure Support Options (http://aka.ms/AzureSupportOptions)

imageSupport costs will have impact on the business values of an application throughout the lifecycle. Not just the support subscription costs, but also the associated activities to initiate, monitor, manage, and document support activities.

Azure Compliance Page (http://aka.ms/AzureCompliance)

For some industries, if it is not compliant, there is really no point of investigating the cost of a solution. If compliance is a requirement, investigate early. For Azure, this page lists out all the achieved certificates. Pay attention to those referenced links like Azure HIPAA Implementation Guidance. Some specifically document the implementations for a cloud applications to become compliant.


Closing Thoughts

Cloud is about the ability to deliver instant gratification, and grow or shrink the capacity based on demand. Those days of rolling out a patch in 6 weeks, deploying a branch office in 3 months, building a cluster in two weeks, are long gone. The delivery needs to be on demand, in the next hour, within a half-day, or something relatively quick.

At the same time, cloud is not an one-size-fits-all platform. And there are legitimate reasons not to deploy resources in cloud. Get all the facts, learn how cloud works, and assess the risks. Know what you pay for, set a realistic expectation, then cloud responsibly and happily.

So where to start? Learning by practicing is what I recommend. For those who do not subscribe MSDN which offers Azure monthly usage credit, sign up a 30-day free trial at http://aka.ms/Azure200 and follow http://aka.ms/Azure101Series and http://aka.ms/Azure102Series to start making cloud work for you. Use Azure as your datacenter, your global networks, and your colossal storage drive in cloud.

SharePoint in Azure, My Presentation in MVP Open Days

This is the presentation I delivered in US MVP Open Days 2015. The key difference between on-premises deployment and cloud deployment of SharePoint or any application is the consideration of application fabric, particularly cloud service (or compute). Which is a focus on my delivery.

Another Memorandum to IT Pros on Cloud Computing: Virtualization , Cloud Computing, and Service

As IT considers and adopts cloud computing, I thought it is crucial to understand the fundamental difference between cloud and virtualization. For many IT pros the questions are where to start and what is the road map from on-premises to cloud, to hybrid cloud. In my previous memo to IT pros on cloud computing I highlighted a number of considerations when examining virtualization and cloud. In this article, I detail more on the underlying approach of the two and how they are different on problem domain, the scope of impact, and most significantly service model.

In the context of cloud computing, virtualization is the capability to run multiple server, network or storage instances on the same hardware, while each instance runs as if it is the only instance occupying and consuming the entire underlying hardware. Notice networking and storage devices generally present themselves via some form of an OS instance, namely for network and storage there is a software component. However here I uses the term, hardware, to include the needed software and hardware to form an underlying layers for a network or a storage device to run.

Virtualization is an enabler of cloud computing and a critical component in a cloud implementation. Virtualization is however not cloud computing and should not be considered at the same level as illustrated in the figure later in this article. And IT pros must recognize that virtualization is a mean, while cloud computing is an intermediate step and also a mean for the ultimate goal which is to deliver a service to customer.

These terms: virtualization, cloud and service are chosen carefully, and not to be confused or used interchangeably with simulation, infrastructure or application, for instance. The three terms signify specific capabilities and represent an emerging model of solving business problems with IT.

Here, I will present the concepts of virtualization, cloud and then service to make these points that

  • Cloud computing is to deliver services on a user’s demand, while virtualization is a technical component enabling cloud computing.
  • Virtualization is a necessary, yet not sufficient condition for cloud computing.

First to fully appreciate the role virtualization plays in IT, let’s first realize the following.

Isolation vs. Emulation/Simulation

A key concept in virtualization technologies is the notion of isolation. The term, isolation, signifies the ability to provide an instance of server, network, or storage a logical (i.e. virtualized) environment such that the instance runs as if it is the only instance of its kind occupying and consuming the entire underlying supporting layer. This isolation concept differentiates virtualization from the so-called emanation and simulation.

Either emulation or simulation mimics the runtime environment of an application instance. While running multiple emulated or simulated runtime environments, each remains running within a common and single OS instance. Essentially emulation or simulation runs multiple runtime environments in a shared address space, therefore either approach can potentially introduce resource contention and limit the performance and scalability rather quickly. At the same time, virtualization isolates or virtualizes at an OS level by letting each virtualized environment have its own OS with virtual CPUs, network configuration and local storage to form a virtual machine or VM. And a VM can and will actually schedule assigned virtual CPUs, hence the underlying physical CPUs, based on an associated VM configuration. As explained below, this is server virtualization.

Virtualization in the Context of Cloud Computing

Cloud computing recognizes virtualization in three categories. They are server virtualization, network virtualization and storage virtualization. Servers represent the computing capabilities, the abilities to schedule CPUs and execute compute instructions. Network is to direct, separate, or aggregate communication traffics among resources, while storage is where applications, data, and resources are stored.

These three are all implemented with virtualization. And each of the three plays a vital part. The integration of the three forms what is referenced as “fabric” as explained in the article “Fabric, Oh, Fabric.”

Server Virtualization

Hence server virtualization means to run multiple server instances on the same computer hardware, while each server instance (i.e. VM) runs in isolation, meaning each server instance runs as if it is the only instance occupying and consuming the entire hardware.

Network Virtualization

This is an ability to establish multiple network instances, i.e. IP addressing scheme, name resolution, subnets, etc. on the same network layer while each network instance runs in isolation. Here I don’t call it network hardware, instead of a layer since a network instance is wrapped within and requires a form of OS which runs on computer hardware, so there is also a software component with networking.

Multiple network instances in isolation means each network instance exists as if the network configuration is unique on the supporting network layer. For instance, in a network virtualization environment, there can be multiple tenants which all have an identical network configuration like 192.168.x.x while within each virtual network when referencing the same IP address like, virtualization will ensure tenant A’s network and tenant B’s network remains operating correctly. And a reference within one network will not be confused with one in other network even both have the same IP configured with respective logical network.

Storage Virtualization

A similar concept applies to storage virtualization which is a capability to aggregate storage devices in various kinds while presenting all as one logical device with a continuous storage space. For instance, storage virtualization enables a user to form a logical storage volume by detecting and selecting storage devices with unallocated storage in various interfaces like SATA, SAS, IDE, SCSI, iSCSI, and USB. This volume then aggregates all available storage from those selected devices and presented to a user as a single formatted volume with continuous storage. This concept has been long realized by SAN technologies which is an expensive and specialized storage virtualization solution. Lately, storage virtualization has become a software solution, been integrated into OS, and become a core IT skill and integral part of JBOD storage solution for IT.

Cloud Fabric

Fabric represents three things: compute, network and storage. Compute is basically the ability to execute instructions in CPUs. Network is how resources are glued together. Storage is where application and data are stored. This fabric concept is something we all have been exercising often and sometimes not fully aware of. For instance, when buying a computer, we mainly consider a number of items: the chip family and processor speed (compute), how much hard disk capacity and the spindle speed (storage), while network capabilities are expected and integrated into the mother board. The fact is that when buying a laptop, we focus on the fabric configuration which forms the capabilities of a solution, here a laptop, a personal computing solution. This practice is applicable from an individual’s hardware purchase, to IT’s running datacenter facilities. For cloud computing, because the transparency of physical locations and hardware where applications are running, we can use fabric to collectively represent the IT capabilities of compute, network, and storage relevant to a datacenter.

Fabric Controller

This term denotes the management solution for managing compute, network and storage in a datacenter. Fabric controller, as the name indicates, controls and is the owner of a datacenter’s fabric. The what, when, where and how of IT capabilities in a datacenter are all managed by this logical role, Fabric Controller.

There is another significance of employing the term, fabric. It signifies the ability to discover, identify, and manager a resource. In other words, a fabric controller knows if there are newly deployed resources and is able to identify, bring up, monitor or shut down resources within fabric.


Virtualization and Fabric

From a service provider’s view point, constructing cloud is mainly to construct cloud fabric, so all resources in cloud can be discovered, identified, and managed. Form a user’s point of view, all consumable resources in cloud are above the virtualization layer.

Cloud relies on virtualization to form fabric which wraps and shields the underlying technical and physical complexities of servers, networks, and storage infrastructures from a user. Virtualization is an enabling technology for cloud computing. Virtualization is nevertheless no cloud computing itself.

Commitments of Cloud Computing

What is cloud? Cloud is a state, a set of capabilities which enables a user to consume IT resources on demand. Cloud is about the ability to consume resource on demand, and not implementations. What cloud guarantees are the capabilities which a solution must deliver. Cloud is about the existence of particular computer programs or an employment of specific products or technologies.

Above all, NIST SP 800-145 (800-145) has define a set of criteria including essential characteristic, service models, and deployment models. NIST essentially specifies what capabilities cloud has and how they must be delivered. 800-145 has been adopted as an industry standard for IT on adopting cloud. Here is how I interpret the definition on essential characteristics of cloud computing.

  • A cloud must enable a user to self-serve. That means that a user is to call no one and email no one, when consuming cloud resources. A user should be able to serve oneself to allocate storage, create network, deploy VMs and configure application instances without the service provider’s intervention. Just this requirement is very significant already.
  • A cloud must be generally accessible for a user. Self-service and ubiquitous access are complementary. Self-service implies always accessibility, while always accessibility enables self-service. You can’t self-serve if you can’t get to it. And if you can’t get to it, you can’t self-serve. This requirement is mandatory and logical.
  • A cloud employs resource pooling. In my view, this implies actually three things: standardization, automation, and optimization. A pool is a metaphor for a collection of things with a same set of criteria. A swimming pool with water is an obvious example. Putting things in a resource pool implies that there is a standardization process based on set criteria to prepare resources for automation. Automation improves efficiency, not necessarily always the overall productivity. For instance, shortening the time by automating a process from 8 hours to 2 hours, by this time reduction itself does not necessarily increase the productivity accordingly since the assembly line may be idling for the rest of 6 hours. Unless relevant materials are continuously coming into the process line to optimize the productivity. Although this example is apparently trivial, the applicability is apparent for resource pooling to encompass standardization, automation and optimization as a whole to better manage cloud resources.
  • Elasticity is perhaps one of the most referenced cloud computing terms since it vividly describes cloud computing to be able to grow and shrink the capacity of a target resource in responding to demands, by visualizing a rubber band expands and retracts as the applied force varies.
  • Finally, for measured service, some call it a charge-back or show-back model, others may state it as pay-as-you-go. The significance of either is the underlying requirement of designed-in analytics. If a cloud enables a user to self-serve, is accessible anytime, grow and shrink based on demands, etc. as the first four essential characteristics state, we need a fundamental way to understand and develop date usage patterns to carry out a realistic capacity planning. Analytics is an essential part of cloud computing, must be architected and designed into a cloud solution and definitely not an afterthought.

All the above are five essential characteristics defined in 800-145. In other words, to be qualified as a cloud, these five attributes are expected, given, guaranteed, and committed. This is tremendous for both user and IT.

Is Virtualization Cloud Computing

The answer should be an obvious “NO.” In virtualization, the requirement is rooted on providing an isolation environment. There are no requirements on a virtualization solution must establish all including self-servicing, offer general accessibility, employ standardization/automation/optimization as a whole, grow and shrink virtualized resources based on demands, and provide analytics of consumptions.

Unique Values of Cloud Computing

There are three service models (IaaS, PaaS and SaaS) to consume cloud as defined in 800-145. From the definition, it is apparent that software is an application, platform means an application runtime and infrastructure implies a set of servers (or VMs) to form the fabric of a target application.

Some observations I have from examining these service models. There are three sets of capabilities offered by cloud, i.e. three way a user can consume cloud.

  • Enabling a user to consume an application as a service is SaaS.
  • Providing an application runtime such that a user can develop and deploy a target application as a service is PaaS.
  • Provisioning application fabric, i.e. compute, network and storage, for a target application is IaaS.

Notice all three must be delivered as a service. Application, platform and infrastructure are much described in 800-145, however it was unclear what a service is. This is an essential term which surprisingly NIST did not define in 800-145. In addition to what has been stated in “An Inconvenience Truth of the NIST Definition of Cloud Computing, SP 800-145,” this is another unfortunate missing part of NIST definition. So what is a service? Without some clarity, IaaS/PaaS/SaaS are designated for ambiguity.

“Service” in this context means simply “capacity on demand” or simply “on demand.” The three service models are therefore

  • IaaS: Infrastructure or a set of VMs as a whole forms an application infrastructure is available on demand
  • PaaS: Platform or an application runtime is available on demand
  • SaaS: Software or an application is available on demand

On-demand is a very powerful word. In this context, it means anytime, anywhere, on any network, and with any device. Cloud enables a user to consume an application, deploy an application to a runtime environment, or build application fabric as a service. It means anytime, anywhere, on any network, with any device a user is guaranteed to be able to build application fabric (IaaS), deploy application (PaaS), and consume an application (SaaS). This is where cloud delivers most of its business values by committing the readiness for a user to consume cloud resources anytime, anywhere, on any network, with any device.

On-demand also implies a series of management requirements including monitoring, availability, and scalability relevant to all five essential characteristics. To make a resource available on demand, the fabric controller must know everything about the resource. The on-demand requirement, i.e. delivered as a service, is a requirement which a cloud service provider must ensure every delivery fulfills.

Cloud Defines Service Models, Virtualization Does Not

Virtualization on the other hand defines what it delivers, i.e. an isolated environment for delivering compute, network, or storage resources. Virtualization however does not define what a virtualization service model is. For instance, server virtualization is the ability to create VMs. Server virtualization does not define if a VM must be instantaneously created and deployed upon a user’s request anytime, anywhere, on any network, and with any device. Missing a service model in virtualization is a very obvious difference why virtualization is no cloud, and cloud goes far beyond virtualization. Virtualization and cloud are contributing to business values at different dimensions. Those effects from the former are localized and limited, while the impacts from the latter are fundamental and significant.

Cloud Deployment Models

For cloud, 800-145 defines four deployment models including public, community, private, and hybrid. Those with different views (see “An Inconvenience Truth of the NIST Definition of Cloud Computing, SP 800-145,”) may consider otherwise. Regardless how many deployment models, it is a classification to address the ramifications in ownership, manageability, security, compliance, portability, licenses, etc. introduced by deployment.

Virtualization has no concern on a deployment model. The scope of virtualization is confined within the ability to virtualize a compute, network, or storage instance. Where a virtualized instance is deployed, who’s using it, etc. has no influence on how the virtualized instance is constructed. What is running and how an application is licensed within a VM, a logical network, or a JBOD volume is not a concern to virtualization. The job of virtualization is to construct a virtual instance and let the fabric controller manage the rest. For instance, a VM is a VM regardless it is deployed to a company host, a vendor’s datacenter, or a personal laptop. It is still a VM as far as virtualization is concerned.

Closing Thoughts

Examining the definitions of virtualization and cloud, it is clear that

  • Virtualization assures system integrity within a VM, a network, or a storage volume while multiple instances of a kind running on the same underlying hardware. At the same time, cloud computing guarantees a user experience presented by five essential capabilities.
  • Virtualization does not define how a VM to be delivered, while cloud computing commits all deliveries are on a user’s demand.
  • Virtualization technology is transparent on where a VM, a logical network, or a JBOD volume is deployed and who are the intended users. Cloud computing on the other hand categorizes deployment models to address the business impact on management, audit, security, compliance, licensing, etc.

Overall, cloud is about user experience, enabling consumption on demand, rapid or even proactive response to expected business scenarios on resource capacities. Virtualization on the other hand is a technical component which isolates multiple VM, network, or storage instances running on the same hardware layer. The focus and impact on virtualization is relatively localized and limited.

It should be very clear that cloud and virtualization are operating at different levels with very different scope of impact. Needless to say that virtualization is no cloud, and cloud goes far beyond virtualization.