A Secure Software Supply Chain with Containers

The concept of software supply chain is not a new one. What may be new is that CI/CD (Continuous Integration/Continuous Delivery) with containers makes it conceptually easy to understand and technically practical to implement. Here’s a process diagram illustrates this approach with five steps.

image

CI/CD Process

A software supply chain is here the “master” branch of a release, while development activities at other branch level are not considered here. The start of a master branch is where and when code or a change is introduced, while on the other end of the master branch is a production runtime environment where applications run. The process, as shown above, are highlighted in 5 steps.

  1. A CI/CD process starts by committing and pushing code to a centralized repository. Code here encompasses all programmatic assets relevant to defining, configuring, deploying and managing a relevant application workload.
  2. Changes made on the master branch triggers the CI process to automatically (re)build and (re)test all assets relevant to the workload which the master branch delivers.
  3. The successful outcome will generate a container images which are automatically versioned, tagged, registered and published in a designated trusted registry. In Docker Enterprise Edition, this is implemented as a Docker Trusted Registry, or DTR. The function of a trusted registry is to secure and host container images. Important tasks carried out here are to at a binary level scan for known malware, check vulnerabilities and digitally sign a container image upon being successfully processed. The generation of a container image signifies application assets are successfully integrated and packaged, which signifies the end of CI part.
  4. CD kicks off, executes and validates the steps for deploying the workload to a target production environment.
  5. Upon substantiating containers, referenced container images are then as needed pulled to a local host and start the container instances, hence an application or service.

Notice that Continuous Delivery is a reference of capability and not the state. Continuous Delivery signifies the ability to maintain payload at a production ready and deployable condition at all time. It is not necessary suggesting payload once validated is deployed to production immediately.

Once Version of Truth

A software supply chain starts with developers commit and push the code into the master branch of a release’s centralized repository. To have one version of truth, centralized management is essential. Nevertheless, as preferred we can operate a centralized repository in a distributed fashion like github. Further, a centralize source code hosting solution must properly address these priorities including role-based access control, naming and branching, high availability, network latency, single-point-of-failure, etc. With source control, promoted code can be versioned and tagged for asset and release management.

Triggering upon Pushing Changes

When changes introduced into the master branch which is the supply chain, CI must and will automatically kick off validation process which includes a set of defined criteria, namely test cases.

Test-Driven, a Must

Once the development criteria (or requirements) are set, test cases can be developed accordingly prior to writing the code. And this essentially establishes the acceptance criteria and force a developer to focus on designing code guided by what must be later validated. Which in essence designs in the quality.

When changes are made, there is no point to “manually” execute all test cases against all the code. Let me be clear. The regression tests are necessary, but the repetitions with manual labor is counterproductive and error-prone. A better approach is to programmatically automate all structured tests (i.e. the criteria are structured and stable, like canned test cases) and let a tester to do exploratory testing which may not be performed with scripted, expected or even logical steps, but with the intent to break the system. The automation makes regression tests consistent and efficient, while exploratory testing adds extra values by expanding the test coverage.

Master Branch Has No Downtime

A test-driven development in CI/CD holds a key objective that the master branch is always functional and ready to deliver. This may first appear to some of us as idealistic and over-committed. While in fact, considering an automobile production line, as material and parts are put together while moving through one station to another, an automobile is being built. If at any time, a station breaks down, it must be fixed at the scene since the whole production line is on hold. Fixing what stops a subject moving from one station to the next on the spot is necessary to keep the production producing.

Software supply chain, or a CI/CD pipeline, with containers is a digital version of the above-mentioned model where artifacts are definitions, configurations and scripts. As these artifacts are integrated, built and tested throughout the pipeline, the process to construct a service based on containers is validated. If a step fails the validation, the pipeline stops and the issue must be immediately addressed and resulted, so the process can continue to the next step, hence material flows through the pipeline. To CI/CD, a master branch is the pipeline and must be always kept functional and ready to deliver.

Containers Are Not the Deliverables

It should be noted here that artifacts passing through the CI/CD pipeline are neither container images, nor container instances. What the pipeline validates are a set of developed definitions, configurations and processes based on application requirements and presented in mark-up language like json or yaml. In Docker, they are dockerfile, compose yaml file and template-based scripts, for example, to define application architecture with configured Docker runtime environment for a target application delivered as containers upon instantiation.

Container images and instances are in a way by-products. Container images and instances are however not intended to be deliverables. A container image generated by a CI/CD pipeline should always first programmatically created by the initial CI and later reference or updated by CD. The key is that images must be pulled or generated by executing CI. With Docker, thanks for natively configuration management as code, a release may employ a particular version of a container image. And upgrade or fall back a software supply chain may be as easy as changing the reference version, followed by redeploying the associated payload.

Trusted Registry, the Connective Tissue

CI once successfully generated a container image should register and upload the image to a trusted registry for security scanning and digital signing, before CD takes over and later pulls or updates the images, as needed to complete the CI/CD pipeline. Technically CI starts from receiving code changes and ends at successfully register a container image.

Fail to register a resulting container image will prevent CD from progressing upon referencing the image. In other words, a trusted registry is like a connecting tissue holding and keeping CI and CD fully synchronized and functional with the associated container images. A generated container image does not flow through every step of a CI/CD pipeline, the image is however the focal point to the validity of produced results. As shown in the above diagram, I used a dashed line between CI and CD to indicate there is a dependency of the trusted registry. Failing a registration will eventually fail the overall process.

Closing Thoughts

The essence of CI is automatic testing against defined criteria at unit, function and integration levels, as configured. These criteria are basically test cases which should be developed prior to code development as acceptance criteria. This is a key. Development must fully address these test criteria at coding time to build in quality.

Software supply chain is a better way. Wait, make that a much better way than just developing applications. I remember those days when every release to production was a nightmare. And code promotion was an event full of anxiety, numerous crashes and many long hours. Good riddance, so long those days. CI/CD and particularly with containers presents a very interesting and powerful combination for quality and speed, which is unusual to be able to achieve both at the same time. Docker with Jenkins, github and Azure, a software supply chain is in my view more realistic than what many of us believe. Which I will detail in an upcoming post.

Advertisements

What Is and What Is Not Cloud Computing and Why Do I as an IT Pro Care

This was a presentation which I delivered at Stratford University in PDF format. My focus was on bringing clarity of cloud computing to the audiences so we can better articulate the business values of cloud computing and why cloud is or perhaps is not the solution for a business problem.

Additional resources is available at http://aka.ms/selected.

Introduction of Microsoft Virtual Academy at HTTP://AKA.MS/MVA1

A knowledge portal with free self-training contents of Microsoft products and solutions is Microsoft Virtual Academy at http://aka.ms/MVA1. Developed courses as shown below are presented by topics and products for an individual to develop and customize a learning plan to best fit one’s interests and objectives. And the contents are packaged in modules with presentations, videos, whitepapers, etc. and they are available online as well as for download. Each module ends with a self-assessment to facilitate learning. There are also live events like Jump Start and Virtual IT Camps delivered by subject matter experts offering global IT community an accelerated learning experience on virtualization, hybrid cloud, Azure, Office 365, Application development and more.

image

For an IT pros in any discipline and at any technical level, Microsoft virtual Academy offers an effective vehicle for developing IT skills profile and maintaining competency. And it is very easy to access the contents. Simply go to Microsoft Virtual Academy at http://aka.ms/MVA1. Pick and start a course of your interest and register/log in with your Microsoft Account which you can sign up one at http://outlook.com, if not already have one. Your Microsoft Account also comes with 15 GB OneDrive (i.e. cloud storage), Outlook-alike Internet email, and Office Web Applications all free.

Once you have registered in Microsoft Virtual Academy, it is now up to you how fast and how far you would like to go from virtualization to Azure, hybrid cloud and DevOps. Do it. Learn it. Enjoy it. And be an IT pro.

Deploying windows 7 and windows 8.1 enterprise clients to Microsoft azure available for msdn subscribers

As announced at TechEd 2014 North America, deploying Windows 7 and Windows 8.1 Enterprise clients to Microsoft Azure is now available for MSDN subscribers. The following is a typical user experience.

Log into Microsoft Azure Management Portal with a MSDN subscription account, click VIRTUAL MACHINE workspace, then click NEW to start the process of deploying a VM interactively.

image

Click either QUICK CREATE or FROM GALLERY. With QUICK CREATE, pick the dropdown of IMAGE and click “more images” to bring up the image gallery.

image

Or FROM GALLERY, bring up the image gallery directly.
image

From the image gallery, scroll down to select an intended Windows 7 or Windows 8.1 client and proceed with the deployment process as shown below.

image

image

image

Notice now there are VM AGENT, CONFIGURATION EXTENSIONS, and SECURITY EXTENSIONS available. Here Microsoft Antimalware is to be included in the VM deployment.

image

Once deployed, we can now capture image while a VM is still running as opposed to previously a VM must be first syspreped. Here, simply click CAPTURE while highlighting the running VM to create an image.

image

Once captured,the image of a VM is placed under IMAGES tab as shown, here while the VM is still running.

image

Remotely connecting to a VM is much the same as before. Click CONNECT from the bottom black menu bar, while highlighting the VM on the instance page. After providing the admin credentials and accepting the certificate for establishing secure commination, RDP brings a user directly into desktop. Here notice Task Manager reveals that the antimalware is running as configured.

image

And the Start screen shows a standard Windows 8.1 layout with a power button placed next to user icon on upper right corner.

image

This is a Windows 8.1 Enterprise client on demand deployed in minutes in Microsoft Azure.

Windows Azure Infrastructure Services IP Address management (Part 2 of 2)

This two-part series details the IP address management of Windows Azure Infrastructure Services including:

We can use Windows Azure PowerShell to assign a static IP to a VM deployed to a Windows Azure virtual network (VNET). The following explains.

Assigning Static VNET IP Address to a VM

imageTo assign a static IP to a VM, use PowerShell. And the VM must be deployed to a VNET. Windows Azure PowerShell v0.7.3.1 released on March 11, 2014 supports static VNET IP address. To install Windows Azure PowerShell and connect to your Windows Azure subscription, follow the instruction at http://aka.ms/AzureCmdlets.

 

image

In PowerShell ISE Command Add-On pane, simply search the string, azurestat, you will find out there are four Windows Azure PowerShell cmdlets associated with managing a static IP of a VM deployed to a Windows Azure virtual network, as shown on the left.

A typical routine starts with using Test-AzureStaticVnetIP to confirm the availability of a target IP address. The cmdlet returns a list of suggestions if the target IP address is not available.

Once the availability of a target IP address is confirmed, run Set-AzureStaticVNetIP and Get-AzureStaticVnetIP to assign and verify the IP assignment of a target VM. And as needed, use Remove-AzureStaticVnetIP to remove a static IP from a VM object.

With a static internal IP addresses, VMs in a VNET will maintain the same IP addresses across shutdown, reboot, and re-imaging. And the deployment order of VMs to a VNET does not impact the IP assignments anymore. Supposed two VMs were at the state, Stopped Deallocated, and then later get started again in a random order. The new internal IP addresses will likely be different. However if the two VMs were assigned with static IP addresses, the same IP address will be in the VM upon restart.

Sample Session of Assigning a Static IP

The following user experience is based on Windows Azure PowerShell already configured with connectivity to a Windows Azure subscription by following the instructions provided at http://aka.ms/AzureCmdlets.

imageThree VMs, vm1, vm2, and vm3 were deployed in order to myTestNet which is configured with 10.0.0.0 address space. Upon the initial deployment, 10.0.0.4, 10.0.0.5, and 10.0.0.6 are assigned to the three VMs as shown. VM2 is the target VM for a static IP assignment.

 

image

For this particular deployment, it is obvious which IP addresses are available. In an event that Windows Azure Management Portal is not accessible, we can use Test-AzureStaticVnetIP to confirm the availability of a target IP address within a VNET.

 

 

imageOnce a target IP address is confirmed available, we can use Set-AzureStaticVnetIP to assign a static IP to a target VM. Here, on the left I use a small routine to first test the availability of a target IP, 10.0.0.9, in an intended VNET. If the IP is available, assign it to a target VM, here vm2. A sample script is available at http://aka.ms/StaticVnetIPSample.

 

imageNow vm2 has an IP address as 10.0.0.9. If this is a static IP address, redeploying all three VMs will not change the IP address of vm2 at this time.

 

 

imageTo redeploy the three VMs, I first shut down them from Windows Azure Management Portal and brought all to the state, Stopped (Deallocated), which released the internal IP address of each VM. As shown on the left, neither VM had an IP address.

 

imageThe three VMs were at this time started in the order of vm3, vm1, and vm2. This order is different from that of the initial deployment. And upon the deployment, notice that vm3 which was first started got the IP address, 10.0.0.4. The 2nd started VM was vm1 which got 10.0.0.5. Nevertheless, the last one started, vm2 (instead of getting the next IP address, 10.0.0.6) maintained the assigned static IP, 10.0.0.9. In other words, once a static IP address is assigned to a VM instance, the deployment order no longer impact the IP address assignment of the VM.

Best Practices for Static IP Assignments

The recommended usage pattern is to employ separate subnets for static IP address VMs and dynamic IP address VMs. Use a static IP address subnet for all static IP address VMs, while it is needed to specify a different static IP address for each VM deployed to this subnet. And use a dynamic IP address subnet for PaaS web/worker roles and those Infrastructure Services VMs without the needs for static IP addresses.

Managing VMs with static IP assignments in a separate subnet can help standardize the processes and configurations, hence better automation and streamlined operations.

Additional Resources

Windows Azure Infrastructure Services IP Address management (Part 1 of 2)

This two-part series details the IP address management of Windows Azure Infrastructure Services including:

We will first examine some basic concepts to better understand how IP addresses are assigned in Windows Azure and what are the implications.

VIP vs. DIP

Upon deployment, a Windows Azure VM instance has two IP addresses:

  • A VIP (Virtual IP address) is the public IP address pointing to the cloud service where the VM is deployed to. Notice that a VIP once assigned is not released from a cloud service till every VM instance in the cloud services has either a “Stopped (Deallocated)” status or deleted.
  • a DIP (an internal IP assigned by Windows Azure with DHCP) is the IP address assigned to the VM for communicating within Windows Azure. Notice that a DIP once assigned is not released from a VM till  the VM has a “Stopped (Deallocated)” status.

image

The following illustrates a conceptual model where multiple VMs are deployed to a cloud service. A VIP is assigned to Windows Azure’s public interface and pointing to this cloud service, while each VM within the cloud service has an individual DIP assigned by Windows Azure via DHCP.

image

Notice that if these VMs are deployed to a virtual network, a DIP will be assigned from or released to the address pool defined in the virtual network configuration.

Endpoint vs. Port

In Windows Azure, accessing a cloud service from Internet requires an endpoint which is a pair of two ports associated with the VIP of the cloud service. The public port of an endpoint is the one facing Internet, while within Windows Azure the corresponding port is the private port. The above diagram depicts the concept of an endpoint as the vehicle to access a cloud service from Internet, while a defined endpoint effectively connects a public interface and a private one of a cloud service with a port translation at the edge of Windows Azure where the VIP points.

Stopped vs. Stopped (Deallocated)

A Windows Azure VM instance can be shut down with two fundamental ways. One is to shut down a VM within the VM instance itself as shown below. This will bring the VM to a “Stopped” state. At this time, although the instance is stopped, it is however not deallocated, and consequently it is still being charged by minute based on the pricing model as detailed in http://aka.ms/waPrice.

image

Another way to stop the VM instance is to operate directly from Windows Azure Management Portal by highlighting a VM an click the Shutdown button from the black menu bar as shown above. Which in addition to shutting down the VM, it also deallocates the instance as the status, Stopped (Deallocated), indicated. At this time, the VM instance is not being charged any more.

The storage cost of a VM is introduced by the associated VHD file which include an OS disk and additional data disk, if added. This storage cost is always there since regardless the state of a VM, storage capacity is consumed to store a VHD file which is stored as page blobs in an associated storage account and depending on if geo-replication is maintained, there may be additional storage and transmission costs. The article, http://aka.ms/HADR explains the two storage account types and how they work.

IP Assignments in Windows Azure Virtual Network (VNET)

When allocating addresses, Azure reserved the first three and the last IP addresses in an address space and a subnet. For instance, defining an address space, 10.0.0.0/24, as the following results in a usable address range of 10.0.0.4 to 10.0.0.254 where the first three and the last IP addresses of this address space, i.e. 10.0.0.1-3 and 10.0.0.255 are reserved for Azure’s use. This behavior is consistent throughout the subnets. Essentially any address space allocated in Azure, the first three and the last one are reserved for system use.

image

When deploying VMs to a VNET, the DIPs (i.e. internal IP addresses) of VMs are allocated from a configured address pool (as defined in VNET) in the order of each VM is deployed. Therefore deploying the same VMs in a different order to the VNET or deallocating then redeploying VMs in a VNET will likely result in different internal IP addresses assigned. For example, two VMs in a VNET had had a Stopped Deallocated state and then both were restarted in a random order. The new internal IP addresses assigned to the two VMs will likely be different than those internal IP addresses previously assigned before deallocation. And it is an issue for VMs requiring persistent IP addresses throughout the lifetime of those VMs. However, if a static IP address is assigned to a VM, the same predictable IP address will be assigned with the VM upon restart. For a VM deployed to a VNET, a static IP can be assigned to the VM using Windows Azure PowerShell.

In Part 2, we will walk through a sample session on assigning a static IP to a VM which is deployed to a VNET.