bringing current cloud talk to you through my experiences

Dell Technologies

Dell Technologies: Helping customers with their hybrid journey using Azure Stack

In my engineering role within our Azure Stack product team at Dell Technologies, I have the opportunity to talk about what value Microsoft and Dell Technologies bring to our customers’ hybrid strategy for our internal sales teams and our important partners. In addition, I also get to discuss these same values directly with our customers. In these discussions, I get to hear about their cloud goals, requirements and provide deep technical guidance to ensure that the targeted solutions will meet their objectives. There are many strategies that come out of these discussions that customers are looking to achieve; they could be looking to reduce their OnPrem IT spend by moving their core infrastructure to a public cloud or they may be looking at how they can target workloads that should live in the public cloud do so while still supporting infrastructure OnPrem in a more modernized way, whether it be application modernization or IT modernized management. It’s those conversations where the customers are seeing a real business value by taking a hybrid approach across their organizations. This ability to harness the power to embrace both public cloud and local datacenter, and may provide edge capabilities, is what is enabling customers to improve their IT agility and efficiencies. At Microsoft’s core, it’s always been about the Enterprise and running workloads within the customer’s datacenter. With the advent of Azure, Microsoft continued to see the importance of the OnPrem infrastructures that customers have traditionally invested in while still seeing the importance that public cloud infrastructures bring. This mindset has allowed Microsoft to invest and build their assets that are uniquely built for hybrid. Many of these hybrid strategies evolve into delivering new application patterns using intelligent cloud and intelligent edge.

Over the past couple of years, Microsoft has been honing their hybrid strategy to enable their partners, like Dell Technologies, to deliver modernized infrastructure built on Enterprise-proven assets integrated with connectivity to Azure Services. With the Azure Stack portfolio, customers can find the solution that meets their hybrid goals. Each of these have different delivery models, control planes, services, and targeted workloads. These variations across the Azure Stack ecosystem help to support complex customer requirements, many through Microsoft validated or Integrated systems via their hardware partners.

Before we jump into what those solutions look like from Dell Technologies, I’m going to circle back to our customer discussions and what we look to glean from these discussions to identify the appropriate solutions to meet their requirements. During these informative meetings, we want to understand:

  • Use-Case Scenarios
  • Current OnPrem investments
    • Are they heavily invested in one virtualization Platform – i.e. Microsoft, VMWare, nothing, or something else?
  • What are their long term OnPrem and Cloud goals
    • Are they going to shift their OnPrem investments to the Public cloud?
    • Investments in one Public Cloud ecosystem or heavily skewing toward one over another
    • Have they identified workloads that have business or technical requirements that would prohibit them from living in public cloud
    • Investing toward application modernizing to take advantage of cloud-native design principles or simply a lift-and-shift to the cloud of their current workloads?
      • Container Workloads – Kubernetes/AKS/GKE/etc, OpenShift, AWS ECS,
    • Moving to streamline development pipeline operations w/ DevOps
  • Management plane
    • Standardizing on a cloud platform control plane; Azure Resource Manager (ARM), Amazon EKS, Cloudflare or VMware Cloud Foundation (VCF)

This list is not exhaustive but you get the idea that there are lot of things to consider as a customer traverses their path to the cloud and considerations they must be aware of, as well as their decisions, for both their OnPrem and Cloud investments. This includes things they may not have considered as well, such as: learning new tools or platforms, costs associated with unforeseen expenses in the cloud, application migrations or re-architecture requirements. As we discuss all these things with the customer, we want to ensure they are thinking about these as we move through this engagement. They will not all be understood, though sometimes they are, but it’s a case of planning and re-visiting the plan often.

Breaking down this list to bucketize them will help arrive at the best solution. Depending on the outcome of these discussions, you may find that the customer’s, or your, initial thoughts around a specific solution may not meet their overall requirements and you need to shift entire solution sets. That’s ok. At Dell Technologies, we have a broad playbook that allows us to shift to something else that is a better fit for their needs. We don’t want to shoehorn a solution just because that is the only thing we can provide. That path will not end in a satisfactory way and lead to customer issues and may abandon the course. Not good.

This is going to be focused around the Azure Stack portfolio so assuming our conversations and outcomes from the above list are keeping on this track, we will discuss some of these considerations.

If we look at the image of the Azure Stack portfolio, there are three distinct product offers. Each one has services and values that provide the ability for our customers to deliver their application needs directly to their customers:

Starting with Azure Stack Hub, which was the first solution delivered as part of the Azure Stack family, provides customers the ability to deliver a cloud-native experience OnPrem, behind their firewall, either connected for fully disconnected to Azure Public. These allows customers to deliver applications that take advantage of Platform-as-a-Service (PaaS) services, such as Azure Functions, Web and API applications, Event Hub, and IoT Hub that can be consumed from a local Hub instance but also has the same control plane as Azure Public resources. In fact, Azure Stack Hub is just another region that can be targeted like other Azure regions (i.e. US East, US West, etc) using the same tools you are familiar with; Azure CLI, PowerShell, and even Visual Studio, for example. Just keep in mind that Hub is a subset of Public Azure, so you need to ensure that the services you want to deploy to Hub are available. Do not assume that it is available for your application use since it does not have the scale factor that Azure Public has. Scale for Hub is from 4 – 16 compute nodes per region/scale-unit. However, if a customer is considering a full investment into the Azure ecosystem and they are trying to unify their control plan between OnPrem and Public cloud, using ARM, this could be the best path for them. Think of Azure Stack Hub as a development platform which provides:

  • A way for customers to implement a standardized DevOps practice between Azure Public and OnPrem
  • Modernize OnPrem applications from traditional monolithic application models to lightweight microservices while still close by to the OnPrem data source instead, keeping latency low and performance high while allowing time for the application to be migrated over time.
  • Delivering cloud-native applications in a completely disconnected pattern for edge or far-edge deployments. Think in terms of remote locations: Mines, Combat zones, Oil and Gas industries, etc.
    • Example: Process IoT sensor data from a mining operation that measures critical mine shaft data (air quality, oxygen levels, gas levels, seismic activity) that are processed locally in real-time for the safety of the miners.
  • Support of containers using Azure Kubernetes Engine (AKSe) and, soon, Azure Kubernetes Service (AKS).

Moving to our right in the image, to the center solution, we are to Azure Stack HCI.
Out of the portfolio, Azure Stack HCI has evolved the most over the last few years. Initially referenced as Windows Server Software-Defined Datacenter (WSSD), more of reference architecture, it was built upon a strong virtualization platform in Hyper-V, storage built upon Storage Spaces Direct (S2D) and, the optional, Software-Defined-Networking (SDN) to deliver a modernized, scalable virtualization platform. Dell Technologies was an early provider with WSSD solutions through our PowerEdge-based S2D Ready Nodes product line that were the building blocks to support virtual machines for those customers who were strong Microsoft datacenters. Over the last year, Microsoft has invested heavily in maturing the platform and is now a 1st-party citizen with Azure Public services; Azure Monitor, Azure Backup, etc. This provides a native integration layer directly with Azure that other solution providers just cannot deliver today. What this translates to is that customers running Azure Stack HCI can have a highly scalable infrastructure built on a proven and reliable Hyperconverged Infrastructure hardware delivered as an integrated experience. Dell Technologies has taken the years of experience delivering Hyperconverged Infrastructure solutions across different platforms, like VxRail using VMware to engineer and deliver these solutions. Scalability is key to the platform and Azure Stack HCI will scale from 2 nodes to a maximum of 16 nodes. Over the past 6-8 months, Microsoft and Dell Technologies have gone further and improved the Azure Stack HCI platform by introducing:

  • Dell EMC Integrated System for Microsoft Azure HCI, improving the original Ready Node concept that we launched with a few years ago by adding a new AX product line specifically designed and targeted for Azure Stack HCI. This new platform provides new full stack Lifecycle Management features that are simply not available in Ready Nodes or PowerEdge servers.
  • Microsoft introduced HCI-OS, a pared down OS focusing only on HCI services optimized to deliver best in-class virtualization. This OS uses an Azure subscription-based billing for the HCI Operating system and will provide faster updates, aligning more to the Azure release model. Also, it will target HCI features, such as stretch-clustering, that will not be available on Windows Server Datacenter.

All of this allows for a performance-driven platform optimized for delivering the best experience for virtualized workloads. Circling back to our original list above, use-cases for HCI will be targeting:

  • Windows Server or Hyper-V Refresh from Server 2008/2012/2016/2019 to Azure Stack HCI: If a customer is Microsoft-centric today and has a Hyper-V presence, it’s a natural progression to the more modernized HCI platform.
  • Virtual Desktop Infrastructure (VDI) – As you can imagine, with the pandemic, many customers have looked at investing in VDI and Azure Stack HCI is a great consideration for this use-case. Powerful and performance driven, Dell Technologies has configurations that can support the demand.
  • Remote Office / Branch Office (ROBO) or Edge Infrastructure – Cluster sizing down to a minimum of 2 nodes makes using HCI at the edge or branch office.
  • High Performance, latency sensitive applications – Any applications that are highly performance driven or require low latency, such as Microsoft SQL server.
  • Azure Kubernetes Service (AKS) – Microsoft released AKS recently for Azure Stack HCI making container support running OnPrem a reality giving you the ability to support containers using a proven and scalable solution like AKS.

For management, HCI is done with Windows Admin Center. Unlike Hub, which uses ARM, using Windows Admin Center’s native integration of HCI OS Cluster management operations and Dell Technologies Lifecycle Management integration, allows customers to manage many HCI cluster instances from a single instance.

The net here is that if the customer is a strong Microsoft customer today, looking to invest more in Microsoft or shift their support to Microsoft, Azure Stack HCI is a strong factor aligning to their use cases.

Finally, let’s talk about the final solution in the image, Azure Stack Edge. Unlike the other two in the Azure Stack portfolio, Azure Stack Edge is delivered directly by Microsoft through the Azure portal. This solution is also very specific in the target workloads. Since it isn’t scalable like the other two platforms, this is looking at those workloads which are targeting lighter duties for IoT and/or Machine learning (ML) capabilities. Also, keep in mind that Edge always comes with a GPU or FPGA for those ML-focused workloads so that cost is always factored into the price. If you aren’t looking to support workloads of this type or don’t need GPU/FPGA support, it may not be the best or cost effective for your particular needs. Since this is directly sold by Microsoft, Dell Technologies typically does not speak much about it except that it is available and what the targeted workloads typically are. For more information for Azure Stack Edge, I urge you to read more about it at Azure Stack Edge | Microsoft Azure.

One thing that we have not discussed today, Azure Arc. Azure Arc’s goal is to offer simplified management, faster application development, and consistent Azure Services. You can easily organize, govern and secure OS’s, Windows or Linux, SQL Server and Kubernetes clusters across datacenters, the edge and even multicloud environments directly from the Azure Portal.

We have discussed quite a bit today but this is only the tip of the spear. There are a lot of considerations for any of these solutions and within those solutions, different configuration options to support the targeted workloads. Below you will find reference documentation to align to the information I’ve reviewed today. Also, if you are considering a move to the Azure Stack Portfolio, I urge you to reach out us at Dell Technology or directly with Microsoft to get a conversation started. We can partner with you on your journey, answer any questions and help size any solution that is required.

Integrated System for Azure Stack Hub | Dell Technologies US

Microsoft Azure Stack HCI | Dell Technologies US

Azure Stack Hub | Microsoft Azure

Azure Stack HCI – Hyperconverged Infrastructure | Microsoft Azure

Azure Arc – Hybrid & Multicloud Management | Microsoft Azure

Deploying the AKS-Engine to Azure Stack Hub

Hello…I’m back again but I’m having to change direction a little bit from my planned blog for this month. Initially, my follow-up to my previous blog was to be about going through a walkthrough of deploying workloads to Azure Stack HCI that has AKS deployed to it. Basically, putting it all together after you have AKS up and running. However, I’ve hit a snag with the latest AKS Preview and have been unable to get it deployed to my 2-node Azure Stack HCI instance.

So, with that, I’m going to change it up a little and shift my focus to Deploying the AKS Engine to Azure Stack Hub followed by deploying a Kubernetes cluster to it. I’m a bit surprised, really though, as my Azure Stack Hub ASDK instance – a single node instance of Hub that allows you do some functionality testing and get familiar with how hub works – is still running as I’ve not done much with it as of late and it has been unable to start in my nested virtualization environment. Anyway, spent a few minutes this morning working through the startup issues and got it to boot up and here we are…Recovering from that may be a story for another day.

Before I jump into the deployment process here for the AKS Engine, I did want to note what is different here versus what I did in my last blog with deploying AKS to Azure Stack HCI. The AKS Engine is a command-line tool that allows you to deploy and manage a Kubernetes cluster in both Azure Public and Azure Stack Hub. You use this to create, upgrade, and scale Azure Resource Manager (ARM) native clusters. On Hub, this can be used to deploy a cluster in both connected or disconnected environments. Understand there are difference between running the AKS Engine in Azure Public and onPrem with Azure Stack Hub. Be sure to check out those difference to have the right expectations on what you can and cannot do with the AKS Engine on Hub.

With that out of the way, so let’s do this…

We need to first get our prerequisites inline:

Azure Stack Hub Prerequisites that will need to be carried out by a Cloud Operator. This may be you if you are using an ASDK for this exercise.

Azure Stack Hub 1910+Yes AKS Engine requires Azure Stack Hub 1910+. Currently the latest version is 2008.
Linux Custom Script Extension 2.0.6+RequiredThis be available in your subscription. If it isn’t, reach out to your Cloud Operator to have them add it
AKS Base imagesRequiredLike the custom extension scripts, this needs to be available to your subscription so have your Operator add it. If you are the Cloud Operator, refer here to get it added to the Marketplace.
Service Principle Identity (SPN)RequiredNote that if you are using an AAD SPN, Internet access is required from the VMs in the Kubernetes cluster so that the SPN can be authenticated with Azure AD. If Internet is not available, the Kubernetes cluster will not be functional.
Assign SPN to the Contributor RoleRequiredYou Azure Stack Operator will need to complete this action before you get started. Refer to Assign a role for more instructions

The next Prerequisites will be done by you:

Azure Stack Hub SubcriptionYes To deploy any workloads in Hub, you need a subscription from the tenant portal. Go to Azure Stack Hub Subscription for more info
Resource GroupOptionalI’ll be using MySub-rg for this exercise. You can create a new one or use an existing one from your tenant subscription.
Private Public KeyRequiredRefer to SSH Key Generation for instructions to create a public/private key pair.

This blog assumes you have already reviewed and implemented the above Prerequisites so will not be walking through those aspects here.

We’ve got all the prerequisites secured and we are ready to go. First, we will need either a Linux or Windows VM that we’ll use to host the AKS engine in order to deploy and manage a Kubernetes cluster. For this exercise, I’ll be using a Linux VM based on Ubuntu 18.04 LTS. This is available in my Hub marketplace. So, I’ll  create a new resource for this VM using the portal as you can see below. If you are using Windows, the steps will be the same here.

For the creation of the VM, you will need to specify a few parameters that are specific to your environment. Note, that you will need to provide your SSH public key here that was created as part of the requirements at the start of this. You can use an SSH password but that is not as desirable as it offers less security as a certificate authentication practice.

Choose the Azure VM type/size that you need to support your workloads. For my case in this exercise, I’m taking the default (denoted by the star)

On the next screen, configure to your needs but I chose the default except for the exposed public inbound ports since I need to connect to this VM by SSH. I allowed for the SSH port to be opened.

Once everything is ready to go and you’ve reviewed it for accuracy, click OK to begin the Linux VM deployment.

This will take a bit of time but once complete, you will see it listed in the Virtual Machines blade (found on left menu pane) and this new VM should reflect that it is running.

I now have the VM up and running. You will connect to it and will need to determine what version we will use based on the AKS Base Image that you pulled down as part of the prerequisites. Again, in my case, the AKS Base Image is using the Ubuntu 18.04 LTS version so I need to ensure the AKS Engine version I use aligns to that. This can be determined using the table found here. Since I’m using the latest versions available for both Hub (2008) and Ubuntu 18.04 LTS, my AKS Engine version will be V0.60.1.

Having this info, we will now issue the following command from my Linux console:

curl -o chmod 700 ./ –version v0.60.1

Once we have completed the above command, let’s verify the installation is OK.

  1. Connect to your client VM
  2. Run the following Command:
aks-engine version
  • Next, if you are using the ASDK and, thus, a self-signed certificate, you need to explicitly add the root certificate to the trusted certificate store of the machine. You can find the root certificate in the VM at this directory: /var/lib/waagent/Certificates.pem.

Copy the certificate file with the following command:

sudo cp /var/lib/waagent/Certificates.pem /usr/local/share/ca-certificates/azurestacka.crt sudo update-ca-certificates

OK…we’ve got our AKS-engine installed on our host and our self-signed certificate trusted; we are now ready to deploy a Kubernetes cluster with the AKS engine on Hub.

To do this, we will first need to define our cluster specification that uses a JSON file for our API model. This API model is what is used by the AKS Engine to create the cluster specification.

  1. Let’s grab the API model file from:
curl -o kubernetes-azurestack.json
  1. Open the downloaded JSON file, kubernetes-azurestack.json
nano ./kubernetes-azurestack.json
  1. Find the appropriate orchestratorRelease and orchestratorVersions in this file for the Kubernetes version that your AKS-Engine supports. For mine, it is version 1.17 and 1.17.17, respectively.
  1. From there, find the customCloudProfile in that section and provide the URL to the tenant portal. In my case, since this the ASDK, it is https://portal.local.azurestack.external. If you are using ADFS in your Hub deployment, you will need to also modify the identitySystem as well to reflect this but otherwise, you can leave it blank.
  1. Next, move down to the masterProfile to set the following fields in the snapshot:

A couple of notes on these fields…

  1. dnsPrefix – This needs to be a unique string that will identify the hostname of VMs. I’ve used the name of my resource group.
  2. Count – This is the number of masters you want for your deployment. The minimum is 3 for any HA deployment but for non-HA/Test environments, 1 is sufficient.
  3. vmSize – Enter a supported Azure Stack Hub VM size
  4. distro – Define what Linux distro you are using that maps to the correct AKS-Engine and Kubernetes version you are using. In my case, this is Ubuntu 18.04
  5. Then we want to modify the section right below this for agentPoolProfiles

More info on these specific fields:

  1. Count – Enter the number of agents you want for your deployment. The maximum of nodes to use per subscription is 50.
  2. vmSize – Enter a supported Azure Stack Hub VM size
  3. distro – like the previous entry, provide the linux distro you will be using.
  4. Finally, in the linuxProfile section:

The public key is what was created as part of the prerequisites at the start of this blog. Note: This private key must have no spaces and must be pre-faced with ssh-rsa like in the output above.

Now, we’ve prepared our API model specific via the downloaded JSON file, mapping it to the requirements for our environment. Next up, let’s get that cluster created.

Before you start the actual deployment, it’s always good to ensure that Hub is running correctly so you can do this yourself if you are the Cloud Operator (or running the ASDK) or you will need to ask your Cloud Operator to do this if you don’t have access to Hub directly.

Assuming you do have the required Cloud Operator status, you run Test-AzureStack Powershell command from the Privileged Endpoint. For more information on connecting to the Privileged Endpoint, please refer to “Use the privileged endpoint in Azure Stack Hub”

Let’s issue the following command to deploy our Kubernetes Cluster:

aks-engine deploy \
–azure-env AzureStackCloud \
–location local \ If using ASDK, this should reflect local, otherwise should be location for your multi-node stack

–resource-group MySub-rg \ Modify to reflect your Resource Group
–api-model ./kubernetes-azurestack.json \
–output-directory MySub-rg \ This is going to be in my Resource group that was specified above
–client-id xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \ This is the SPN application id you created as part of the Prerequisites
–client-secret xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \ SPN client-secret that was created as part of the Prerequisites –subscription-id xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \ Sub ID to your tenant
–identity-system adfs  Optional: Only required if using ADFS, otherwise, you can omit.

Once that completes, we should have an active Kubernetes cluster so let’s verify it is working as expected.

At completion, you should see that the deployment succeeded like below:

To verify it’s working as expected, we need to grab the public IP address of any one of the master nodes VMs found in the Hub portal:

Then, from a machine that has access to your Hub instance, connect via SSH into this master node using an SSH client, like PuTTy. For the SSH username, use the default ‘azureuser’ or the one that you specified in the API model configuration file. I used sysadmin for my case. Also, use the private key that you provided for the deployment of the cluster.

If successful, you should now be at the Master node’s prompt where you can issue the following commands to create a sample deployment of a Guestbook application called mongo-deployment. Note: This only works for connected Hub instances.

kubectl apply -f

Query the list of pods:

kubectl get pods

Then you can also view the deployment logs using:

kubectl logs -f <pod name>

You can also issue the command:

kubectl cluster-info

As well as review the node states:

kubectl get nodes

That’s it! You have successfully deployed the AKS-Engine and a Kubernetes Cluster to your Azure Stack Hub instance. You’ve also deployed a sample application to the cluster. You can clean up the sample application using kubectl as well:

kubectl delete deployment -l app=mongo

A couple of things to consider now that you have it running. Since this is not AKS in the sense that it is a managed service like you would expect to see in Azure Public. Some general housekeeping is up to you, the owner, to do. One of those chores is to rotate your service principal secret regularly. For more of a complete reference to the AKS Engine, please refer to the online documentation at GitHub.

I hope you found this information useful. Check back soon for a follow-up here where I’ll configure the Kubernetes Dashboard which is included with Kubernetes that provides a clean way to view your cluster status. Also, I’ll be walking through workload deployments against both Azure Stack Hub and Azure Stack HCI.

Deploying Azure Kubernetes Service on an Azure Stack HCI OS Cluster

Hopefully you have heard this week’s announcements for Microsoft’s latest Operating System offering; Azure Stack HCI OS is a specialized virtualization operating system, deployed in an integrated system in a hyperconverged model, and delivered as an Azure hybrid service. Delivering as an Azure hybrid service allows the OS to receive the latest and greatest in up to date security, performance, and features that you are already used to when consuming Azure services. With Azure Stack HCI, you can deploy and run Windows and Linux virtual machines in your datacenter or at the edge, using your existing tools, processes, and skillsets while simultaneously extending your datacenter to the cloud, with the ability to integrate Azure capabilities such as Azure Backup, Azure Monitor, Azure Update Management, and Azure Security Center.

Today, we are going to talk about bringing the Azure Kubernetes Service (AKS) directly into Azure Stack HCI. This brings an on-premises implementation of AKS, which automates the management and orchestration of containerized applications at scale. Before we dig into what it looks like to deploy AKS on Azure Stack HCI, let’s first talk about why you would use Kubernetes and how it’s transforming the cloud and datacenter landscape. Of course you can manage a few containers manually using Docker or similar tools, but when you build for any scale beyond a lab, when applications make use of tens, hundreds, or thousands of containers, which is where the need for at-scale management and orchestration comes into play, and where Kubernetes shines.

Kubernetes, in essence, is an open-source orchestrator for automating container management at scale. AKS takes this tool and simplifies the management, deployment, and governance of it both in Azure Public Cloud and on-premises. Within Azure Stack HCI, AKS significantly simplifies and streamlines the deployment of a Kubernetes infrastructure by providing wizards for setting up the infrastructure and integrating it into Azure Stack HCI.

Some of the functionality offered by AKS during preview and initial launch includes:

  • Deployment containerized applications at scale to a cluster of Virtual Machines, known as a Kubernetes cluster, running across the Azure Stack HCI cluster
  • Automatic failover when a node in the Kubernetes cluster fails
  • Deploy and manage both Linux and Windows-based containerized applications
  • Schedule workloads
  • Monitor health
  • Scale up or down by adding or removing nodes to the Kubernetes cluster
  • Manage networking
  • Discover services
  • Coordinate application upgrades
  • Assign pods to cluster nodes with cluster node affinity

There are several important features that AKS brings to Azure Stack HCI that simplifies the process of setting up Kubernetes, including:

  • A Windows Admin Center (WAC) wizard for setting up Kubernetes and its dependencies; kubeadm, kubelet, and a Pod network add-on
  • A WAC wizard for creating Kubernetes clusters to run your containerized applications
  • PowerShell cmdlets for setting up Kubernetes and creating Kubernetes clusters to allow you to script the host setup and Kubernetes cluster creation

Let’s get started with deploying an AKS instance to our Azure Stack HCI cluster…

To start, I’m going to do this deployment using PowerShell but you can use the Windows Admin Center as well. Additionally, I’m deploying AKS to a 2 node Azure Stack HCI cluster that is deployed in Hyper-V but can be deployed to any of the following:

  • 2-4 node Azure Stack HCI Cluster
  • Windows Server 2019 Datacenter failover cluster
  • Single node Windows Server 2019 Datacenter

First, will need to download and install the AksHci PowerShell Module. This can be downloaded directly from Microsoft’s Azure Kubernetes Service on Azure Stack HCI registration page. The downloaded package contains the which has the PowerShell module. You will need to extract this zip file to %systemdrive%\program files\windowspowershell\modules on each of the Azure Stack HCI nodes.

Here is what the PowerShell Modules folder should look like on each of the nodes once extracted, adding the AksHci, Kva, Moc, and MK8SDownloadAgent modules added.

Next, we need to import the new AksHci module:

Upon import completion, close all open PowerShell windows and then prepare the nodes for deployment. We do this by running checks on every computer to see if all the requirements are statisfied to install AKS on Azure Stack HCI. From an Admininstrator PowerShell prompt, run the following command: Initialize-AksHciNode

You should see “Done” displayed in green text as you above. This means we are ready to configure your deployment using Set-AksHciConfig

This command has the following options but only imageDir, and cloudConfigLocation parameters are required on any multi-node deployments.

Although Microsoft states that only imageDir and CloudConfigLocation are required, in my deployment, I did have to specify the vnetName parameter as well. Note that the path to the directory for both imageDir and CloudConfigLocation on a multi-node deployment must point to a shared storage path such as C:\ClusterStorage\Volume01\Images.

The location needs to be on a highly available share so that the storage will always be accessible. It can also live on an SMB share, such as \\Fileshare\Images if preferred. In my deployment, I’m pointing to my HCI cluster volume as shown in the example below.

Once we have all the necessary configurations completed and validated, now it’s time to kick off our new AKS deployment. To do this, we install the AKS on Azure Stack HCI agents/services and the AKS Service Host by running the following command: Install-AksHci

We now have successfullly completed our installation of the AKS Service host so let’s verify that is fully deployed appropriately by running the command: Get-AksHciCluster

As you can see, we have successfully provisioned the host for my clustergroup-management…

In order to access your clusters using kubectl, we will need to get the kubeconfig by running the command: Get-AksHciCredential

It does require to parameters:

  • clusterName – name of the cluster, in this case “clustergroup-management”
  • outputLocation – the path to where you want the kubeconfig to saved to. Default is %USERPROFILE%\.kube

We will use the kubeconfig file later in this blog so keep it handy.

Now that we have AKS successfully deployed to our Azure Stack HCI cluster, we need to create our first Kubernetes cluster…To do this, we will use the New-AksHciCluster command. There are several parameters that can be used which I’ve provided below, however, only -clusterName is required. If you choose to just pass the required parameter, the number of Control Plane Nodes, Linux Nodes will default 1 while Windows Nodes will default to 0.

My first cluster deployment is using the default values but I did specify their parameters. I executed this command directly from my Windows 10 management machine but you can also deploy directly from the Azure Stack HCI hosts.

Let’s get a list of our deployed AKS host and Kubernetes clusters by running the following command: Get-AksHciCluster

If we want to scale a Kubernetes cluster, we can easily do this by running the command: Set-AksHciClusterNodeCount

We now have an AKS cluster fully deployed and operational with 1 Control Plane and 3 Linux worker nodes.

After AKS has been fully deployed, you can see your new kubernetes cluster in the Azure Stack HCI Cluster Manager view within Windows Admin Center. Click on the Azure Kubernetes Service in the lower left corner of the tools pane and this will present your new kubernetes cluster as seen below.

That’s it! You have successfully deployed Azure Kubernetes Service on Azure Stack HCI.

But wait…there’s more…Let’s walk through connecting your clusters to Azure Arc for Kubernetes.

When an AKS on Azure Stack HCI cluster is attached to Arc, it will appear in the Azure portal. Before we start, you will need to ensure that you have the following requirements ready:

  • An AKS on Azure Stack HCI cluster with at least 1 Linux worker node up that is up and running – should have that, we just completed it.
  • You’ll need your kubeconfig file that we generated earlier…This allows us to access the cluster and cluster-admin role on the cluster for deployment of Arc enabled Kubernetes agents.
  • Have the AKS on Azure Stack HCI Powershell module install – again, we have already done this so you should be ready to go.
  • Azure CLI version 2.3+ is required for install the Azure Arc Enabled Kubernetes CLI extensions. Install Azure CLI if not already or update to the latest version to ensure you are at the correct version.
  • You will need an Azure subscription that you own are at least a contributor
  • Run the below commands in a PowerShell Administrative window.

Login into Azure w/ Azure CLI

Register the two providers, below, for Azure Arc enabled Kubernetes:

You can check the registration status by running the following commands:

Next, create a resource group in Azure Public to hold the connected cluster resource. You can use an existing resource in East US or West Europe as well. If you want to create a new resource, use the command below:

Create a new service principal. It’s best practices to create with an informative name but you can also use an existing one if you would like. I’ve created a new one below. If you are using an existing Service Principal account, you will need the service principal’s appID, password and tenant values.

Using the values generated during the Service Principal, we kick off the Azure Arc onboarding using the command: Install-AksHciArcOnboarding

Once the above command completes successfully, the onboarding will continue. You can run the kubectl command above in the output or you can go to the Azure Portal to see if it has been completed. If it has completed the onboarding process, you will see the following in the Azure Arc for Kubernetes blade.

After Azure Arc has completed its onboarding, you can go back into Windows Admin Center Cluster Manager and under the Azure Kubernetes Service tools pane, you will now see that Azure Arc is tied to this cluster instance.

So, to recap, we have walked through deploying an Azure Kubernetes Service host on an Azure Stack HCI cluster. From there we created our first Kubernetes cluster and finally we tied it to Azure Arc in Azure Public.

I hope you found this information helpful. This is a preview and updates are constantly being made to the flow of the deployment so some steps may evolve over time.

In the next blog, we will discuss deploying Windows and Linux applications to your Kubernetes cluster – Stay tuned!




Microsoft Announces Next Phase of Their Azure VMWare Solution to Accelerate Cloud Journey

Microsoft recently announced the next iteration to the Azure VMWare Solution, which aims to make it easier and cheaper to allow migration of on-Premise VMware applications to Microsoft’s Azure cloud. This next generation version, is now available in Preview for US East and West Europe regions, targeting 2nd half of 2020 for general availability.

The new version of the solution enables customers to extend beyond their datacenter or a complete migration of on-Prem VMware apps to Azure without the need to re-architect the applications as they have traditionally required. This, in turn, provides cost savings, lowers effort and risk to get VMWare application workloads to Azure.

Under the covers, the Azure VMWare Solution gives customers the ability to use the same VMWare foundation, VMWare Cloud Foundation (VCF), in Azure as they do today in their private datacenters. This is a first-party Azure Service that’s built and supported by Microsoft and endorsed by VMWare. Since it’s a native implementation of VMware’s VCF it benefits customers to move to Azure without having to learn anything new, plus they are able to get Microsoft’s hybrid use benefit option as well.

Before we jump into the use cases that this solution would target, let’s talk about what VMware Cloud Foundation (VCF) is and how it helps with orchestration and a customer’s cloud journey, especially in a multi-cloud scenario. VCF is a solution that VMWare developed that provides an integrated stack which bundles Compute Virtualization (VMware vSphere), Storage Virtualization (VMware vSAN), Network Virtualization (VMware NSX) and Cloud Management and Monitoring (VMware vRealize Suite) into a single platform that can be deployed on-premises where a customer is responsible for managing the physical infrastructure as well as the virtual machine workloads or be run in an As-A-Service offering in public cloud where the cloud provider, such as Azure, AWS or GCP, manages the underlying physical infrastructure and the customer responsibility is to manage the virtual machine workloads.

Cloud Foundation introduced a new abstraction known as Workload Domains which consist of one or more vSphere clusters that are provisioned by SDDC Manager. These Workload Domains are resource containers with specific policies configured to handle performance, availability and security.

This framework provides a standard architecture based on VMware Validated Designs with the complete lifecycle automation in mind, providing a clear path to a modernized hybrid environment.

Dell Technologies has the longest running modernized datacenter strategy with a broad portfolio of solutions to meet this. One proven solution is VCF on VxRail which delivers an experience you won’t find from any other infrastructure running VCF today. Building upon native VxRail and Cloud Foundation capabilities that bring unique capabilities with unique integrations jointly engineered between Dell Technologies and VMware will simplify, streamline and automate the operations of your entire SDDC from Day 0 through Day 2 operations.

Having on-Prem integrated solutions is a key component of a modernized infrastructure today, and Dell Technologies provides the necessary components to support a strong Hyper-Converged datacenter, but that’s only a piece. What about the Public Cloud? Well, deploying and managing VMware solutions have been available for a while with offerings from Amazon Web Services, like “VMware Cloud on AWS” as well as Microsoft announcing support for 3rd-party solutions in April, 2019 from CloudSimple and Virtustream. However, now Microsoft and VMware have formed a strong partnership to deliver a true 1st-party Azure service that allow enterprise customers run their VMware technology, including VMware vSphere, HCX, NSX-T, and vSAN, on Azure without the cost, effort or risk associated with having to re-architect applications.

Wow…so you can now run native VMware workloads in Azure, but what are some scenarios that you could use for this?

Let’s look at just some of the possibilities…

  • Reduce Datacenter footprint through consolidation and/or retirement
  • Expand Datacenter operations, seamlessly and elastically for on-demand short periods of time to handle capacity constraints on-Prem
  • Disaster Recovery / Business Continuity, using a VMware stack deployed in Azure as a primary or secondary disaster recovery site for your on-Prem VMware-based workloads
  • Application Modernization that allows you to tap into the expanding Azure ecosystem to develop more cloud-native applications in a controlled way without having to abandon or rebuild or your VMware-based environments.

As mentioned previously, Azure VMware solution is in preview but you can review pricing for this new service here.

How Does Azure Stack Hub or HCI fit into this picture?

Today, there is no path to allow Azure VMware Solutions services to run in an Azure Stack Hub or HCI instance as this service is only available to Azure Public scenarios where the VMware workloads will run in Azure in a colocation scenario and then be directly managed by Microsoft. However, as these applications begin to modernize and take advantage of native cloud architectures, it would be possible to deploy those applications to local instances of Azure Stack Hub.