I spy, with my little eye…Encryption at Host in Azure Gov Cloud?

One of the features that has been missing from Azure gov cloud is encryption at host. The restriction of dm-crypt specific to certain Linux operating systems and the cpu overhead using bitlocker makes this a big win, not to forget federal compliances you are trying to achieve. It feels like it is some kept secret and I am not sure why? You still need to access the portal with a special link just to provision with it enabled in commercial cloud. No bicep/arm template examples and a lot of the documentation seems to be from 3rd party blogs. Well, look no further!

I published a quick arm template that enables encryption at host, but before we deploy, we need to make sure the feature is enabled. Check if it is enabled by running Get-AzProviderFeature -FeatureName "EncryptionAtHost" -ProviderNamespace "Microsoft.Compute" and if it is not registered, register it by running Register-AzProviderFeature -FeatureName "EncryptionAtHost" -ProviderNamespace "Microsoft.Compute"

Once the feature has been registered, you can create a VM using this link for gov cloud https://portal.azure.us/?feature.enabledoubleencryption=true&feature.enablehostbasedencryption=true When you get to the disk section, there will be an option to enable encryption at host.

Screenshot of the virtual mahine creation disks pane, encryption at host highlighted.

Using an ARM template is as easy as adding a securityProfile with encryptionAtHost set to true

          "securityProfile": {
              "encryptionAtHost": true
          },

For a complete sample, please go here https://raw.githubusercontent.com/jrudley/vmencathost/main/azuredeploy.json

I haven’t seen any announcements for encryption at host for gov cloud, but then again, I don’t see many for gov cloud to begin with. Hopefully, this makes your FedRAMP and CMMC journey a little easier πŸ™‚

Azure Run Command via API

I had a scenario where I needed an end user to be able to run a few adhoc commands via Azure automation runbook and return the results. I am a big fan of Azure Automation as it has a nice display of the jobs and how it categorizes exceptions, warnings and output. The VM is running Ubuntu, but unfortunately, you cannot run adhoc commands using the Invoke-AzVmRunCommand cmdlet. You need to pass in a script 😦 I tried to do an inline script and also export it out then reference it in the runbook, but it would just display nothing. Knowing that az cli can run adhoc commands, I figured I would research the API.

I was getting no where with the Microsoft docs as the response was not the one I was getting. One simple trick I did was run the web browser developer tools and just monitor the API call being sent from the portal. In the picture below, you can see the API call and the JSON body which has a simple command of calling date. You can copy the API call directly from the devs tools in the specific format you want.

Now that I can make the call, I noticed it is sent asynchronous. Looking at the next call in my dev tools, I saw this URI being called with some GUIDs.

I tried to research this call https://docs.microsoft.com/en-us/rest/api/compute/operations/list#computeoperationlistresult but I didn’t see an explanation for the guid’s in the URI. What I did figure out is that the response from invoke-webrequest has a header key called Location and azure-asyncoperation which both have a URI that matches the call Azure was using in the portal. We can do a simple while loop to wait until the invoke-webrequest populates content which has our stdout from the runcommand. It will look something like this in an Azure runbook:

Connect-AzAccount -Identity

$azContext = Get-AzContext
$azProfile = [Microsoft.Azure.Commands.Common.Authentication.Abstractions.AzureRmProfileProvider]::Instance.Profile
$profileClient = New-Object -TypeName Microsoft.Azure.Commands.ResourceManager.Common.RMProfileClient -ArgumentList ($azProfile)

$token = $profileClient.AcquireAccessToken($azContext.Subscription.TenantId)

$auth = @{
    'Content-Type'  = 'application/json'
    'Authorization' = 'Bearer ' + $token.AccessToken 
}

$response = Invoke-WebRequest -useBasicParsing -Uri "https://management.azure.com/subscriptions/$($((Get-AzContext).Subscription.Id))/resourceGroups/ubuntu/providers/Microsoft.Compute/virtualMachines/ubuntu/runCommand?api-version=2018-04-01" `
-Method "POST" `
-Headers $auth `
-ContentType "application/json" `
-Body "{`"commandId`":`"RunShellScript`",`"script`":[`"date`"]}"

Foreach ($key in ($response.Headers.GetEnumerator() | Where-Object {$_.Key -eq "Location"}))
{
       $checkStatus = $Key.Value
}

$contentCheck = Invoke-WebRequest -UseBasicParsing -Uri $checkStatus -Headers $auth
while (($contentCheck.content).count -eq 0) {
$contentCheck = Invoke-WebRequest -UseBasicParsing -Uri $checkStatus -Headers $auth
Write-output "Waiting for async call to finish..."
Start-Sleep -s 15
}

($contentCheck.content | convertfrom-json).value.message

As you can see, I am using a managed identity and logging in with it. The runbook calls the runcommand with a POST then it hits a while loop to wait for it to finish then output the results.

Azure Devtest Labs “Install Windows Update” not working

I was building out a formula in Devtest labs the other day and added a few artifacts, including “Install Windows Updates”. My goal was to build an ARM template that deploys DTL, creates a VM, then a custom image off that VM. Everything worked great, but as a good IT pro, I double checked my work. Looking at the VM, you can drill down into the artifacts section and see the status of each artifact applied. I had green everywhere, so it looked good. Upon further inspection, I noticed the Windows Update task finished extremely quick.

Clicking on the task to get more details, I saw this:

It just displayed the updates and rebooted. Well, maybe it did install? I checked the update history on the VM and it was empty. It also displayed those updates to be installed. Curious, I went to the PowerShell file in the packages folder C:\Packages\Plugins\Microsoft.Compute.CustomScriptExtension\1.10.12\Downloads\4\PublicRepo\master\0b7a713c381a8cbecf04f92122d1c0b07324871e\Artifacts\windows-install-windows-updates\scripts\artifact.ps1 and saw the cmdlet:

I was able to run this script and reproduce the same output as above. I did some digging and it looks like this cmdlet needs additional parameters now unlike in the past.

I updated it and re-ran the script which displayed:

This is what I would expect to see. Now, the bigger problem is that this is in Microsoft’s artifacts repo https://github.com/Azure/azure-devtestlab/tree/master/Artifacts/windows-install-windows-updates If you are using the public repo for your artifacts and have this specific artifact being consumed, i’d double check to actually make sure you are indeed patching your operating system. I did submit a pull request with the fix, so hopefully they review it soon.

Edit: Microsoft approved my pull request to fix this. Shouldn’t be an issue now πŸ™‚

Azure Gov B2C

While most people consume Azure commercial cloud, Azure gov is another beast. It seems the lack of documentation makes each project a bit more challenging.

I am currently doing a POC for using B2C in gov cloud. B2C supports local accounts which makes it great to put application end user accounts in their own tenant instead of creating my own identity provider. Typically, when you create the B2C tenant, you link it to a subscription. I did not have this option as I could only create just the tenant. It typically looks like this when all is working:

Create a new Azure AD B2C tenant selected in Azure portal

Mine looked like this:

I looked for the feature provider Microsoft.AzureActiveDirectory and it is missing all together. I popped a ticket to Microsoft and they said B2C is supported and you don’t need to link it to a subscription. I was a bit confused because a subscription is a billing boundary and if I used MFA or conditional access, how could it be billed? Well, you can’t do this. After pleading my css case, I was told that this is in preview and engineering knows about this. What stings a bit more is that the Azure feedback item has been open since 2017 😦

Either way, I kept moving forward to see what I can do with this POC. The first thing to call out is that the endpoints are not documented at the docs.microsoft.com site. You must use the endpoints button in your b2c registration you created. What I noticed is that instead of b2conline.com and your typical tenant tld, you need to add .us.

Once I had my endpoints configured correctly, my user flows were working just fine as accounts were being created in the tenant. Now, let me create local accounts in the portal in the b2c tenant. Nope, said could not create user. I am confused why my app could do it just fine. I found the api doc for create user and tested against my commercial sub and it worked, but when I tested against the gov graph endpoint, it failed. It said the property creationType was missing. Alright, so I added creationType=LocalAccount into the json body and the api call worked. Guess this is an azure.portal.us issue.

The last issue I found is that the tenant type is set to Preview tenant. I couldn’t find anything what this meant until I stumbled across the 2016 announcement post. Information about your tenant type is available in your B2C Admin UI. If it says β€œProduction-scale tenant”, you are good to go. If you have an existing β€œPreview tenant”, you must use it ONLY for development and testing. The lack of documentation of what is preview and not is hurting. This sounds like a red flag as I can’t deploy production apps into a preview tenant.

I popped a couple of tickets to Microsoft and will update this post once I get more information. More to come!

Edit: no date yet when it will GA in gov 😦