Containers in the cloud

Design by contract Tutorial, part 5/6: [Azure Devops] Mock your interfaces using Swagger, Wiremock, Docker, Azure Devops, Terraform and Azure

Put a little magic in your life: create the auto-deploy pipeline

We’re close to the end of our journey.

So far we’ve basically done everything we need. In this last step we’ll also make it happen automagically: we want to be able to do changes to our code (which, in our scenario, is the wiremock service configuration) and have them get deployed on Azure without us having to do anything.

We’ll use Azure Devops -formerly called Visual Studio Team System (VSTS) or, even earlier, Team Foundation Server (TFS) Online- for this. There are other services we could use as well, like Github or Bitbucket, and they’re equally good.

But whatever your service, in general this process is called CI/CD, short for Continuous Integration / Continuous Delivery. Simply put, CI means that your code is built and tested as soon as you push changes in source control. if the build or any test is not successful, the code changes are rolled back, guaranteeing (well, as far as your tests are concerned) that the code in the repo is correct. CD is the next step, taking the build and deploying it, usually in a test server, then in a staging one and then to production.

So as a first step, create a free account in Azure Devops. You can use the same Microsoft account you used in Azure or different. Once you’ve logged in, create a new project. Let’s call it GraubFinanceMockService.

By default we got a Git repository with the same name as the project. Let’s clone it in our development PC (i’m using C:\src\test, but feel free to use whatever you like).

Make sure you have git installed (or download it from here), then open a command prompt and type (replace the URL with your details):

cd c:\src\test
git clone https://dev.azure.com/YOURUSERNAME/GraubFinanceMockService/_git/GraubFinanceMockService

You’ll be asked for credentials of course (you might want to cache them). After that you’ll get a folder named GraubFinanceMockService. Move in there the folders we created during our previous steps: openapi, wiremock and devops.

Additionally, to avoid committing unwanted files in the repository, create an empty text file on the top folder named .gitignore, open it with a text editor and paste the following:

*.jar
lock.json
**/.terraform/*
*.plan
*.tfstate
*.info
*.backup

Now we’re ready to commit for the first time. Type the following in the command line:

cd c:\src\test\GraubFinanceMockService
git add .
git commit -m 'initial commit'
git push

And our code is there:

Now we’ll start setting up our build. “But wait”, you might reasonably ask, “we don’t really have any code to build, that’s no C# or Java or whatever project, why do we need a build?”.

Well, we do need to build our docker image, and push it in Docker Hub. This way when we change anything in our wiremock config, we’ll get a new image to reflect that.

But before we continue, remember that we have some variables in our tfvars files that we need to replace? Now it’s time to do that. Under Pipelines go to Library, then (+) Variable Group. Name the variable group azureconnectioncredentials, then add four variables (click the lock to set them as secret!):

subscription_id
tenant_id
client_id
client_secret

Be sure to check that “Allow access from all pipelines” is enabled.

But how do you get these values? From Azure CLI. The process is described by Microsoft here, but in short, open a command prompt (remember that from the previous step, we are logged in with Azure CLI already) and write:

az account show
# note the id, that's the subscription id, and the tenant id
az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/SUBSCRIPTIONID"

You’ll get something like the following, which you need to keep secret (the recommended way is to use a password manager):

{
  "appId": "XXXXXXXXXX",
  "displayName": "azure-cli-YYYYYY",
  "name": "http://azure-cli-2019-YYYYYY",
  "password": "ZZZZZZZZZZZZZZZZ",
  "tenant": "TTTTTTTTTTTT"
}

So paste these values to the respective variables in Azure Devops. You got the subscription id and tenant id from the first command (az account show). From the second (az ad sp create-for-rbac) get the appId and put it in the client_id variable, and get the password and put it in the client_secret variable. At the end, click Save.

You did set the variables to secret right? 🙂

We need one more variable group for the not-secret stuff. Create a new variable group, name it azurenames and add the following variables (here with sample values):

azurelocation = westeurope
basename = graubfinancemock
dockerimage = YOURUSERNAME/graubfinancemock
dockerimageversion = latest
envtype = test
SKUsize = B1
SKUtier = Basic

Also here we need “Allow access from all pipelines” to be enabled.

Now we’re ready to create a new pipeline. In Azure Devops go to Pipelines > Builds > New Pipeline. You can click “Use the classic editor” if you’re not comfortable with YAML, but here I’ll use Azure Repos Git (YAML) as I can copy paste the result here. Select your code repository and then, to see how it works step by step, Starter Pipeline.

Our new build will get the sources in a directory on the build server, but nothing more than that. Let’s start telling the build server what to do.

First, we need to tell it to use our variable groups. Delete whatever default code is there and paste the following:

name: WHATEVERWORKSFORYOU

trigger:
- master
pool:
  vmImage: 'ubuntu-latest'
variables:
- group: azureconnectioncredentials
- group: azurenames

We don’t really need distinct stages, we’ll just set up two jobs, build and deploy.

Now let’s get it to create the docker image.

jobs:
  - job: Build
    displayName: Build docker image
    steps:     

Now click on Show Assistant on the right, search for Docker and pick Docker (description: Build or push Docker images etc etc). Connect your container registry as follows:

In Container repository you enter the full name of the docker hub repository (YOURUSERNAME/graubfinancemock in our example) but even better, we can use our variable (same for the desired version). So enter $(dockerimage), then change to tags to:

$(dockerimageversion)
1.0.$(Build.BuildId)

Leave everything else to default values, click Add. Under steps you should have the following:

    - task: Docker@2
      enabled: false
      inputs:
        containerRegistry: 'dockerhub-graubfinancemock'
        repository: '$(dockerimage)'
        command: 'buildAndPush'
        Dockerfile: '**/Dockerfile'
        tags: |
          $(dockerimageversion)
          1.0.$(Build.BuildId)

Now click Save and Run. Et voila:

Having built our service, let’s deploy it. Paste the following at the end of the YAML file:

  - job: Deploy
    displayName: Deploy to Azure
    steps:

Now we need to run our cleanup script, then replace the variables in the tfvars files, then run terraform. Search for task Azure CLI, then configure the Azure subscription. Script type is Powershell Core, script location is Script Path, script path is $(Build.SourcesDirectory)/devops/cleanup.ps1 and script arguments is “-rgName ‘$(baseName)’” (without the double quotes, but note the single quotes). But remember, this is not on the root of our code repository. Click on Advanced and in working directory enter “$(Build.SourcesDirectory)/devops (without the double quotes). You should end up with the following:

      - task: AzureCLI@2
        inputs:
          azureSubscription: 'Free Trial (XXXXXXXX)'
          scriptType: 'pscore'
          scriptLocation: 'scriptPath'
          scriptPath: '$(Build.SourcesDirectory)/devops/cleanup.ps1'
          arguments: '-rgName ''$(baseName)'''
          workingDirectory: '$(Build.SourcesDirectory)/devops/'

Time to replace the variable values. Add another task named Replace Tokens. Change the target files to **/*.tfvars, uncheck the BOM (it creates problems somtimes). Done.

      - task: replacetokens@3
        inputs:
          targetFiles: '**/*.tfvars'
          encoding: 'auto'
          writeBOM: false
          actionOnMissing: 'warn'
          keepToken: false
          tokenPrefix: '#{'
          tokenSuffix: '}#'

Next up, terraform. We have the batch file ready, but we need terraform.exe to be available. So add a task named Terraform tool installer. Change the version to the latest (find it here, at the time of writing it’s 0.12.15).

      - task: TerraformInstaller@0
        inputs:
          terraformVersion: '0.12.15'

Everything’s ready to run our batch script. As we need Azure CLI to be available for terraform to work the way we want to, add another Azure CLI task . Pick the Azure subscription from the drop down (you don’t have to configure it again). Script type is Powershell Core, script location is Script Path, script path is $(Build.SourcesDirectory)/devops/terraformdeploy-pipeline.ps1 (it’s the one that uses the replaced .tfvars file). Click on Advanced and in working directory enter “$(Build.SourcesDirectory)/devops“(without the double quotes). At the end it should look like this:

      - task: AzureCLI@2
        inputs:
          azureSubscription: 'Free Trial (XXXXXXXX)'
          scriptType: 'pscore'
          scriptLocation: 'scriptPath'
          scriptPath: '$(Build.SourcesDirectory)/devops/terraformdeploy-pipeline.ps1'
          workingDirectory: '$(Build.SourcesDirectory)/devops'

We’re ready. The build definition, now complete, should be look this:

name: GraubFinanceMockServiceAutoDeploy

trigger:
- master
pool:
  vmImage: 'ubuntu-latest'
variables:
- group: azureconnectioncredentials
- group: azurenames

jobs:
  - job: Build
    displayName: Build docker image
    steps:     
    - task: Docker@2
      enabled: true
      inputs:
        containerRegistry: 'dockerhub-graubfinancemock'
        repository: '$(dockerimage)'
        command: 'buildAndPush'
        Dockerfile: '**/Dockerfile'
        tags: |
          $(dockerimageversion)
          1.0.$(Build.BuildId)
  - job: Deploy
    displayName: Deploy to Azure
    steps:
    - task: AzureCLI@2
      inputs:
        azureSubscription: 'Free Trial (XXXXXXX)'
        scriptType: 'pscore'
        scriptLocation: 'scriptPath'
        scriptPath: '$(Build.SourcesDirectory)/devops/cleanup.ps1'
        arguments: '-rgName ''$(baseName)'''
        workingDirectory: '$(Build.SourcesDirectory)/devops/'
    - task: replacetokens@3
      inputs:
        targetFiles: '**/*.tfvars'
        encoding: 'auto'
        writeBOM: false
        actionOnMissing: 'warn'
        keepToken: false
        tokenPrefix: '#{'
        tokenSuffix: '}#'
    - task: TerraformInstaller@0
      inputs:
        terraformVersion: '0.12.15'
    - task: AzureCLI@2
      inputs:
        azureSubscription: 'Free Trial (XXXXXXX)'
        scriptType: 'pscore'
        scriptLocation: 'scriptPath'
        scriptPath: '$(Build.SourcesDirectory)/devops/terraformdeploy-pipeline.ps1'
        workingDirectory: '$(Build.SourcesDirectory)/devops'

Did it work? Navigate your browser to https://graubfinancemock.azurewebsites.net/servicehealth and:

Ta da!

We’re basically done. Let’s see how helpful our service is to our developers.

2 thoughts on “Design by contract Tutorial, part 5/6: [Azure Devops] Mock your interfaces using Swagger, Wiremock, Docker, Azure Devops, Terraform and Azure”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s