Tag Archives: dotnet

Design by contract Tutorial, part 1/6: [OpenAPI] Mock your interfaces using Swagger, Wiremock, Docker, Azure Devops, Terraform and Azure

Think before you speak write code: Define the specs and create the Swagger file

The first question we need to answer before we start development is always one and the same:

What problem am I trying to solve?

For our scenario, let’s say we are GraubLaser, a factory in GraubĂźnden, Switzerland that makes expensive laser engraving machines. Why? 1) because lasers are cool and also 2) because at some point I was interviewed by this company – GraubLaser is not its real name- and while I decided not to join them as it’s around 80km from where I live, the people were super nice and the machines really cool.

Let’s imagine that we are extending our invoicing system with the ability to:

  1. Ask our financial data provider, GraubFinance, if a customer is creditworthy, so that we allow them to order an expensive item with less risk of not getting paid.
  2. Tell GraubFinance if a customer is creditworthy or not, following a transaction we had with them.

In REST terms one way (there are many!) to implement this would be for GraubFinance to create a service on their system for us to call. The spec sheet could look like this:

Service URLs:
http://api-test.graubfinance.ch/api/1.0/CustomerTrust
http://api.graubfinance.ch/api/1.0/CustomerTrust

GET : Reads info about a certain customer

Parameters (in path):
- taxid : the customer's tax id, string, e.g. CHE-123.456.789

Returns (json in message body):
- A CustomerTrustInfo object with the following properties:
  - name : the company name, string, e.g. GlarusAdvertising AG
  - taxid : the customer's tax id, string, e.g. CHE-123.456.789 
  - trustlevel : the creditworthiness level, enum as string, e.g. OK
    - valid values : 
      - OK (no problems reported) 
      - WARN (minor problems reported, e.g. delays in payments but not arrears)
      - BAD (arrears reported)

POST: Sends info about a certain customer following a transaction we had with them

Parameters (in path):
- taxid : the customer's tax id, string, e.g. CHE-123.456.789
Content (json in message body):
- A CustomerTrustReport object
  - reportid : a unique id from our reporting system, string, e.g. 2dcc02d9-6402-4ce9-bf44-3d2cbe8bcd5e
  - reporttaxid : our own tax id, string, e.g. CHE-123.456.789 (indicating which of our subsidiaries had the transaction)
  - taxid : the customer's tax id, string, e.g. CHE-123.456.789 
  - trustlevel : the creditworthiness level, enum as string, e.g. OK
    - valid values : 
      - OK (no problems encountered) 
      - WARN (minor problems encountered, e.g. delays in payments but not arrears)
      - BAD (arrears encountered)
Returns (json in message body):
- A ReportStatus object
  - reportid : the id from the CustomerTrustReport
  - status : indicates if the report could be processed and saved, , enum as string, e.g. OK
    - valid values : 
      - OK (no problems encountered) 
      - ERROR (an exception occured)
- details : (only in case o error) gives problem details, string, e.g. Error: InsufficientDiskSpace exception

Authentication: 
Standard OAuth 2.0 bearer JWT token required which must contain the scope "CustomerTrust". If the token is missing or invalid, the service must return HTTP 401. If the token is valid but does not contain the correct scope, the service must return HTTP 403.

So basically the GET operation would give us info about a certain customer, identified by a tax id. The POST would give the provider info about our experience with them. As authentication is a big subject by itself, we won’t talk a lot about it here.

So the Swagger file could look like this (notice that for convenience we added the mock and the local dev machine URLs):

openapi: "3.0.2"
info:
  title: GraubFinance Customer Trust service
  version: "1.0"
servers:
  - url: http://localhost:8888/api/1.0/
    description: local dev machine
  - url: https://graubfinancemock.azurewebsites.net/api/1.0/
    description: mocking service
  - url: http://api-test.graubfinance.ch/api/1.0/    
    description: staging
  - url: http://api.graubfinance.ch/api/1.0/  
    description: production
paths:
  /CustomerTrust/{taxid}:
    get:
      operationId: GetCustomerTrustInfo
      summary: Reads info about a certain customer
      parameters:
        - name: taxid
          in: path
          description: Customer's tax id
          required: true
          schema:
            type: string
          example: "CHE-123.456.789"
      responses:
        '200':
          description: CustomerTrustInfo
          content:
            application/json:    
              schema:
                type: object
                properties:
                  name:
                    type: string
                    example: "GlarusAdvertising AG"
                  taxid:
                    type: string
                    example: "CHE-123.456.789"
                  trustlevel:
                    type: string                
                    enum: [OK, WARN, BAD]
                    example: "OK"
        '401':
          description: Unauthorized, JWT token not present or invalid
        '403':
          description: JWT token valid but does not contain necessary scope 
        '404':
          description: Customer tax id not found
    post:
      operationId: PostCustomerTrustReport
      summary: Sends info about a certain customer following a financial transaction 
      parameters:
        - name: taxid
          in: path
          description: Customer's tax id
          required: true
          schema:
            type: string
          example: "CHE-123.456.789"   
      requestBody:
        content:
          application/json:
            schema:
              type: object
              properties:
                reportid:
                  type: string
                  example: "2dcc02d9-6402-4ce9-bf44-3d2cbe8bcd5e"
                reporttaxid:
                  type: string
                  example: "CHE-123.456.789"
                taxid:
                  type: string
                  example: "CHE-123.456.789"
                trustlevel:
                  type: string                
                  enum: [OK, WARN, BAD]
                  example: "OK"
      responses:
        '200':
          description: Success
        '401':
          description: Unauthorized, JWT token not present or invalid
        '403':
          description: JWT token valid but does not contain necessary scope 
        '404':
          description: Customer tax id not found                
components:
  securitySchemes:
    OAuth2:          
      type: http
      scheme: bearer
      bearerFormat: JWT            
security: 
  - OAuth2: [CustomerTrust]

You could certainly improve the API spec -I would- but I don’t want to focus on this.

We should keep this somewhere. Create a directory for our project, let’s call it GraubFinanceMockService. Inside this create another one called, say, openapi, and save the swagger file inside as CustomerTrust.yaml. We’ll use the folder structure in the next steps and ultimately add it to source control.

The real purpose of this tutorial begins now. So let’s create the fake service!

Design by contract Tutorial, part 2/6: [Wiremock] Mock your interfaces using Swagger, Wiremock, Docker, Azure Devops, Terraform and Azure

When the blue pill is good enough: create the mock service

Wiremock is an HTTP server that is designed to be easily configurable so that it acts as a mock server. I love it. In fact I started writing my own (Lefkogeia) and stopped when I found Wiremock since it was so obviously exactly what I needed.

It’s very easy to run standalone (they also have also a not-free cloud service, well worth its money). Follow these steps:

  1. Install Java (the JRE is enough) if you don’t have it already
  2. Inside our base directory GraubFinanceMockService, create an empty folder, say wiremock.
  3. Download the jar from this page using the link “Once you have downloaded the standalone JAR“. Save it in the wiremock folder.
  4. With a text editor, create a file named start-wiremock.cmd (or whatever script extension your OS has) and paste the following (assuming java is somewhere in the path):
java -jar wiremock-standalone-2.25.1.jar --global-response-templating --port 8888

Check the jar file name to match whatever you downloaded. You can also change the port to whatever you like.

To test that everything works, run the script and then open a browser and navigate to http://localhost:8888 . In the command window you should have the Wiremock logo and a few details, and in the browser you should get “HTTP ERROR 403 Problem accessing /__files/. Reason: Forbidden”. If so, it all works correctly. If not, well, check the error message in the command window. To stop the service just close the window.

Now let’s configure it.

In the wiremock folder, create two subfolders “mappings” and “__files” (if you ran it, they will have been created already). In the mappings folder you will put the service definitions and in the __files folder any necessary attachments like image, xml or json files.

So create a text file in mappings, name it “base.json” and paste the following:

{
    "request": {
        "method": "GET",
        "url": "/servicehealth"
    },
    "response": {
        "status": 200,
        "body": "Service is up and running!",
        "headers": {
            "Content-Type": "text/plain"
        }
    }
}

Now run the script and navigate your browser to http://localhost:8888/servicehealth . That’s what you should see:

Now that we got the basics working, let’s create the mock service. We’ll create a new json file in the mappings folder -you can have as many as you want, they’re combined. Let’s call the new file CustomerTrust.json.

Remember we need to respond to GET and POST for the url http://HOSTNAME/api/1.0/CustomerTrust/ID right? A first try for the GET could be like this:

{
    "request": {
        "method": "GET",
		"urlPattern": "/api/1.0/CustomerTrust/([a-zA-Z0-9-.]{1,})"
    },
    "response": {
        "status": 200,
        "bodyFileName": "CustomerTrustInfo.json"
    }
}

Notice that with urlPattern, the accepted path is now a regular expression –which gives you a lot of flexibility.

In the __files folder, create a text file named CustomerTrustInfo.json with the following content (remember we just convert stuff from our swagger file):

{
  "name": "GlarusAdvertising AG",
  "taxid": "CHE-123.456.789",
  "trustlevel": "OK"
}

Run the script again and navigate your browser to http://localhost:8888/api/1.0/CustomerTrust/123 . You should get the contents of CustomerTrustInfo.json.

Now, obviously, the tax id we gave in the url (123) doesn’t match the taxid in the json (CHE-123.456.789). But Wiremock can take data from the request and use it in the response: it’s called response templating. Remember the –global-response-templating in the script? It enables exactly this behavior.

So to get the tax id from the request path, change CustomerTrustInfo.json as follows:

{
  "name": "GlarusAdvertising AG",
  "taxid": "{{request.requestLine.pathSegments.[3]}}",
  "trustlevel": "OK"
}

The double brackets {{ … }} tell wiremock that this part should be substituted. The expression request.requestLine.pathSegments.[3] tells it to get the 4rd part (it counts from zero) of the url’s path. The path part of url is /api/1.0/CustomerTrust/(taxid) so the tax id is in the 4th place.

So start the script and navigate to http://localhost:8888/api/1.0/CustomerTrust/CHE-12345. You should get:

{
  "name": "GlarusAdvertising AG",
  "taxid": "CHE-12345",
  "trustlevel": "OK"
}

Ok, but what about authentication? As OAuth 2.0 dictates, our request has to have a valid JWT token in the header named “Authorization” with content “Bearer (token)”. E,g,

Authorization: Bearer 0b79bab50daca910b000d4f1a2b675d604257e42

Now for our example let’s keep it relatively simple. We won’t validate anything except that the header is there. If not, we’ll return HTTP 401. Let’s add a 404 “not found” for good measure as well. So change the CustomerTrustInfo.json as follows:

{
	"mappings": [	
		{
			"priority": 10,
			"request": {
				"method": "GET",
				"urlPattern": "/api/1.0/CustomerTrust/([a-zA-Z0-9-.]{1,100})",
				"headers": {
					"Authorization": {
						"contains": "Bearer"
					}
				}
			},
			"response": {
				"status": 200,
				"bodyFileName": "CustomerTrustInfo.json"
			}
		},
		{
			"priority": 90,
			"request": {
				"method": "ANY",
				"urlPattern": "/api(.*)",
				"headers": {
					"Authorization": {
						"contains": "Bearer"
					}
				}
			},
			"response": {
				"status": 404,
				"body": "Server path {{request.path}} not found",
				"headers": {
					"Content-Type": "text/plain"
				}					
			}
		},			
		{
			"priority": 99,
			"request": {
				"method": "ANY",
				"urlPattern": "/api(.*)"
			},
			"response": {
				"status": 401,
				"body": "401 Unauthorized",
				"headers": {
					"Content-Type": "text/plain"
				}				
			}
		}
	]
}

Notice the priorities. If the request matches the one in the first priority 10 (with the correct path and the Authorization header) the json is returned. If the request matches the one in priority 90 (under/api with Authorization header), 404 is returned. If the request matches priority 99 (under /api but without the Authorization header), the 401 is returned.

This done, let’s also create the POST part. This could be like that (we’ll put in the mappings section with the rest):

		{
			"priority": 20,
			"request": {
				"method": "POST",
				"urlPattern": "/api/1.0/CustomerTrust/([a-zA-Z0-9-.]{1,})",
				"headers": {
					"Authorization": {
						"contains": "Bearer"
					}
				}
			},
			"response": {
				"status": 200,
				"bodyFileName": "ReportStatus.json"
			}
		}

In the __files folder, create a text file named ReportStatus.json with the following content:

{
  "reportid": "{{jsonPath request.body '$.reportid'}}",
  "status": "OK"
}

Let’s test it. As this is a POST, we can’t test it with a browser. We’ll need curl for that, so install it if don’t have it already. In the wiremock folder, create a file named CustomerTrustReport.json, which we’ll use to test the service:

{
    "reportid": "edbf8395-ac62-4c8c-95c3-d59a488dee7e",
    "reporttaxid": "CHE-123.456.789",
    "taxid": "CHE-555.456.789",
    "trustlevel": "WARN"
}

Then in a command window enter:

curl -X POST "http://localhost:8888/api/1.0/CustomerTrust/CHE-123.456.789" -H  "accept: */*" -H "Authorization: Bearer 1234" -d @CustomerTrustReport.json 

You should get the following:

{
    "reportid": "edbf8395-ac62-4c8c-95c3-d59a488dee7e",
    "status": "OK"
}

Of course, if you omit the Authorization header in curl (the -H “Authorization: Bearer 1234” part) you’ll get the 401 error.

So we have a working mock service! That’s how the complete mapping file CustomerTrust.json looks like:

{
	"mappings": [	
		{
			"priority": 10,
			"request": {
				"method": "GET",
				"urlPattern": "/api/1.0/CustomerTrust/([a-zA-Z0-9-.]{1,})",
				"headers": {
					"Authorization": {
						"contains": "Bearer"
					}
				}
			},
			"response": {
				"status": 200,
				"bodyFileName": "CustomerTrustInfo.json"
			}
		},
		{
			"priority": 20,
			"request": {
				"method": "POST",
				"urlPattern": "/api/1.0/CustomerTrust/([a-zA-Z0-9-.]{1,100})",
				"headers": {
					"Authorization": {
						"contains": "Bearer"
					}
				}
			},
			"response": {
				"status": 200,
				"bodyFileName": "ReportStatus.json"
			}
		},		
		{
			"priority": 90,
			"request": {
				"method": "ANY",
				"urlPattern": "/api(.*)",
				"headers": {
					"Authorization": {
						"contains": "Bearer"
					}
				}
			},
			"response": {
				"status": 404,
				"body": "Server path {{request.path}} not found",
				"headers": {
					"Content-Type": "text/plain"
				}					
			}
		},			
		{
			"priority": 99,
			"request": {
				"method": "ANY",
				"urlPattern": "/api(.*)"
			},
			"response": {
				"status": 401,
				"body": "401 Unauthorized",
				"headers": {
					"Content-Type": "text/plain"
				}				
			}
		}
	]
}

Of course we can -and in reality should- add a lot more. Especially useful would be to verify that the json provided by the POST has the correct attributes in place. One way to do that would be to check the json via further request matching on the POST, and then have a second POST section with lower priority (say, 50) that does not verify the json. This second POST then should return HTTP 400 (Bad Request) and a message like “Invalid JSON”.

But let’s stick with this for now (this post is arguably too long already) and start working on deploying the mock service. It’s docker time!

Design by contract Tutorial: Mock your interfaces using Swagger, Wiremock, Docker, Azure Devops, Terraform and Azure

Fake it till you make it: Introduction

As with many topics, this one came up due to a real need.

You see, at work we have in-house as well as outsourced development. The outsourcers’ locations literally span continents as well as time zones. And yes, the software each one develops must play nice with everyone else’s.

So in our situation, design-by-contract is imperative. It simply doesn’t work any other way. We need to let each one know what is expected of them and then give them relative freedom to develop using their tools and methods -as long as the input and output is defined.

Since we mostly use REST services, the way we usually do it is by giving them Swagger files. This does a decent job of explaining what they need to build, be it either the provider (a REST API) or the consumer (the caller of the REST API). But still, there are many cases where there are gaps and they need to test with “the real thing”, or at least something that’s close to being real.

So what I usually do is build and deploy a mock -a fake service that looks a lot like the real thing. That enables all of us to get most of the work done; any differences that arise during the integration testing will be (usually!) minor and (usually!) easily fixable.

My work (and this guide, walk-through, tutorial, nameitwhatyouwant) has the following steps:

  1. Write the Swagger, or OpenAPI, file. This is versioned; whenever there are changes, which is normal in every project, I issue a new version and notify everyone involved. The swagger files are kept in a git repository (we use Azure Devops, a.k.a. VSTS) so the changes are traceable.
  2. Create a mock (fake) service using Wiremock.
  3. Create a docker image for the mock service using Rodolphe Chaigneau’s wiremock-docker image.
  4. Using Terraform, I build an Azure resource group and host the docker image in an app service.
  5. In Azure Devops, I build a deployment pipeline that deploys all changes, be it in the Docker container or the Azure configuration, whenever a change is pushed in the git repository.
  6. Then everyone involved can test the service using the swagger editor, curl or whatever tool they like -SoapUI, Postman, Postwoman, younameit.

I could, of course, not bother with most of it and just run Wiremock locally. But remember, it’s not just for me. It has to be useful for many people, most of them outside my company’s network. They will use it to test the client they’re developing or verify the service they’re building.

Note that all of the steps explained in this guide are cross-platform. I’ve tested them with both Windows 10 and Ubuntu 19.04. In both OSes I’ve used just the simplest tools -a text editor and git command line- but normally I use VS Code (and occasionally vi for old time’s sake 😊). Whatever little scripting there is, it’s done in Powershell Core, also cross-platform.

Let’s start, shall we?

Step 1 – [OpenAPI] Define the specs and create the Swagger file

Step 2 – [Wiremock] Create the mock service

Step 3 – [Docker] Build the container for the mock service

Step 4 – [Terraform] Ship these containers

Step 5 – [Azure Devops] Create the auto-deploy pipeline

Step 6 – [Swagger] Test via Swagger UI and curl