When we deliver integration solutions on the Azure iPaaS platform we are usually building a composite application style approach where we have a solution that contains some Logic Apps, a Function and maybe some other bits that will deliver the functionality for a small group of interfaces.
This is a great model which provides a lot of flexibility for building solutions. Over the course of time you will build up a whole group of these solutions and your integration platform is closer to a microservice style architecture than it would have been in years past.
The challenge comes when we are working on big projects. If you are working on a big IT programme there may be hundreds of interfaces required between many systems to implement all of the integration requirements for the programme.
When we build micro-integrations which each have their own devops pipelines the problem we might have something like in the below picture.
Here we have our Azure DevOps project for your Integration Platform with 2 example repos. Each repo represents a group of interfaces being developed by a project within an IT Programme. Each interface has its own release pipeline.
I like this approach because you need to think about the future and while the projects/programme will deliver their requirements, the projects will end and the interfaces which are delivered will continue to live on and may be changed by future projects. Being able to change and deploy each interface in isolation with minimal dependencies on other things is an approach which will allow you to deliver integration in an agile way. When things are tightly coupled together and inflexible it becomes hard to change things without it impacting lots of other stuff.
The problem however comes when you are delivering lots of interfaces for your IT programme and you need to release new versions of many interfaces. Suddenly you have 50 pipelines to run to deploy all of the updates you need!
A while back I came across a brilliant Azure DevOps task which is one of my favourites. Its called Release Orchestrator. It is basically an Azure DevOps task which will run another Azure DevOps pipeline. You can find out more about it on the below link:
https://marketplace.visualstudio.com/items?itemName=dmitryserbin.release-orchestrator
If you imagine that I have created all of my individual interfaces, which each have their own pipeline. This lets me deploy every interface in isolation. I will then create a project Devops Pipeline that will only use the Release Orchestrator task and it will call all of the interface pipelines needed to deploy all of the interfaces involved in the project.
The below picture visualises what this looks like.
Below is a view of one of my pipelines. You can see that when this pipeline runs there are a list of tasks which use Release Orchestrator to call other pipelines. It will run the release of all of the components and interfaces required by the project.
With this approach I can also manage any dependencies if needed in the deployment order.
In big projects where you have many developers working on many interfaces at the same time, the project pipeline approach allows all of the interfaces to evolve in isolation, but you can still bring things together at the end for the project release.
In the future when new projects come along you can also add new project pipelines which may deliver new interfaces and also change stuff from interfaces delivered by a previous project. Below in the picture you can see that project 1 delivered interfaces 1-3 which each had their own DevOps pipeline and there was a project DevOps pipeline for project 1 using Release Orchestrator to run the pipelines for interfaces 1-3. Later project 2 comes along which adds interfaces 4-6 which each have their own DevOps pipeline. Project 2 is also making changes to interface 2 & 3 which were originally implemented by project 1. The project 2 Release pipeline using Release Orchestrator can run the interface pipelines for interfaces 2-6.
A closer look at the Project Release Pipeline
If we look closer at the project release pipeline, its just like any other multi-stage pipeline. In this case it has a stage for test and production.
The stage for test contains a list of steps like I showed earlier in the article. It is a list of steps which are all release orchestrator tasks which just call the other pipelines that you want to run.
Within the configuration for each step you need to point to the Release Definition for the pipeline you want to run and also the stage for the environment you want to deploy to as shown below.
Hopefully this shows how easy it is to create a project deployment pipeline to easily deploy multiple interfaces in one shot. I do think Release Orchestrator has been one of the most useful things we have setup in the DevOps process of our integration platform.
One of the things I am keen to understand is how we can use some equivelent to release orchestrator with YAML pipelines. Probably one of my next things to check out on this subject.
This is a cross post from integration playbook
