As an integration architect you usually work with customers trying to build a platform which deals with all of the different types of integration they want to do and you try to find the right balance of technologies to build an effective and efficient platform which will have a good reputation within your customers business. This all sounds great and is done with the best of intentions but the problem is that stealth IT is a thing.
How many times do we find out there has been some application investment or a new business scenario going on for ages and no one ever thought to speak to the integration team then they rock up at the last minute saying how they desperately need a feed of data from A to B to be delivered in 3 days otherwise the integration team will cause this business critical project to fail and we will make IT look bad. Yes we have all been there many times.
The technical side of this problem is these types of solutions often involve the introduction or architectural and technical debt. Then when you consider they will be lashed together at speed and probably have a bunch of functional issues because there is no requirements. As an integration architect you have been evolving this platform that you are quite proud of because its implemented in line with good practices and then this project is going to make you implement a piece of shit solution on your platform which you just know is going to come back and bite you in the arse at some point.
Oh the challenges of an integration architect, lets call this a tactical integration project because it’s a reactive solution to a problem rather than a strategic integration solution which is one which is in line with IT strategy and is well thought out.
Well things are different now if you are in the world of Azure and iPaaS. There are a number of good things about Azure which can help you to implement this solution in a way that doesn’t involve strapping a nasty growth onto your strategic integration platform, and actually we now have a strategic approach for dealing with tactical integration.
Lets take a look at a specific example and see how we can solve this. In this example scenario the business has bought a new SaaS application and they need to feed the application information about staff structure which is in a very specific format and they have a very specific data source they want the information to be sent from (which isn’t the system of record for staff). There are a few issues which an architect wouldn’t like about this from day 1 but the tight deadline means we are really restricted in the options we have. Changing the architecture in such a short time isn’t feasible in this case so lets implement what the business has asked for but lets try to do it in a controlled manner.
The high level architecture looks something like below:
In the architecture the User Sync logic app is triggered on a scheduled basis and will call to the SQL database using the data gateway. It will loop the rows returned and call a sub Logic App to submit each record to the downstream SaaS application.
The User Sync Logic App looks like the below picture:
The load one user logic app looks like the following:
In the above logic app the actions are:
- Make an HTTP call to authenticate with the vendor api and get a token
- Make an HTTP call to workout if the user exists (http 404 if they don’t)
- If the user does exist make an HTTP PUT to update the user
- If the user doesn’t exist make an HTTP POST to create the user
What we did next is to ring fence this mini integration solution so that we know what it is, where it lives. We have 2 Azure subscriptions for our Azure assets for each environment. One is for our proper integration platform which is kept clear and well managed. The other we (probably inappropriately call the Integration Ghetto). Inside the ghetto subscription we put the tactical integration solutions we have been forced to build and which aren’t inline with our integration strategy. By ring fencing them we can control who has access and who can extend the solution or preferably ensue that no one extends it.
In addition to the ghetto subscription containing our technical debt we also structure the solution entirely inside its own resource group. This means we can easily restrict ACL’s to control access and when it comes to billing data we can easily identify this solution and workout exactly how much the business should be cross charged for this tactical solution rather having the costs hidden within the IT budget.
With Logic Apps we have this easy provisioning to setup your app and today you don’t even need to worry about a hosting plan which is a big win for these tactical solutions. When there was the app service plan for Logic Apps a number of releases ago you would have needed to be more conscious about the cost of the plan and if you were efficiently using it. Not today.
The final consideration for us was when it’s a tactical solution like this you often don’t have all of the environments you would typically have so you even have the option of developing the solution In production using the Azure portal directly.
Hopefully there is some food for thought here around how Logic Apps and Resource Groups can make it easy to implement those nasty tactical integration mini projects we get in a way that doesn’t spoil your strategic integration investments.
