Over the last few years I’ve worked with a number of different organisations and one of the interesting trends I have seen is around the core mindset to how that organisation does integration. Your first thought is probably “Mike isn’t this just their architecture” but I would argue that many companies have an immature integration architecture and way of working when it comes to integration. What I mean rather than their architecture is more like their first thought about how they solve an integration problem. For argument sake lets use the analogy of the hammer and nail. When they see an integration nail, which hammer are they going to instinctively grab.
With most organisations they tend to have one default hammer and that’s the way they tend to see all integration problems. The interesting thing within this trend is that different organisations have different hammers. Of the organisations I’ve worked with I tend to see the following approaches:
- Every integration problem is seen as needing a batch file based approach between systems
-
Every integration problem is seen as needing web services or API’s
While some of these approaches are considered more modern than others the common trait among these organisations is that they by default pick these hammers because that’s how they have always solved their integration problems. They have potentially even been quite successful in the past too.
One of the changing things in all of these types of organisations however is the application landscape. Each organisation has probably built up an approach based on the types of applications they build and buy over the years but with companies making significant investments in SaaS applications and also vendors releasing new versions of applications which often try to move customers on in the way they integrate then the organisations are tending to find their default hammers sometimes have limitations. Lets take an example to explore this further. If we have a customer who has a lot of on premise legacy applications which are 10-20 years old they are probably heavily reliant on big on premise database platforms, something like a big oracle or sql database. There is a fair change the applications are difficult to integrate with at any layer above the database so over the years they typically export and import data at database level using batch based processes. An example might be a list of orders are received from a partner as a batch file and its bulk loaded into the database then stored procedures process the data. The challenge for this organisation is when they invest in a new application and choose something like Dynamics CRM online which is a SaaS based product and doesn’t support the database level integration and doesn’t lend itself that well to batch based processes. We now have 1 application that likes to talk API and another application that likes to talk batch ETL. This organisation now has a challenge as it will potentially have to deliver an integration solution which works differently to what they are used to.
The point here is that in todays integration world there are many different types of integration scenarios that we come across. Even for organisations that are pessimistic to change you will struggle if you only try to use one approach to solving all of your problems so if you are investing in a way to do integration which will support your transition to a digital business you need to be open minded and prepared to have different approaches to delivering integration solutions. By doing this you will then have the best opportunity to leverage the assets you already have but to also bring in new assets too.
With this in mine, this is why I am a big fan of integration platforms based on the Microsoft technology stack. I have loads of different ways I can solve problems, I can match my solutions to the customer based on the following:
- The project requirements
- The skills they have
- The budget we have available
- The scale of processing we require
- The constraints of the systems we will integrate with
The platform will then allow me to have the agility to change around things without too many problems throughout the lifecycle of our integration platform as the applications and projects bring in new requirements.
Once your platform starts becoming mature then you can begin defining solution blueprints based on the common patterns. This is now your new set of tools. You have replaced your single hammer with your shiny new set of tools. These blueprints are common approaches to solving the common integration problems you find in your organisation. Hopefully by this point your now starting to see a culture change in the organisation where you have gone from the point where someone mentions the word “interface” and immediately most people in the room are picturing a batch file in CSV format being passed between 2 apps, to a position where being to ask about requirements and non-functional requirements and then begin to overlay these on your list of solution blueprints to find the right match.
Now one thing to note is that I may have used a batch based approach in my example, but I do see exactly the same issue in organisations who by default see the other options as the way they will solve a problem. Just as many companies who are API-obsessed are convinced that all integration is an API and they daisy chain API’s together and create brittle and coupled interface chains of dependency between applications. They often don’t see the benefits of asynchronous processing with queues which can be really powerful when combines with API’s.
Anyway a little brain dump from me there this evening, but I would be interested to hear if other people also find these common trends in organisations about how they approach integration and also if we are seeing a more open approach in the organisations that tend to be winning with integration.
