Im now up to part 3 in the journey of our evolving Azure based Integration platform. In this phase we had an integration project which did not involve our SaaS application investment but instead involved more B2B integration but this time involving on premise applications.
The on premise part of the integration development was based on flat files and SFTP again so this meant that we did not need to go down the hybrid integration route just yet. We were able to use BizTalk Server which we already had as part of the core of our integration platform to be able to pull these files from the on premise SFTP server without all of the complexity of having to join these domains up. Once we had the data in BizTalk we needed to implement a process which would split the data then send it to multiple destinations where there would be a one or more return messages in the future at some point where we would need to aggregate the data back together and return a response file back to the SFTP server on premise.
Without getting too much into the detail these new requirements meant that we would need to compliment BizTalk with somewhere to store some state data to support the splitter and aggregator pattern so that it was effectively implemented. To do this entirely in BizTalk would potentially been possible but the solution would become very complicated and externalising this state would keep the solution a lot simpler. Its not uncommon to get requirements like these and the solution is often to add a custom SQL database to your BizTalk’s database server. That was an option we could choose here but hang on, we are using Microsoft Azure. We considered the options and decided that it would be a good approach to use the SQL Azure Database option instead. Conceptually its the same, there is a SQL database that is storing the data but instead we have chosen to off load a large part of the maintenance and management of the database infrastructure to Microsoft rather than having to worry about this ourself. This also meant that we had added a SQL Database capability to the Azure based integration platform which we would be able to use to add other custom databases if we required for any other projects. We can just turn the crank and scale up the amount of through put units we want to use if we need to use the SQL Database more. Perfect!
The next thing that came up later in the project was a request to create some email notifications for the business about certain parts of the process. The challenge here was that in our Azure platform we arent connected to the on premise network and we are not easily able to connect to exchange. This could have been a big blocker and in projects a few years back we would be waiting for the infrastructure team to sort a load of stuff out to enable this. In Azure the story is different. We have our cloud tool kit and in this case we could go into the Azure portal and to the add-ons section. In there we can add SendGrid to our subscription. This adds an email service which we can access over SMTP and literally 5 minutes after deciding we could try send grid I was able to configure our BizTalk send port to call SendGrid and bingo the emails were being sent. Now thats what the agility of the cloud is all about!
The next requirement we needed to deal with was to develop a dashboard for some business users to be able to manage and check some of this state data. We had BizTalk 360 but unfortunately this wasnt really the kind of requirement we could put in there. Instead we created a custom ASP.net application which could provide a dashboard to the SQL Database. For this custom dashboard we needed something to host it in and Azure Websites was a perfect choice. This gave us a really lightweight web hosting capability but we are able to securely talk to the Azure SQL Database and we were also able to connect the website to Windows Azure active Directory so that we can secure access to the portal.
The below diagram shows what the integration platform looks like now:
You can see that we have build upon our initial investment from the previous phase and used BizTalk Server again, but this time we have also added the following capabilities:
- Azure Website for custom ASP.net hosting
- SendGrid as an Azure add-on to send email
- Azure SQL Database to store some state
- Azure Active Directory to secure the website
We have made no significant capital expenditure in extending the platform, the bits we have added would increase the operational expense of our monthly Azure usage but taking advantage of the Enterprise Agreement prices we are getting some great prices here.
