Last night I attended the NEBytes user group in Newcastle. I was due to present my session about Azure Hybrid Connectivity. While I was there I met John Timney who is a UK based Sharepoint MVP and works as an Enterprise Architect. John presented the first session which was called Goal Alignment in Hybrid Sharepoint projects. I really enjoyed this session where John talked about the challenges of managing Goals from an EA perspective and how they can be related to the technology solution. There were a lot of bits which got me thinking about how this also applied to BizTalk projects.
Lets take a couple of examples. How many times do you walk into a new BizTalk project and hear that the aim of the project is to deliver against a strategic goal such as the below:
- I want to create an ESB
- I want to have all applications use the X system for retrieving product data
- I want to have a flow of data from system x to system y
- I want an API because we need to have one to keep up with our competitors
The problem with these goals is they are all expressing a solution or technical approach and are not aligned to anything which could be considered to deliver value to the organisation. John explained that his view and the research he has done suggests that good goals should deliver something aligned to objectives like the following:
- Maintain or increase profitable revenue to the business, now or in the future
- Maintain or reduce the operating costs of the business, now or in the future
- Maintain or reduce the amount of money tied up within the business, now or in the future
- Support or provide a solution to something constraining those things above
Thinking about the goals we might apply to integration projects in terms how how they relate to the above objectives means that we have added a measurable element which then means we can decide if something was successful or not.
This lead me to thinking what might be some good goals might look like that would relate to integration projects:
- I want to be able to allow my applications to work together effectively
- I want to exchange data with my partners in a secure yet manageable way
- I want to simplify the management of my data across applications
- I want to be able to rent applications rather than buy them
- I want to allow developers outside of my organisation to be able to build apps that leverage some of my assets
- I want to reduce the amount of infrastructure that I need to manage and maintain
Lets take a look at a couple of the examples above and look at how they might relate to their associated objectives.
I want to be able to allow my applications to work together effectively
If the customer is able to get their key applications working together in an effective way then there should be measurable improvements in the operating cost to the business to maintain their IT solutions. This should also have the benefit for future projects where there are reuse opportunities to use the same integration approach again which will give those future projects cost savings.
I want to exchange data with my partners in a secure yet manageable way
Being able to exchange data with partners effectively can have benefits both in terms of operating cost reduction but also in improving profitability. I have worked on a number of projects over the years where the ease of a partner being able to integrate with your company is a key differentiator in winning new B2B business. Data is incredibly important when dealing with business needs, that is why resources like Grouparoo are there to help with data architecture so a business can have the necessary storage that is used for their analytics.
By working with the customer to understand what it is that they actually want to achieve we can see that the goals that are expressed are now open aspirations which would clearly benefit their business in some way. This then gives the Enterprise Architect and Integration Specialists a lot of understanding around the business drivers which can help to produce the right integration strategy to deliver on these. It is then important that these goals and objectives are clear to everyone within the team so that everyone is clear what we are trying to achieve and it also gives people the opportunity to challenge the things that are happening on the projects. If the goal is to build an ESB then everyone would be heads down building this thing that was an ESB (and hopefully everyone understands that this is an approach plus some products rather than just a product, but thats a whole other story). Its unlikely we would get too many challenges to that, but if we knew the goals were the ones above we might have cases where someone in the team says “hang on this ESB that we are building might not be the best approach for this particular part of the project because our goals are to x + y + z and there are other ways we could achieve that which might have some benefits.
Even if the challenge was not correct at the very least everyone is clear on the big picture and we have stopped to check we are going in the right direction. This kind of constructive challenging should be encouraged but can not be done if everyone is unclear on the goals, objectives and strategy.
Bringing in the Architecture
With the goals we have identified above this now allows your architecture specialists to think about the right things to be putting in place to deliver projects which are aligned to a strategy which will achieve these goals. With these open goals an integration person should immediately be able to start seeing key words and phrases which point to architectural solutions and technologies which can help you deliver them. The architects can then find the patterns among these solutions to ensure that the right trade offs and decisions are made to deliver consistent solutions. Lets take a look at those goals and see what kind of things they suggest:
- I want to be able to allow my applications to work together effectively
To me this suggests the organisation may have a number of different types of applications and potentially a lot of them. Its going to be important to catalogue these applications and their capabilities and to come up with the preferred integration patterns for each application. We probably need to consider how we would measure that the applications would work effectively together and this could be things like “how easy is it to integrate with the app in a new project” or “do the users find a seemless experience when moving from one application to another”. In this case Id probably be looking to identify the key applications which we need to get a solid story with and the ones which would bring the most value to the business. Having success here can possibly help to get everyone on the side of seeing integration as a thing that supports positive application experiences rather than always being the pain point.
This is really about finding your preferred EAI patterns and then choosing the right tech for each one.
- I want to exchange data with my partners in a secure yet manageable way
In this goal we have 3 key elements B2B data exchange, security, managability. The first thought here is that there are probably some negative experiences in the organisation at present so its a good idea to find out what they are and to see if you can do anything about them or at least not repeat them. For the3 main elements you can potentially crack all three in one go if you have an industry standard approach available to you but if not you need to consider what capabilities your partners have and if you have the power to get them to do it your way or if you have to do it their way.
The current approaches many companies are starting to prefer for these type of integrations is an API based approach which can have lots of benefits such as giving your partners a rich experience and a model which can support very rapid improvements and new features. I guess one of the key architectural considerations here is to try and limit the number of approaches you will use. If you integrate with lots of different partners in lots of different ways then you will find it difficult to keep the costs of your solutions down.
- I want to simplify the management of my data across applications
This kind of goal is often about master data and system of record and synchronizing data across applications. This kind of goal can be one where the complexity of your solutions can get out of hand very easily and particularly in those larger organizations where you have customer data spread across many systems. Having proper task management solutions like Profit OKR Software can help in achieving the goal.
You need to be making choices around your approaches to bringing together this data and why you would be doing it. There are some organisations where the approach is to copy everything to a data warehouse and then applications would read from there and other companies where they use a specialist master data system which will merge and match records and other companies where they will mash together application services to provide a composite view of data across systems. In all of these cases you need to think about things like staleness of data and other factors.
If we considered a company doing this for the first time they might be looking at a big investment in an MDM solution which would be the source of the truth and the golden record. For one company this might be right but if you look at the objectives an architect at another company might also say that “we only have a limited number of cases where we would do this kind of thing so using something like BizTalk to aggregate some application services could let us create a composite record which would achieve the same thing without the huge investment in a new technology”.
- I want to be able to rent applications rather than buy them
A desire to rent applications should be a trigger for a Microsoft focused architect to see opportunities around Azure. The important thing here is that renting applications, platform or infrastructure is not always the best approach. Often it is a good one but its not the right approach all of the time. Maybe we should have qualified this goal with a challenge back to the business or IT senior management to understand why it is that they would like this goal. We would probably find that the answer relates to one of the objectives we defined earlier. The most likely ones are reducing cost and improving speed to deliver.
Understanding this extra detail might help us to define that the right approach is not always to rent and by assumption use the cloud. An example might be that we decide that having certain applications rented may actually mean the cost is higher when we consider test and development environments compared to just looking at production. An example might be comparing using BizTalk Server from MSDN for test environments versus having many BizTalk Services subscriptions supporting your test environments. The cost comparison is not as straightforward as just comparing the production equivalents. Ive seen this with other vendors too. This kind of thing may affect which applications or services you rent, or alternatively how you use those applications. Perhaps you have some environments on premise and some in the cloud.
- I want to allow developers outside of my organisation to be able to build apps that leverage some of my assets
As an integration architect this goal screams out API project. You can see why for some organisations this would be such an attractive opportunity. I can incentivise external people to make apps based on my company which hopefully make me look better to customers but save me having to build or manage the apps. This definitely relates to the increasing revenue objective with potential new business opportunities but from an architecture and technology perspective there are a number of key things to consider here such as: “How do i ensure these apps dont expose security vulnerabilities”, “how do I provide a good SLA to the apps but not affect my internal business” and a whole bunch of other considerations.
The good thing is that in the Microsoft space there are a lot of API related technical options available to you. These include: Azure API Management, BizTalk Hybrid Connections, Web API, Service Bus, BizTalk Services, Azure Web Role and Website hosting. All of these things can offer some capability to create or manage an API, provide the back end behind your API or allow you to reach down on premise to get to your internal applications.
- I want to reduce the amount of infrastructure that I need to manage and maintain
In this case we are probably quite close to the earlier goal about renting applications but also there are perhaps opportunities to look at things like throw away infrastructure when doing upgrades by using virtualization or cloud you can simply rip away a whole server and everything on it and deploy a new one in an automated fashion. Now that could certainly help to reduce some of your costs.
Summary
I mentioned how I found John’s session to be very thought provoking and the above is a bit of a brain dump of what was going through my head at the time and after the session. I guess what I am saying here is that in most BizTalk or integration projects we usually come in and see the solution that the customer wants. They want BizTalk, they want an API or what ever it might be. What id like to do is to reinforce Johns message which is equally valid for Sharepoint and BizTalk is that we need to take a step back and to really understand what it is that the customer wants to achieve. Once we know that we can use our skills as architects and technical people to find opportunities to choose the right technology and right approaches to deliver the right solutions for the customer.
This will then give us a much bigger of being successful and being able to measure that success against something rather than finding ourselves in a position where we have delivered an all singing all dancing ESB but find that the organisation still finds that integration still isnt really working for them and they still have the same old problems and then the natural reaction is that the technology is to blame for why things didnt work.
Id love to hear if this is something people think is relevant and if they have similar experiences. Also it would be great to hear if people have ideas on what could be common goals we see on integration projects. Maybe there is the opportunity to do a template of somekind that will fit the 80% of many customers scenarios
