In this article I want to talk about a real world example of a project which involved an API Gateway component and how even after some initial success the project deviated from the vision and where the architecture ended up and some of the problems associated with that.
The original requirements of this project were that there was to be a burst of development of websites to support different initiatives happening in the business. Most of these initiatives needed interaction in one form or another between the websites and the existing LOB applications. Some of these were custom developed and some were off the shelf. At the time all were on premise applications.
Original Architecture
The original architecture that had been proposed was like in the below diagram.
As you can see in this simplified diagram of the architecture, there was to be some web servers in the DMZ which host existing and new websites developed in Java. As we know, Java is one of the oldest programming languages and people who do not know how to compile and run java program would be at a loss when it came to these websites. While this diagram looks relatively simple, it is really looking at the conceptual view and it doesn’t show some of the complexity here. Some of the problems we identified with this architecture are:
- We create a tight coupling between the java websites and each of the LOB applications
- Each of the LOB applications exposes different protocols all of which would need to be used by the java website. EG: Web Service & SQL
- We would be punching more holes through the firewall than we would like
- The existing services are not necessarily easy for the java site to integrate with
- The existing applications could have many endpoints and operations all requiring different addressing and configuration to support there use
The problem here was that the solutioning to this point hadnt gone to a deep enough level to identify some of the problems and challenges that this project would have until it came into the technical architecture area. In our area we had foreseen projects like this in the pipeline and had prototyped our proposal for new ways to implement service based solutions already so we knew we had some tools available to help us make this project a lot easier for us to do which would also have a number of longer term benefits which could be reused on other projects.
Phase 1 Architecture
The below picture shows the first phase of our implementation of the new services based architecture. We knew that we had some existing apps which would be challenging to open up to a new way of doing services so in most places we implemented the Application Facade pattern which we used to create an adapter which had the Generic Service Endpoint and message handlers described in the background post and would allow the application to be “plugged in” to our new architecture.
We knew that some of the existing on premise applications were big and complex things but the Service Container concept we had made it easy for us to break out the Application Facade services into multiple containers and we just had to findout the balance of what was best placed in which container. We also introduced the API Gateway component. The gateway was intended to be:
- As simple and light weight as possible
- To implement routing to the appropriate service container to handle the message
- Make it easy for the intended API Gateway consumer(s) to work with the underlying sub-system
- Implement security patterns specifically aimed at the intended consumers
In the real world we knew that this API Gateway would end up having some logic in it, but it was difficult to predict up front what that might be. We expected non functional logic like caching might be required.
The architecture we had put in place to this point was not final by any means but it supported an underlying level of flexibility and agility which would allow us to be effective and deliver now, but to also make quite drastic changes in the future under the hood with minimal changes to the consumers of services.
Some of the key elemements in this architecture were:
- The Service Container made it easy for us to add new services in the architecture which use all of the same dev/test/deploy patterns minimizing the effort of introducing a new component to the architecture
- The Generic Endpoint made it so that the API Gateway only needed one endpoint per downstream service and not lots
- The API Gateway used a WCF client implementation which meant that it was the same code to make a service call regardless of which service you were calling
- We implemented a routing rules module which made it easy to change which message was sent where
- Although the API Gateway pointed directly to an Application Facade Service the level of coupling between them was not that high
At this point we had a bunch of initial success with this first project and things were going well.
Vision of Future Architecture
We had a vision for how projects in the future might look. This is represented by the below diagram.
With this vision we thought that the use of the Application Facade Services would continue and we would get more smaller application facade services as functionality was slowly added but also broken out in a way which would make it agile and flexible. By this point the deployment and operations maturity had been well established from the previous phase of the architecture should be well established and automated so adding new components was just more of the same.
We also expected that some Composite Services would have begun to be used. It wasnt clear up front what the road map was in enough detail to predict what composite services were likely so I hoped that the scrum teams would identify new stuff and we would have evaluated the options for a service to become a composite service in its own right.
I had also hoped that the initial success would encourage some investment in a service virtualizition capability. This would then replace our custom implementation for routing rules and allow these to be centralized in the service virtualization engine. There were a number of options back then for what this capability could have been implemented in from things such as Managed Services Engine to SOA Software and Sentinet. This services virtualization capability would have also improved our monitoring and management capabilities to a more mature level too. One benefit which would have helped the service virtualization tool if we had acquired one would be the canonical messaging format would have meant that we wouldnt need to build mapping capabilities for most services.
The final piece of this expected architecture was that we knew there would be other website projects down the line and I’d expected that we would have more API Gateway components in the architecture. In the API Gateway we could support the generic endpoint if a consumer could easily use that too, but more often than not the consuming application would need things to suit them. An example might be that the API Gateway would expose a traditional REST or traditional SOAP set of services but under the hood it uses the frameworks and approaches used by our architecture.
The key thing with the API Gateway was they were all light weight and relied on functionality which was provided under the hood. The gateway’s themselves would be more concerned with their consumers and would end up adapting functionality to suit that consumer group. They would also deal with the different context the application was in. One of the big differences here was that some of the websites were for anonymous users, some were for internal employees and some for business partners and some for members to manage their own data. One of the expectations I had was that these different types of consumers would be likely to need their own gateway to bend the functionality fron under the hood to suit that application. The user type was just one example but down the line we expected to have other similar scenarios.
Actual Later Architecture
As often happens on real world projects, over time people move to other areas, things are outsourced, other projects are delivered on the architecture and so on. When I came back to look how this project’s journey had evolved compared to the initial vision for it there were some big differences. The below picture shows the actual architecture after a period of time.
If we look at this you can notice a couple of key things:
- There are no composite services
- There is no service virtualization
- The API Gateway has many consumers
- The API Gateway has gotten very big
So we had gone from this space where we had begun building an architecture which was made up of small, discreet services which could work well together and not a million miles from what is the beginnings of a foundation to a microservices or SOA based architecture, to where we now had a monolithic API Gateway which was now called the “Integration Services Layer”. What had happened? The size and complexity and number of tightly coupled dependencies meant that we had lost a lot of the architectural agility which was a key element of what we had built.
Well what had happened was that the API Gateway had ended up getting lots of functional logic placed inside it. This grew and grew until this became one of the more complex pieces and one of the biggest components in the architecture where as it should be one of the smallest. Other projects had come along and thought “We have an Integration Services Layer, why do we want to create another component lets just use the one we have”. This meant that suddenly the API Gateway is consumed by many applications and each one has different security requirements which are all handled by the gateway and different ways that functionality needs to be bent to suit it each application. For example an operation about getting products for one application might be different to getting products for another so we end up with 2 versions of the same thing which do different things.
Features which really should have been pulled out into composite services were instead build in line buried within the API Gateway.
Without going into too much detail I think some of the main reasons that this hit problems were:
- The most experienced people moved onto other projects
- The agile process was too focused on getting features build and past UAT and not interested in the architecture
- After changes in the teams there was too much of a gap between development and architecture to maintain and lead the development in line with the original intentions
- We did not have time up front or during the initial projects to clearly document and outline how every thing was intended to work and be build so that down the line there was something to follow. Although im not convinced this would have mattered anyway after points 2 and 3 above
- We had too many people who werent that experienced in both architecture and development to keep this going but also to even see that there was a problem building
The thing is this kind of thing happens more often that you would like in the real world. At the end of the day the problems all boil down to experience and leadership and the way your development and architecture teams work together.
