How much waste is in your Azure environment

As part of the Azure Spring clean initiative I am writing this post to talk about waste in your cloud solutions and discuss what we can do about this.

https://www.azurespringclean.com/

What is waste in Azure

Cloud waste comes about from inefficient use of the cloud. We all know how awesome Azure is at helping us enable users by building great applications but as we transition from development to production and the application lives on through its lifecycle things will change and what might have been an efficient application when we first built it can overtime become inefficient. When we use the term inefficient in the context of cloud waste we are usually referring back to the costs associated with running and maintaining the application.

If we multiple up the waste within one application across the entire organization then even a small amount of waste in each area can add up to a significant amount of money being spent on something which does not provide you value in return.

In FinOps one of the most important concepts is called Cloud Unit Economics. This is about the difference between the cost and revenue/value associated with your cloud operations to determine if you break even or generate a profit from your use of cloud. One of the advantages of cloud is that billing date makes it easier to understand some of the costs to serve your application, but one of the key challenges to achieving profitability is that your cloud operations overtime will potentially generate a lot of waste.

What kinds of waste can I have

In Azure there are a number of different types of waste you can have. These include things like:

Resources which are no longer used but have not been deleted
Idle resources are sitting around doing nothing
Over provisioned resources which are capable of performing much more work than they are being allocated
Data & log storage which isnt providing value
Inefficiently used environments which you are paying for but are not using
Cost Commitments which are not being fulfilled
Opportunities to get lower prices through reservations

How can I measure how much waste I have

Measuring your wastage is not the easiest thing to do. There are so many types of waste that need to be considered differently.

A measurement of waste would really involve a comparison of the billing data with the resource usage metrics and then comparing them across all of your resources to identify the level of efficiency with which each resource is being used.

For many companies they find that Azure Cost Management is not sufficient to achieve the level of efficiency that they require and this is when they go out to 3rd parties like Turbo360 to get a tool that will help them.

In our integration team last year we had felt we were keeping on top of our costs quite well but when we did a deeper analysis of our costs we found there were still some areas we were missing waste.

Why do I have a problem with waste

When I speak to customers about their cloud cost management one of the most common responses I hear back is that the reason they have a problem with cost management and in particular with waste is because it is not directly within the responsibility of some ones job role to deal with waste.

At home we have a garbage collector who comes to take away our waste. You might have a heating engineer who comes to your house to look at reducing the heat loss from your house. Its the same in business we have job roles in manufacturing, HR, finance etc which are looking at business operations to be efficient and remove waste.

In Cloud we have the FinOps practitioner but their role is often more a governance, data and collaboration role. They advocate for cost efficient use of the cloud in your organization but be honest, in your organization who is your Cloud Garbage Collector?

Am I causing a problem for others with waste?

It is interesting to consider how my inefficient cloud usage can affect others outside of the organization. If we think of green software initiatives then every time I am inefficiently using the cloud then I am unnecessarily contributing the the carbon footprint of cloud operations. If we think that companies like Forbes and Accenture have estimated that cloud waste is approximately 30-40% of cloud costs then all of that waste is contributing unnecessarily to the carbon footprint.

A great way to help meet your green software goals is to reduce your cloud waste.

How do I identify waste

The challenges of looking to manage waste starts with the challenge of accessing billing data and then democratizing its access so your engineering teams can take ownership of managing their own waste. You can see below an example where I have empowered my integration team to have access to their own billing data so they can see how the solutions they build contribute to the overall spend on cloud.

When the team has access to the billing data they are the people best places to identify inefficiencies and waste within the solutions they build. If we can help them to sort through the large number of resources they have to help identify those key resources which might have inefficiencies then that will help a lot.

Below you can see some resources flagged up with potential rightsizing opportunities.

If we look at the metrics for that resource you can see the machine is doing very little work and is a potential candidate for rightsizing.

There are a number of ways you can identify these opportunities to deal with waste. As a minimum you should be evaluating the recommendations that Azure offers. Its surprising how many companies do not look at these. If you want to go further thou your 3rd party vendor probably offers additional features to help you identify waste like above.

What can I do about waste

In addition to whatever tool you use to help you identify waste you also need to look into your process about how you will deal with waste. This is really a people and process issue. As we said earlier in the article one of the key issues is the lack of responsibility in a job role for someone to get rid of waste. You should address this if you want to be successful. Your senior management (CFO, CTO) should be accountable for cloud costs and cloud waste. Without this accountability you will find it challenging to get people to make changes. Removing waste is not a sexy cool job that delivers new cool features for the business user so it doesnt get the love it deserves but cleaning up your waste can have a direct and significant impact on your bottom line so it should be viewed as important.

Going past the people aspects you need to begin changing culture. Bringing cost reviews into the process of design, build, test, deploy and not just leaving them until months after go live. Below you can see an example of where we look at the cost changes on specific resources in our cost reviews and we discuss the factors driving the change. Sometimes its a good change and it can be increased use of your application but sometimes its a bad change and your inefficient application that scales for performance but does not scale in a sustainable way.

Once you have the people lined up to address waste, the first step to starting to do something about waste is doing reviews of cost with a tool that will help you identify waste so you can focus your time on reviewing items flagged as inefficient.

The next step is to put a plan in place about how you will execute those changes to make your cloud more efficient. Sometimes you will have low hanging fruit such as the below example where there was a misconfigured database in UAT which could be changed to a lower sku to make a big yearly cost saving.

Sometimes the change will be more difficult and will involve more steps beyond just Azure. You will need to engage other stakeholders, do an impact assessment and plan changes. This may even mean you need to evaluate the cost of doing the change with the pay back period where the cost savings will break even. This is a good example of why thinking about cost as early as possible is a good idea. Sometimes the change that is needed post go live is going to cost a lot to do and isn’t worth it.

How can I measure if I am improving

To measure if your improving, this is where your teams using their democratized cost data can be challenged to demonstrate how they have reduced waste. As a practice you could have a quarterly meeting where teams can demonstrate their costs to the wider team and share with others details of changes they made and what the impact was. In the below example we can see where automation of the enabled/disabled state of Logic Apps was used to reduce the cost of Logic Apps resulting in a $12,000 per year cost saving.

Having your teams show their progress & share their success will give each other ideas on improvements they can make.

As a senior manager accountable for the waste/cost you could even incentivize teams by offering a team night out for the most interesting cost saving idea. This will help drive the culture change which is key to organizational efficiency.

What did we learn from our Integration Team

I mentioned earlier that we had been fairly on top of our integration team costs but when we did a deeper analysis of our Azure usage we found the following:

We could save $12000 per year by optimizing Logic App usage
We had approx $18000 per year of resources that were no longer being used
We had approx $10000 per year of reservations which we werent taking advantage of

I was quite surprised how much extra waste we found when we really looked into it. We were a pretty high performing team yet we still had a significant amount of waste. The most surprising one for me was the unused resources. I guess this is a side effect when you have multiple concurrent projects running in parallel. Things change interfaces get replaces and clear no gets missed. We had a gap in our garbage collection.

On reflection, I think we were running at approximately 8% of our integration Azure Integration platform spend is waste. This is considerably less than the average estimates for waste proposed by Accenture and similar organizations but it is still more than we wanted.

When we looked at the wider organization you can see there is a higher amount of waste that we would mine into looking for optimizations.

My challenge to the reader of this article is “How much waste do you have”?