I owe a huge favour to my friend Will Houston who had this problem and just before me and saved me a lot of head scratching as im sure I would never have worked this out!
The behaviour I was noticing was that I had setup a VM Scale Set and registered it as a self hosted build agent for Azure DevOps. It was working ok then I noticed it had stopped kicking off builds.
I hadnt really got around to troubleshooting this but I was chatting to Will who had told me about a problem he had which he had fixed earlier in the week and immediately I thought “oh ****” I bet that is the problem I am about to run into and sure enough it was.
The background is that when you setup a build agent Azure DevOps adds a tag to your Azure VM Scale set called __AzureDevOpsElasticPool.
The value of this tag matches the name of the agent pool you have configured in Azure DevOps.
Gotcha #1
The problem I had is that I re-ran my terraform which setup the network and build agent to do an update and the terraform didnt include the tag so terraform removed the tag from the scale set which meant that Azure DevOps stopped recognising it as the build agent because this tag was missing.
When you add the tag back with the name of the agent pool again it will start working again.
Gotcha #2
If you add the tag before you add the agent pool in DevOps then DevOps thinks the scale set is already registered with an agent pool so you cant add it.
This is a bit of a chicken and egg scenario but if you just build the scale set first and register is then when complete make sure you either add this tag to your terraform or ignore it then it should work.
Looking past the questionable approach of using this tag by Azure DevOps, if you fall into this trap then hopefully this will help you get out and save some time. Again, pints for Will for this!
