The cloud allows development and test teams to be very agile, as they can spin the resources they need in a matter of minutes, whether for quick prototyping, learning, or scalability tests. That can come with headaches if costs are left to spiral out of control.
We will investigate what reactive and proactive controls can be put in place so that cost overruns are prevented. We will also create a workflow to periodically notify development teams with the resources currently running, so that they can be proactive in shutting down unneeded resources.
Using Azure Cost Management
The first step is to review the excellent Cost management best practices from the Azure documentation.
You will certainly want to define budgets for all your subscriptions. This is very easy through the Azure portal. Under Cost Management, first review the Cost analysis and Advisor recommendations panes for useful insights. Then, under Budgets, define your budget targets, and set up notifications as certain thresholds are reached.
For more advanced scenarios, you can even automate actions such as tearing down resources if thresholds are exceeded.
Notifying users of running resources
For more proactive cost management, I like to be notified a few times a day with the list of costly resources currently running in my sandbox subscription. For that purpose, I have created a workflow to report on running resources. Workflows allow programming complex integration logic without writing any code, by using hundreds of prepackaged connectors and predefined actions. Feel free to use the following guidance as a baseline, and customize it for your needs.
You can create a Workflow in Microsoft Flow, but you will need a Premium subscription to follow the steps below. Otherwise you can create a Logic App in Azure.
Create a new workflow, and as Trigger type, select Recurrence to have the Workflow run on a schedule.
In the Recurrence Trigger, select the time interval. Create two actions of type Initialize Variable and create a one String variable to store the e-mail content as we build it up. We will see the use of the second one later.
Create a new action of type List resource groups, and enter your Subscription ID.
Create an Action of type Control – Apply to each. Select the value output of the List resource groups action. Within the loop, create an Action of type List resource by resource group. Select the Subscription and the Name output of the List resources by resource group action.
Still within the loop, create an Action of type Control – Apply to each. Select the value output of the List resource by resource group action. Within the inner loop, create an Action of type Compose. Enter a template containing the resource name. Optionally, you can even compose a link that will allow recipients to directly navigate to the resource, so that they can directly shut down a VM for example. (Replace microsoft.onmicrosoft.com with your tenant, that you will see in the URL when you navigate to a VM in the Azure portal).
Still within the inner loop, create an Action of type Control – Switch. Select the Type output of the List resource by resource group action. Here you can create Switch branches to handle every resource type you want to include in the report email.
For example, if you will be using HDInsight, create a branch for HDInsight. Select the value output of the List resource by resource group action. When you navigate to an HDInsight cluster in the Azure portal, notice that the URL is of the form .../resourceGroups/.../providers/Microsoft.HDInsight/clusters/... So we will use Microsoft.HDInsight/clusters as the type in the Switch branch. Create an Action of type Variables – Append to string variable and build up an HTML table row for our email, using the output of the previous Compose action.
Next, we will add a branch to report on active Virtual Machines. This requires slightly more logic, as we only want to report on VMs that are running (technically, that are not deallocated, since a VM could be in stopped mode but still allocated to a host and therefore incurring costs).
To get the VM state, we will issue a REST query to the Azure Resource Manager API. Create an Action of type HTTP with Azure AD. As Base Resource URL for the connector, enter https://management.azure.com. Build the URL as shown, using the Id from the Apply to each loop that iterates on every resource in the resource group.
At this point we could parse the JSON output of the API, but the response is quite complex, so that would require building some navigation logic. Easiest is to serialize the JSON to a string and check if it contains the state VM deallocated. Create an Action of type Set variable. Select the ResponseAsString variable and the Body from the HTTP with Azure AD action.
Create an Action of type Control – Condition. Select the ResponseAsString variable, does not contain, and enter the string VM deallocated. In the If yes branch, create an Action of type Variables – Append to string variable and build up a row in the email.
At the very end of the flow, create an action of type Office 365 Outlook – Send an email. Under Show advanced options, enable the Is HTML flag. Build an HTML table and include your variable containing the rows built in the previous actions.
You could nest that action within a condition action that checks if the Resource list is not empty, so that no email is sent if no resource is running.
You can extend the workflow by adding switch branches for other types of resource that incur significant cost, such as SQL Databases, Cosmos DB, and Azure Analysis Services.