Volume I · Part II · Chapter 6
Built-In Tools
It is possible to open three perfectly useful tools and receive what look like contradictory instructions. A machine-sizing report recommends a smaller instance, a budget alert says the project is spending too much, and the autoscaler wants to create more instances. Before we schedule a meeting to determine which tool has failed, it would help to know what each one was asked.
The sizing report summarizes at least a week of one machine’s use, the alert compares charges that typically arrive within a day with a monthly threshold, and the autoscaler responds to the last few minutes of CPU. All three could be right. The difficult part is deciding whether today’s load is temporary, whether the proposed smaller machine can handle the work we have promised, and whether somebody has already agreed to pay for it.
I would start with the provider’s tools. Rebuilding an existing console in different colors is an expensive way for a capacity team to become busy, particularly when the organization already has people who know how to collect and interpret the evidence. There will be gaps worth filling, but the most consequential gap is usually the connection between a resource and the reason somebody needs it.
How to read this chapter
This chapter covers the provider tools a capacity team starts with — inventory, monitoring, billing, recommendations, alerts and caps, autoscaling and quota — and the meaning each tool attaches to its numbers.
- Keep inventory, use, and cost distinguishable — what exists, what it did, and what was charged.
- Collect measurements with their meaning attached — where a metric comes from and what it counts.
- Read a bill without multiplying it — billed and effective cost, snapshots, and joins that double-count.
- A recommendation is the beginning of an experiment — Advisor’s assumptions, stated so they can be tested.
- Alerts, caps, and the person who receives them — notices, the few controls that stop, and who acts.
- Autoscaling asks for capacity — the control loop, quota versus supply, and their different errors.
- Make the report end in an action — every report should end with a decision someone can own.
Keep enough detail to trace a recommendation back to the workload, measurement period, and agreement it assumes.
Keep inventory, use, and cost distinguishable
When I look at a service, I want to move among three views without losing track of what each tells me. Inventory shows what exists and where; telemetry shows what it did during an interval; billing shows what was charged under the commercial arrangement. A resource that disappeared yesterday may be absent from today’s inventory and still account for a substantial part of this month’s bill. A workload that stopped running may have left billed resources behind, while an apparently unused resource may still be needed for recovery.
The distinction seems obvious until someone joins the three views, drops the records that do not match, and presents the result as a complete account. We need the unmatched records precisely because they may explain what we have failed to understand.
Azure Resource Graph can list the VMs in every subscription under a management group with one read-only query:
az graph query \
--management-groups YOUR_MANAGEMENT_GROUP_ID \
--first 1000 --output json \
--graph-query "
Resources
| where type =~ 'microsoft.compute/virtualmachines'
| extend p = properties
| project id, subscriptionId, location, zones, tags,
vmSize = tostring(p.hardwareProfile.vmSize),
powerState = tostring(p.extended.instanceView.powerState.code)
| order by id asc" > vm-inventory.json
Like every command in this chapter, it was checked against Microsoft’s documentation in October 2026, not run against a live account.1 Three properties of the result matter before anyone compares it with a bill. Results are trimmed to what the caller may read and, in Microsoft’s words, arrive “without any indication that the result might be partial.” A request returns at most 1,000 rows, so when count is below total_records, repeat it with --skip-token set to the returned skip_token. And changes are indexed after a short delay, so keep the collection time with the file.
The powerState column earns its place. A VM shut down from inside its operating system reports PowerState/stopped: it keeps its hardware and its compute charge. A deallocated VM has given up both, though its disks still bill and its cores still count against vCPU quota.2
The query covers VMs only; scale sets, managed services, backups, and reservations need their own collection. A cheap completeness check is to compare the subscriptions in Resource Graph’s ResourceContainers table with those on the bill. A billed subscription the collector cannot see may have been cancelled, may sit outside its scope or permissions, or may be something nobody is watching.
The next connection is to a workload owner, and here a technically successful inventory can become an organizational puzzle. A tag cannot establish that the named group initiated the work or controls the decision we need made, and unresolved ownership deserves an exception queue rather than a convenient guess. For a shared platform serving many small applications, the platform owner may be the right accountable party, but still needs a way to explain consequential changes in demand; the cloud inventory cannot infer why a customer suddenly needs twice as much work.
Collect measurements with their meaning attached
Azure Monitor separates a virtual machine’s host, guest operating system, workloads, and application.3 Host platform metrics such as Percentage CPU and Available Memory Bytes arrive each minute with no configuration or ingestion charge and are kept for 93 days.4 Anything from inside the guest, such as the operating system’s own counters or a database’s statistics, needs Azure Monitor Agent and a data collection rule that somebody maintains.
Ask what “available” counts before a memory-sizing decision. Linux’s MemAvailable, for example, includes page cache the kernel can reclaim,5 so the cache that keeps a database’s reads off storage in Chapter 5’s example can look like memory nobody needs.
The Azure CLI can retrieve a VM’s Percentage CPU for an interval.6 Set CAPACITY_VM_ID to the VM’s full resource ID and put UTC timestamps such as 2026-09-01T00:00:00Z in CAPACITY_START and CAPACITY_END:
az monitor metrics list \
--resource "$CAPACITY_VM_ID" \
--namespace Microsoft.Compute/virtualMachines \
--metrics "Percentage CPU" \
--start-time "$CAPACITY_START" \
--end-time "$CAPACITY_END" \
--interval 1m --aggregation Average \
--output json > cpu-metrics.json
The result is one VM’s one-minute averages, not a fleet-wide utilization score. Missing points remain missing, and a minute’s average cannot show the peak within it. Keep the resource ID and interval with the file, and check the exit status and returned coverage before using it.
Coverage is where the interesting questions are. Which inventory objects have no observations? Have we measured a month-end job in a quiet month? Last year’s seasonal peak survives only if someone exported it before the 93-day window closed, and moving or renaming a resource may lose its history.7
I would much rather see a report say that CPU observations exist for a stated interval, memory evidence is missing, and no recovery test was run than receive a green efficiency score whose meaning I have to reconstruct. Put completed work and service quality beside the observations, and they become evidence for the workload investigation in Chapter 5.
Where a database or platform team already runs a better pipeline, I would rather work with them than pay to reproduce it, provided we can inspect each measurement’s definition, keep enough history and stable identifiers, and reproduce a disputed result.
Read a bill without multiplying it
Azure Cost Management can export actual, amortized, or FOCUS-formatted cost data to storage on a schedule.8 The FOCUS dataset, offered for Enterprise Agreement and Microsoft Customer Agreement scopes, puts billed and effective cost in one file, the columns Chapter 1’s cost record draws on. Microsoft labels it 1.2-preview while the specification has reached 1.4,9 so record the version before comparing providers.
In FOCUS, BilledCost is what was or will be invoiced, and EffectiveCost spreads commitment purchases over the usage they cover; BillingCurrency is the unit for both. ServiceName comes from a mapping Microsoft calls incomplete, so treat it as a label rather than an ownership record.10 Adding the two cost columns together would count one expense twice.
A constructed example shows how far apart they can be. A team pays $36,000 at the start of March for a one-year VM reservation; treat each month as a twelfth of the term. March’s export has a purchase row with a BilledCost of $36,000 and, as FOCUS requires, an EffectiveCost of zero. The covered usage, including unused hours, carries $3,000 of effective cost a month and no billed cost. March’s billed cost therefore exceeds its effective cost by $36,000 − $3,000 = $33,000, and the next eleven months reverse it: 11 × $3,000 = $33,000.11 Finance may need March’s cash figure, but a trend built on billed cost will show March as the year’s most expensive month even if the workload never changed.
Exports are also snapshots. Each run writes its partitions to its own folder with a manifest.json that lists them. If the export’s default overwrite setting is turned off, a daily month-to-date export leaves about thirty runs a month, each holding every day so far; concatenating them for a steady 30-day month counts 1 + 2 + … + 30 = 465 days of cost, 15.5 times what was spent. Use every partition of one completed run, as its manifest lists them, and keep the run’s identity with the result.12
Suppose an authorized owner has produced an uncompressed CSV export for the period and scope we need. Save the following as summarize_focus.py and pass that run’s files as arguments; it reads local files and makes no cloud calls:
import csv
import sys
from collections import defaultdict
from decimal import Decimal
if len(sys.argv) < 2:
raise SystemExit("Pass the CSV parts from one export run")
totals = defaultdict(lambda: [Decimal(0), Decimal(0)])
for filename in sys.argv[1:]:
with open(filename, newline="", encoding="utf-8-sig") as source:
for row in csv.DictReader(source):
key = (row["BillingCurrency"], row["ServiceName"],
row["ChargeCategory"])
for i, field in enumerate(("BilledCost", "EffectiveCost")):
amount = Decimal(row[field])
if not amount.is_finite():
raise ValueError(f"Invalid {field} in {filename}")
totals[key][i] += amount
print("Currency\tService\tCharge\tBilledCost\tEffectiveCost")
for key, (billed, effective) in sorted(totals.items()):
print(*key, billed, effective, sep="\t")
Grouping by ChargeCategory puts a purchase like March’s on its own line. The program keeps credits and rows without a resource identifier, and stops on a missing or malformed cost field rather than substituting zero. It is an aggregation, not an invoice reconciliation: keep adjustments, corrections to earlier periods, and scope differences visible when comparing with Finance’s close. The arithmetic can be correct while the files describe the wrong period.
Joins create the next duplicate. If one resource matches three rows in an ownership table, summing the result counts its cost three times. Establish the mapping’s effective dates and uniqueness, or define an allocation whose parts reconcile to the original amount; otherwise somebody’s convenient join becomes an accounting policy without anyone remembering when.
The opposite failure is quieter. Azure resource names are case-insensitive and APIs may return them in different case, so a literal string join can silently drop rows. Formats differ too: FOCUS writes SubAccountId as /subscriptions/ followed by the GUID that Resource Graph calls subscriptionId. Normalize both before joining.13 When this view meets the portfolio’s workload map and demand record, keep any unallocated amount in the result until we can explain it.
A recommendation is the beginning of an experiment
Azure Advisor offers a useful starting hypothesis for a sizing review, and Microsoft documents its method precisely enough to test.14 A resize recommendation looks at CPU, memory, and outbound network over seven days by default, configurable up to 90 days. Thirty-second samples are averaged to the minute, and each 30-minute period keeps its highest minute average, so a burst shorter than a minute survives only in diluted form. Advisor infers from the CPU pattern whether a workload is user-facing. If so, the cheaper size must keep 95th-percentile CPU and network at or below 40 percent and 99th-percentile memory at or below 60 percent; otherwise the limit is 80 percent for all three. Shutdown recommendations ignore memory.
Each of those choices is an assumption we can check. Did the lookback include the month-end close? Was the batch service classified as user-facing? Does the workload depend on something Advisor does not measure, such as the disk throughput limit that comes with each VM size? Is the business paying for speed of response, as it was with the Cassandra cluster in the Introduction at 1 to 2 percent CPU? Microsoft lists its own exceptions, including upcoming traffic, disaster recovery, testing, and fleets kept on one size. Savings are estimated at retail rates, ignoring reservations and savings plans, and Microsoft warns that moving to another VM series can raise cost when reservations cover the original one.
We can retrieve cost recommendations without changing any machines:15
az advisor recommendation list \
--subscription YOUR_SUBSCRIPTION_ID \
--resource-group "" \
--category Cost --output json > recommendations.json
The empty resource-group argument stops a configured default group from narrowing the request. Each recommendation carries a lastUpdated time and, where Advisor estimates one, a savingsAmount and savingsCurrency under extendedProperties.
Before I accepted or dismissed one, I would want to connect the recommendation and its evidence date to the workload’s next important event. Has the application changed? Is a migration about to end? Does the proposed smaller configuration affect a recovery obligation? I would also ask Finance what the reduction means under the commercial arrangement. Freeing capacity inside an existing commitment can be worthwhile even when it does not lower this month’s cash expenditure.
For a stable stateless service, a representative subset may give us a straightforward experiment: agree on completion and latency criteria, make the change through the operating team’s process, compare with the baseline, and preserve a feasible route back. A data service may need more patience, because ordinary traffic does not exercise the maintenance or recovery work that justified the allocation.
Suppose a smaller configuration serves requests successfully but cannot complete a required rebuild. We have discovered that the proposed saving changes the service being supplied. The business may be willing to accept that change, but I would not present it as an equivalent service for less money.
This is also why I want a useful explanation when a recommendation is deferred. “Awaiting a representative peak” gives us a next step, as does “retained for this recovery obligation” or “superseded by the planned migration.” “Engineering refused” mostly tells me that we have run out of patience with each other. The capacity team’s insistence deserves no more authority than Engineering’s refusal unless somebody can explain the work.
Alerts, caps, and the person who receives them
An Azure budget notifies people when actual or forecast cost crosses a threshold; in Microsoft’s words, “Resources aren’t affected, and your consumption isn’t stopped.” At subscription and resource-group scopes, an action group can pass the notification to automation such as a function or webhook, which needs its own design: which work may stop, who authorizes it, and how the service resumes.16
Cost data typically arrives 8 to 24 hours after use, budgets are evaluated every 24 hours, and email follows within about an hour, so an overrun can be nearly two days old when someone reads about it. An application that needs an immediate limit has to enforce it in its own request or admission path, reconcile it with billing, and decide what happens when the limit is reached.
Budgets evaluate actual cost without amortization and now include purchases, so March’s $36,000 reservation purchase would push a budget at the scope that receives the charge past its threshold, although the workload had not changed. Microsoft documents filters for budgets meant to track usage only.17
Hard stops exist in narrower forms than many people expect. Azure’s spending limit applies only to subscriptions that include credits, such as free accounts and Visual Studio subscriptions, not to pay-as-you-go or commitment plans. When the credit is spent, Azure stops and deallocates the VMs and leaves storage read-only; a subscription with credits over several months is re-enabled at the next billing period.18 AWS budget actions can apply an IAM or service control policy, or stop chosen EC2 and RDS instances; Google Cloud’s spend caps, in Preview as of October 2026, pause one eligible service in one project, not instantly, and persistent resources keep accruing charges.19 Before calling any of these a cap, ask which work it stops and which it leaves running.
An alert requires somebody to decide what to do; an automated control can change the service before that conversation happens. Before calling either a cost control, I would want to know who receives the signal, what they can authorize, and what happens to the customer. A development sandbox may be allowed to stop. A shared production service may need escalation and a controlled reduction of optional work, because stopping it would create a much more expensive problem. An AI experiment may sit inside a project boundary that permits intervention without touching unrelated customer activity. Either arrangement needs unglamorous upkeep: test the notification path, confirm the recipient still has the job, and make sure the operating team knows how to resume anything enforcement can stop.
For a platform serving many small applications, I would put the routine aggregate budget with someone able to manage the platform, with a route for exceptional demand. Sending budget alerts to everyone who uses an application builder would be what Chapter 3 calls visibility without agency.
Autoscaling asks for capacity
Chapter 2 argued that an autoscaler’s request is not a supply plan; Azure’s tools show where the gap opens. An Azure Monitor autoscale rule aggregates a metric such as CPU over a time window (ten minutes in Microsoft’s example), compares it with a threshold, changes the instance count by a set step between a minimum and a maximum, and then waits out a cooldown. The job runs every 30 to 60 seconds.20 Nothing in that loop knows whether the subscription may deploy more instances, whether Azure has hardware for them, or how long they will take to become useful.
Permission and supply are separate checks. vCPU quota is set per subscription and region, as a regional total and a limit for each VM family, and a deployment must fit both; capacity is whether Azure has hardware for that size in that region or zone when it allocates.21 Quota can be read in advance:
az vm list-usage --location eastus2 \
--subscription YOUR_SUBSCRIPTION_ID --output table
Supply is tested at allocation, and the two shortages produce different errors with different remedies. Exceeding quota returns OperationNotAllowed and calls for a quota increase. A supply shortage returns AllocationFailed or ZonalAllocationFailed and calls for a retry or another size, zone, or region.22 Starting a deallocated VM or resizing one is also an allocation, so a fleet deallocated every night to save money gives up its hardware nightly and has to win it back each morning.
Even an allocated resource has work ahead of it before it becomes useful. In the constructed portfolio, the eight borrowed batch workers need two days of preparation and testing, and the sixteen new workers are estimated at six weeks for delivery, preparation, and qualification together. Raising a maximum instance count changes neither.
For illustration, give five services permission to request 20 additional workers each. Their combined allowance is 5 × 20 = 100 workers, but suppose the compatible shared pool can supply only 40 at the relevant time. If all five reach their maximum together, we are 100 − 40 = 60 workers short, although each scaling policy was sensible by itself. Five scale sets in one subscription and region also share one vCPU quota, which can run out first. The missing answer concerns which demands take precedence, what supply can be arranged, or which commitment can change.
The shared-pool owner needs to be involved before several teams depend on the same unallocated resources. Holding capacity ready requires funding, and funding alone does not decide which customer receives it when promises compete. I want those decisions made explicitly rather than discovered through simultaneous scaling failures.
An on-demand capacity reservation holds supply directly, and creating one is an early supply test: Azure accepts it only when quota and capacity allow, and bills it from that moment, as Chapter 2 described.23 Volume II, Chapter 18 covers its use for recovery, and Volume II, Chapter 20 its difference from a Reserved VM Instance, which discounts the price without holding hardware.
When reviewing the control loop, I would therefore keep desired, allocated, and usable capacity visible alongside unmet demand. A rising desired count can explain what the autoscaler is trying to accomplish; we still need evidence that the service has acquired the ability to do it.
Make the report end in an action
For our web service, the tools ought to support a discussion its owners can finish. We should be able to see the inventory and qualified failure groups, the experiment establishing usable throughput, the historical baseline and proposed rollout, and the costs of the available options. From there, we can decide who will qualify the reassigned workers, how the batch delay will be handled, which launch scope is feasible, and what remains to be confirmed about new supply.
The companion’s examples/capacity_portfolio/run.py generates a synthetic version of that review without cloud credentials. It demonstrates how the evidence and decisions connect; adapting it to an actual portfolio still requires the collection and specialist validation we have discussed.
If the native tools and our additions are doing their job, we should spend less time assembling those facts and more time with the people who can explain why the workload is changing. That conversation becomes particularly valuable when the bill counts tokens while the customer thinks they have purchased useful work.
Notes and sources
Microsoft Learn, “Overview of Azure Resource Graph”, “Work with large data sets”, “Understand the query language” and “az graph”, checked October 4, 2026. Access-trimmed results, eventual consistency, the 1,000-record limit,
--skip-tokenpaging and theResourceContainerstable. The power-state property follows Microsoft’s VM sample queries.az graphcomes from theresource-graphCLI extension.↩︎Microsoft Learn, “States and billing status of Azure Virtual Machines” and “Check vCPU quotas”, checked October 4, 2026. Stopped VMs stay allocated and billed; deallocated VMs release their hardware and stop compute billing; quota counts cores “both allocated and deallocated.”↩︎
Microsoft Learn, “Monitor virtual machines with Azure Monitor”, checked October 4, 2026. The four layers, and Azure Monitor Agent with data collection rules for guest and workload data.↩︎
Microsoft Learn, “Metrics in Azure Monitor” and “Supported metrics - Microsoft.Compute/virtualMachines”, checked October 4, 2026. No configuration or ingestion cost, one-minute collection, 93-day retention, possible history loss on move or rename, and the VM metric names.↩︎
Linux kernel documentation, “The /proc Filesystem”, checked October 4, 2026.
MemAvailableis estimated from free memory, reclaimable slab, the file LRU lists and zone watermarks. Check any provider metric’s own definition before treating the two as equivalent.↩︎Microsoft Learn, “az monitor metrics”, checked October 4, 2026. Resource, namespace, metric, time-range, interval and aggregation arguments.↩︎
Microsoft Learn, “Metrics in Azure Monitor” and “Supported metrics - Microsoft.Compute/virtualMachines”, checked October 4, 2026. No configuration or ingestion cost, one-minute collection, 93-day retention, possible history loss on move or rename, and the VM metric names.↩︎
Microsoft Learn, “Tutorial - Create and manage Cost Management exports” and “Cost Management dataset schema index”, checked October 4, 2026. Datasets, FOCUS availability (EA and MCA at 1.2-preview; not MOSP), per-run manifests and folders, and overwrite enabled by default; without it, “30 different RunIDs for 30 days.” The 15.5-times figure is this chapter’s arithmetic.↩︎
FinOps Foundation, FOCUS Specification (version 1.4) and the version 1.2
EffectiveCostdefinition, checked October 4, 2026. A purchase covering future charges carries zero effective cost; the reservation is a constructed example.↩︎Microsoft Learn, “What is FOCUS?”, checked October 4, 2026. Billed and effective cost, billing currency,
ChargeCategory, and the incompleteServiceNamemapping. The script is an original teaching example, tested only on constructed files.↩︎FinOps Foundation, FOCUS Specification (version 1.4) and the version 1.2
EffectiveCostdefinition, checked October 4, 2026. A purchase covering future charges carries zero effective cost; the reservation is a constructed example.↩︎Microsoft Learn, “Tutorial - Create and manage Cost Management exports” and “Cost Management dataset schema index”, checked October 4, 2026. Datasets, FOCUS availability (EA and MCA at 1.2-preview; not MOSP), per-run manifests and folders, and overwrite enabled by default; without it, “30 different RunIDs for 30 days.” The 15.5-times figure is this chapter’s arithmetic.↩︎
Microsoft Learn, “Naming rules and restrictions for Azure resources”, checked October 4, 2026, on case-insensitive names and inconsistent returned casing; “What is FOCUS?” on
SubAccountIdand fully qualified*Idvalues.↩︎Microsoft Learn, “Optimize virtual machine (VM) or virtual machine scale set (VMSS) spend by resizing or shutting down underutilized instances” and “Virtual machine and disk performance”, checked October 4, 2026. Lookback, aggregation, user-facing targets, shutdown criteria, exceptions, retail-rate savings and the cross-series warning; VM size sets IOPS and throughput limits.↩︎
Microsoft Learn, “az advisor recommendation” and “Recommendations - List” (API version 2025-01-01), checked October 4, 2026. Read-only listing by subscription and category;
lastUpdatedandextendedPropertiesfields.↩︎Microsoft Learn, “Tutorial - Create and manage budgets”, checked October 4, 2026. Notification without stopping consumption; data and evaluation timing; actual cost including purchases, with usage-only filters; action groups at subscription and resource-group scopes.↩︎
Microsoft Learn, “Tutorial - Create and manage budgets”, checked October 4, 2026. Notification without stopping consumption; data and evaluation timing; actual cost including purchases, with usage-only filters; action groups at subscription and resource-group scopes.↩︎
Microsoft Learn, “Azure spending limit”, checked October 4, 2026. Eligible subscription types and the effect of reaching the limit.↩︎
AWS, “Configuring budget actions”, and Google Cloud, “Manage spend cap budgets” (Preview; updated September 30, 2026), checked October 4, 2026. Google bills overages as normal and requires a manual lift.↩︎
Microsoft Learn, “Understand autoscale settings in Azure Monitor”, checked October 4, 2026. Capacity limits, metric rules, cooldown and evaluation frequency.↩︎
Microsoft Learn, “Check vCPU quotas” and “Resource quota errors”, checked October 4, 2026. Regional and family quotas, separate quota and capacity checks,
az vm list-usage, and theOperationNotAllowedcore-quota error.↩︎Microsoft Learn, “Troubleshoot allocation failures when you create or resize VMs in Azure”, checked October 4, 2026. Capacity errors on create, start or resize, and their remedies.↩︎
Microsoft Learn, “On-demand capacity reservation in Azure”, checked October 4, 2026. Creation requires quota and capacity; unused capacity is billed at the VM rate; supported sizes and deployment types are limited.↩︎