Best practices for monitoring and optimizing storage costs in multi-subscription Azure environments

Cloud Platforms Microsoft Azure
, , , , , , , ,
1

Our organization has experienced significant storage cost overruns across multiple Azure subscriptions, and we’re struggling to get visibility into what’s driving the costs. We have about 30 subscriptions managed by different teams, each with numerous storage accounts storing everything from application data to backups to analytics datasets. Last quarter, storage costs exceeded budget by 40%, but when we tried to investigate, we couldn’t easily identify which teams, projects, or data types were responsible. The Azure Cost Management portal shows total costs but doesn’t provide the granularity we need to take action. We’ve started implementing tagging policies, but many existing resources aren’t tagged consistently. Looking for practical strategies and tools that have worked for others in similar multi-subscription environments. How do you monitor storage costs effectively? What tagging strategies actually work? How do you identify and eliminate waste? Any recommendations for automation or tooling beyond the built-in Azure Cost Management features?

2

One thing that helped us tremendously was setting up automated budget alerts with action groups. We create budgets at the subscription level and at the resource group level for high-cost areas. When costs exceed 80% of budget, an alert triggers an Azure Automation runbook that analyzes the cost increase, identifies the top cost drivers, and sends a detailed report to the responsible team. This proactive approach catches cost spikes before they become major overruns. We also implemented a ‘showback’ model where each team gets a monthly cost report showing their storage usage broken down by resource group and tagged categories. This transparency drives behavioral change - teams started cleaning up unused data and optimizing their storage configurations without us having to mandate it.

3

Don’t overlook the transaction costs - they can be significant depending on your access patterns. We discovered that one team was using Hot tier storage for archival data that was accessed via a batch process that read every file daily. The transaction costs were 3x the storage costs. We moved that data to a separate storage account with Cool tier and optimized the batch process to use blob inventory reports instead of listing operations. This reduced transaction costs by 80%. Use Azure Monitor to track transaction metrics (TotalRequests, TotalIngress, TotalEgress) and correlate them with costs. High transaction costs usually indicate architectural issues that can be optimized.

4

Beyond tagging, implement lifecycle management policies aggressively. We found that 60% of our storage costs were from data that hadn’t been accessed in over 90 days. Created automated lifecycle policies to move data to Cool tier after 30 days of no access, and to Archive tier after 180 days. This reduced storage costs by 35% without any impact on active workloads. The key is setting up policies at the storage account level with appropriate filters so you don’t accidentally archive data that needs to stay hot. Also, enable storage analytics logs and analyze them to understand actual access patterns before implementing lifecycle policies.

5

Tagging is absolutely critical, but you’re right that retroactive tagging is painful. We implemented an Azure Policy that requires specific tags (CostCenter, Project, Environment, Owner) on all storage accounts and containers, with enforcement mode set to ‘deny’ so new resources can’t be created without proper tags. For existing resources, we wrote a PowerShell script that uses naming conventions to infer the appropriate tags and applies them in bulk. This got us to 95% tag coverage in about two weeks. Once tagging is in place, use Azure Cost Management’s tag-based filtering to slice costs by any dimension you need. We create monthly reports grouped by CostCenter and Project tags, which we distribute to team leads. This visibility alone reduced our storage costs by 20% because teams became accountable for their usage.

6

We use a combination of storage account separation and container-level lifecycle policies. For data with strict retention requirements, we use dedicated storage accounts with immutability policies configured at the account level - this prevents accidental deletion and satisfies compliance. For general-purpose storage, we use container-level lifecycle policies with different rules for different data types. Tag containers with a ‘RetentionPolicy’ tag (e.g., ‘Retain7Years’, ‘Retain1Year’, ‘NoRetention’) and use Azure Policy to enforce that lifecycle rules match the retention tag. This provides flexibility while maintaining governance. Also, consider using Azure Blob inventory to get a comprehensive view of all blobs across your subscriptions - it generates daily reports showing blob count, size, tier, and last modified date, which is invaluable for cost analysis.

7

The lifecycle management approach sounds promising. How do you handle the complexity of different data retention requirements across teams? Some of our teams have compliance requirements that mandate keeping data for 7 years, while others have no retention requirements at all. Do you create separate storage accounts for different retention policies, or can you manage this at a more granular level?

8

Based on implementing cost optimization across multiple large-scale Azure deployments, here’s a comprehensive framework for managing storage costs in multi-subscription environments:

Cost Monitoring Tools and Setup: Azure Cost Management provides the foundation, but you need to enhance it with additional tooling:

  1. Tag-based Cost Allocation: Implement mandatory tagging policy using Azure Policy (enforcement mode: deny). Required tags should include:

    • CostCenter (for chargeback/showback)
    • Project (for project-level cost tracking)
    • Environment (prod/test/dev for cost optimization opportunities)
    • Owner (accountability)
    • DataClassification (drives lifecycle policies)

    For retroactive tagging of existing resources, use PowerShell or Azure CLI scripts that infer tags from naming conventions or resource group assignments. Aim for 95%+ tag coverage within 30 days.

  2. Azure Blob Inventory: Enable blob inventory on all storage accounts to generate daily reports showing:

    • Blob count, size, and access tier distribution
    • Last modified and last accessed timestamps
    • Metadata and tags

    Export inventory to a central Log Analytics workspace for cross-subscription analysis. This provides granular visibility that Cost Management lacks.

  3. Cost Analytics Automation: Create Azure Automation runbooks that:

    • Pull cost data from Azure Cost Management API daily
    • Aggregate costs by tags (CostCenter, Project)
    • Identify cost anomalies (>20% week-over-week increase)
    • Generate and email reports to stakeholders
    • Track cost trends and forecast monthly spending

Tagging for Allocation: Effective tagging strategy requires both policy enforcement and cultural adoption:

  • Use Azure Policy to require tags at creation time (prevents untagged resources)
  • Implement tag inheritance from resource groups to child resources where possible
  • Create tag taxonomy documentation that all teams follow
  • Set up monthly tag compliance reports showing which teams have untagged resources
  • Include tag compliance in team KPIs to drive adoption

For cost allocation, the CostCenter tag is critical - map it to your organization’s financial structure so costs can be charged back or shown back accurately. Use Azure Cost Management’s tag-based filtering to generate monthly cost reports grouped by CostCenter and Project.

Lifecycle Management Implementation: Lifecycle policies are the most impactful cost optimization technique, typically reducing storage costs by 30-50%:

  1. Access Pattern Analysis: Before implementing policies, analyze actual access patterns using storage analytics logs or blob inventory last-accessed dates. Common patterns:

    • Application data: Hot tier, accessed frequently
    • Backups: Cool tier after 30 days, Archive after 90 days
    • Logs: Cool tier after 7 days, Archive after 30 days
    • Analytics data: Hot during processing, Cool for historical analysis, Archive for long-term retention

  2. Policy Design: Create lifecycle policies at the storage account level with filters for different data types:

    
Rule 1: Move blobs to Cool tier 30 days after last modification
Rule 2: Move blobs to Archive tier 180 days after last modification
Rule 3: Delete blobs 365 days after last modification (for non-compliance data)

    Use blob prefixes or tags as filters to apply different policies to different data types within the same storage account. For compliance data, create separate storage accounts with appropriate retention policies and immutability protection.

  3. Monitoring and Adjustment: After implementing policies, monitor for unintended consequences:

    • Track early retrieval costs from Cool/Archive (indicates data moved too aggressively)
    • Monitor application errors related to archived data access
    • Measure actual cost savings vs. projections
    • Adjust policies based on observed access patterns

Budget Alerts and Proactive Management: Set up multi-level budget alerts to catch cost overruns early:

  • Subscription-level budgets at 80%, 90%, 100% of monthly allocation
  • Resource group budgets for high-cost areas
  • Action groups that trigger Azure Automation runbooks to analyze cost spikes
  • Slack/Teams integration for real-time notifications to responsible teams

Implement a showback model where each team receives monthly cost reports. This transparency drives behavioral change - teams become cost-conscious when they see their usage quantified. In our implementations, showback alone typically reduces costs by 15-25% within the first quarter as teams identify and eliminate waste.

Cross-Subscription Visibility: For multi-subscription environments, consolidate cost data centrally:

  • Use Azure Cost Management’s scope selector to view costs across subscriptions
  • Export cost data to a central Log Analytics workspace or Azure Data Explorer
  • Create Power BI dashboards showing costs by subscription, CostCenter, Project, Environment
  • Set up automated reports distributed to finance and team leads monthly

Common Cost Drivers and Optimization: Based on analysis across hundreds of Azure deployments, typical storage cost distribution:

  • 40-50%: Unused or rarely accessed data that should be archived or deleted
  • 20-30%: Data in incorrect tier (Hot when should be Cool)
  • 15-25%: Transaction costs from inefficient access patterns
  • 10-15%: Legitimate active data storage costs

Focus optimization efforts on the first two categories for maximum impact. Implement automated cleanup policies for test/dev environments where data retention is typically unnecessary beyond 30-90 days.

Tooling Beyond Azure Native: Consider third-party tools for enhanced visibility:

  • CloudHealth or Cloudability for multi-cloud cost management
  • Spot.io for AI-driven cost optimization recommendations
  • Custom solutions using Azure Cost Management APIs + Power BI for tailored reporting

However, most organizations can achieve 70-80% of potential savings using only Azure native tools (Cost Management, Azure Policy, Lifecycle Management, Automation) if implemented systematically.