The development team at Adobe accidentally blew $80k in just one day while running a single computing job on Azure a couple of years ago. The mistake wasn’t spotted for a week, by which point the bill had skyrocketed to $500,000!
Adobe aren’t the only company to be blindsided by unexpected data costs. Last year, Pinterest (which had paid in advance for a fixed capacity with Amazon Web Services) saw a 41% spike in their costs as users flocked to the online scrapbook during the holidays. This meant Pinterest had to buy additional capacity at a much higher cost. Around the same time, Capital One also saw a 73% hike in their costs for the same seasonal reason.
If even these companies with extensive budgets and large IT teams can fall foul of hidden data costs, the impact these costs can have on SMEs and other organisations could be devastating.
The explosion of cloud computing, multiple integrations and the surge of demand for data storage in the cloud, means even small enterprise businesses now have a significant data transfer requirement. This means that data transfer costs need to be factored into an organisation’s overall IT spend for budgeting purposes. But with a rapidly diversifying marketplace for cloud solutions, staying on top of these costs and how to work them out can be difficult.
In this blog, we hope to demystify these costs, point out some of the hidden costs that can quickly mount up, and help you make an informed decision about the right cloud solution for your business.
Budgeting for data costs
Whether you’re regularly backing up data between systems or securely exchanging large volumes of data between organisations, there are a number of things to consider when choosing the right cloud storage and cloud backup solutions.
You’d think it would be easy to compare different prices. But when you dig down into the hidden costs of data transfer, the complexity can feel like somewhat of a minefield. This is further exacerbated by the complexity of pricing structures even within a single supplier model – they many contain hidden costs and pitfalls.
Let’s start with the overall pricing structures for two big players: Azure and Amazon Web Services. Both Azure and AWS have a free offer within their pricing tiers to help you get started, but ultimately before you invest time in any single provider, it’s a good idea to understand all of the elements to their pricing structures so you don’t get caught out later on down the line.
Microsoft Azure pricing
With its price match promise for comparable services, Azure is less expensive than AWS on Windows and MS SQL databases and servers. Microsoft also proves various trial options to help you assess whether Azure is the right platform for you and ultimately to encourage you to invest more heavily in it.
There are three payment models for Azure:
- Pay as you go – you only pay for what you use, so can be a good option for smaller storage requirements, however costs can quickly spiral if your usage goes up.
- Pre-paid subscriptions – the price per unit for this option (which involves reserving capacity up to a year ahead) is lower than for PAYG so can work well if you know in advance exactly what storage you will need, but due to the reservation aspect, costs can be complicated.
- Enterprise licensing agreements – this is the most cost effective option from a unit price point of view, but the downside is the minimum three-year contract commitment making it the least flexible option.
Amazon Web Services (AWS) pricing
Amazon Web Services offers a wide range of services but this can mean that pricing up and managing costs, particularly for larger volume requirements, can be overwhelming. To try this, Amazon offers a number of cost optimisation tools that we will cover later.
The three pillars of Amazon’s pricing model are:
- Pay as you go – Amazon effectively leases you computing resources so you can pay by the hour and convert capital expenses into operating expenses. This option is the most expensive and should only be used in exceptional circumstances.
- Pay less by using more – this option offers discounts for large volumes but also ties you in to increased reliance on Amazon, which is something to consider if you want to mitigate the risks of overreliance on one provider and maintain a healthy multi-cloud strategy.
- Save when you reserve – 30-50% discount for reserving capacity from 1-3 years in advance, although if you can plan this far ahead you’re probably better off with an on-premise solution!
Amazon Web Services data costs are notoriously difficult to calculate due to the complexity of its pricing structure. If you’re an AWS customer you might have spotted some data transfer costs appearing on your monthly cloud bill. With so many variables including numerous different transfer types each with a different associated cost, staying on top of what you should be paying for AWS data transfer can be an onerous and complex task. Hidden costs can creep in round every corner.
Look out for these general “hidden” costs
Here are some of the most common ways that both Amazon and Azure users can incur additional charges:
- Transferring data out of the cloud
When uploading data to the cloud is free, it can come as a surprise to find out that transferring that same data out of the cloud incurs costs. Always check the small print! Or scroll down for more information about data transfer costs for the two largest providers.
- Leaving volumes unattached
Termination of an EC2 instance deletes the root volume by default, but the attached EBS volumes will continue to run and generate additional costs. Left unattached, costs from Solid State Drive (SSD) backed volumes can also mount up.
- Letting instances lay idle
Always remember when you stop your Azure virtual machine (VM) or Amazon EC2 instances, to also put a halt to the network and compute resources provision or EBS volume too in order to stop accruing costs. Pay as you go service users can get hit with some nasty surprises in their bills afterwards if not.
- Unhealthy instances
Always correctly configure the endpoint to create a healthy application status. A common method of dealing with unhealthy instances is through the use of auto scaling, which both AWS and Azure allow for. Horizontal scaling involves configuring resources into “instance clusters” and defining conditions (eg CPU utilisation above or below a set threshold) to automatically adjust the number of running machines within each of your clusters.
- Redundant static IP addresses
To encourage efficient use of IP addresses, public cloud providers charge a fee on those that lay dormant. AWS users have one free Elastic IP (EIP) address with each EC2 instance running, then additional IP addresses (or any that aren’t associated with a running instance) are charged on top by the hour. Microsoft Azure customers who use the Azure Resource Manager (ARM) deployment model are allowed up to five static public IP addresses per region.
- Underutilising or underestimating pre-booked capacity
Both Azure and AWS offer large discounts for pre-booked capacity. As detailed above, the allowable lead time for pre-booking differs between these services. But what doesn’t differ is the hit you will take if you under or overestimate your future (pre-booked) capacity – such as in the case of Pinterest we referenced earlier. To correctly allocate your storage, you should modify the machine specifications in Azure or the Reserved Instances in AWS to match the resources.
These are just some of the common pitfalls that organisations come across when managing their data capacity and budgets. To fully understand AWS and Azure data transfer costs, we need to dig even deeper.
The hidden costs of Azure data transfers
While the transfer of data into Azure is free, other types of data transfers incur additional costs. The variables are detailed below.
Data transfers between Azure regions (zones)
When both services (between which data is to be transferred) are based in the same region, there are no associated costs. But if each service is located in a different region, the charges kick in based on the “zone” from which the data is being sent.
Data transfers between Azure availability zones
The reason many organisations trust Azure is that it provides a robust and resilient infrastructure to protect against outages. One of the ways that Azure does this is by deploying resources into separate availability zones, so if a data centre in one availability zone goes down, other availability zones are not affected.
Data transfers between networks
Perhaps surprisingly, charges are payable for both inbound and outbound inter-network data transfers, where two Azure networks are connected through Azure VNET peering – even though both are within the Azure network.
Outbound data transfer
Azure allows for up to 5GB per month of zero cost outbound data transfers. Following that, the standard Azure pricing tiers come into play. The tiers go up in price from 5-10TB, 40TB, 100TB, 350TB and 500TB+.
The hidden costs of AWS data transfers
Amazon’s AWS Elastic Compute Cloud (EC2) service is extremely flexible but in offering such a wide range of variables means its pricing can be notoriously complex to navigate. If you’ve already checked through the general list above for culprits of your sky high bills, here are some of the other ways that AWS data costs can rack up.
Data transfers between AWS regions
These data transfers incur the highest costs within the AWS pricing structure. It’s therefore important to consider carefully which regions your data needs to transfer between and plan your data flow accordingly. To compound the complexity, transfer between different regions varies further still.
Data transfers between AWS availability zones
Transferring data between different AWS availability zones also incurs costs. If (and only if) you are using a private IP address, intra-availability zone transfers are free. Otherwise, transferring data between AWS services located in the same region but in different availability zones is chargeable at a rate of $ 0.01/GB for outgoing data.
Re-hosted applications
If re-hosted applications are not aligned and/or configured and/or aligned with AWS features, they can also incur additional costs. In this case we advise they are re-architected through the cheapest route possible.
Outbound data transfers
Whereas transferring data into EC2 is usually free, transferring data out to the internet incurs significant charges.
Luckily, Amazon Web Services provides a number of free tools to help you calculate its myriad of costs. Here are details on how to use AWS Cost Explorer to analyse data transfer costs, giving detailed insights into the line items on your bill.
The best choice for your data infrastructure
Ultimately, public cloud providers are usually more expensive for production environments, but do offer the flexibility for large scale up. Independent hosting providers using solutions like VMware can provide a far simpler and more cost effective option than those offered by these providers. virtualDCS is a good example of such a provider.
Here’s why. Firstly, Azure don’t offer a Service Level Agreement as standard on their simple Virtual Machines. Azure users need to pay for a premium disk if they want a 99.9% SLA.
If they’re after a more extensive 99.95% SLA with Azure, they’ll need to administer two servers, as Azure requires them to have two instances in the same Availability Set to qualify. This requires load balancing and the associated challenges (and costs) of syncing data between two servers. On VMware this is achieved with HA (High Availability). This does not require two active servers and simplifies the process.
The end result? Users need to pay more for a more extensive SLA with Azure, as well as pay to keep the data in sync across different zones. Here at virtualDCS, we offer a 99.95% SLA as standard and our users don’t have those synching data charges to deal with.
Independent cloud providers usually offer simpler prices for solutions, making it easier to budget and removing the chance of expensive mistakes.
A practical example of this is that you might have 30 instances of downtime of one minute each, over the space of a month. Azure would only consider one of these as eligible for a service credit, whereas virtualDCS would consider them all.
Need some help?
If you’d like to discuss how to fill the data capacity gaps in your existing infrastructure, virtualDCS can help.
We don’t just sell a particular off the shelf solution. We help organisations develop a bespoke, cost effective solution that works best for them – including both on premise and in the cloud or a hybrid solution utilising elements of both.
Want to find out more? Read our blog on What Amazon and Azure can’t do.
Then when you’re ready, give us a call on +44 (0)3453 888 327 or email enquiries@virtualDCS.co.uk and we will be happy to help.
