Best practices to manage your data and optimize cloud data costs
Half of businesses struggle to keep their cloud costs under control, according to a recent survey of American IT directors and executives. Nine in 10 businesses currently host some or all of their IT infrastructure in the cloud rather than on-premises — and of those that don’t, 60% say migrating to the cloud is a top priority. By 2025, more than 85% of organizations will be cloud-first, according to Gartner.
But with that comes a whole host of associated costs for businesses, specifically when it comes to managing data in the cloud. IDC has tracked a 29% year-over-year increase in cloud services spending, and businesses spent more than $400 billion keeping their key services online and in the cloud. That’s in part because the cloud makes everything an operating expense , rather than a capital expense. Payment sizes to cloud service providers depend on the data and compute used — which can vary significantly from month to month.
Shifting to the cloud unlocks the possibility to pull in more data from more sources, pushing costs up commensurately. What were once small inefficiencies can be amplified into massive ones as data inputs multiply. The combination of exploding data, near infinite elasticity of the cloud and “pay for what you use” model can make it difficult to forecast costs. And for many, the benefits of managing data in the cloud become obsolete when resulting in unexpected costs.
Overspending is an issue more businesses face when managing data in the cloud. In fact, a recent Forrester study cites that 82% of data management decision-makers report forecasting and controlling costs as a data ecosystem challenge. Businesses can benefit from best practices shared by organizations who have faced these challenges head on.
How Capital One controlled its data costs
Capital One exited its last data center in 2020 and became the first US financial company to move fully on the public cloud . But to fully realize the benefits of operating in the cloud, Capital One needed to modernize its data ecosystem. The company adopted Snowflake as its data cloud, which gave it the ability to scale instantly for nearly any workload. However, as their access to data increased, they needed to build new data management platforms to solve data ecosystem challenges, including tackling Snowflake costs.
“When you have unlimited compute and power, you can very easily go from data starved to data drunk. We built tools to put the proper governance and cost control measures in place to make sure we were provisioning our data platform in a well-managed way,” said Patrick Barch, senior director of product management, Capital One Software. “It was also important for that experience to be self-service so analysts could easily access the data they needed while complying with policy guidelines.
