Cloud migration initiatives are complex, expensive undertakings. Many things can go wrong, which is why organizations perform meticulous planning before migrating important systems. To reduce risk, most organizations start with a POC, migrating non-critical applications first, and use migration assessment tools to identify problems in advance.
While many articles have been written about what can go wrong with a migration and how to prepare, few have focused on migrations that go right. In this article, I collected three stories about organizations that moved business critical systems to the cloud, experienced a smooth migration, and met and exceeded their goals. These stories can serve as inspiration, and also teach us a few lessons about how to do things right when moving to the cloud.
Spotify: Bare Metal to Cloud
Spotify provides media services through an audio-streaming platform that enables users to search, share, and listen to music and podcasts.
Spotify had massive local data centers that were difficult to maintain and provision. The company decided to free up time for their engineers so they could prioritize innovation rather than maintenance. In 2015, they started planning their migration to Google Cloud. They defined two key objectives—minimizing disruption to software development and reducing the costs and complexity of a hybrid operation.
Spotify spent two whole years on the pre-migration preparation phase. They assigned a dedicated Spotify-Google cloud migration team, which oversaw the effort. The migration effort was split into two main parts, one dedicated to services and the other to data. They invested two years in total—one year to complete each part of the strategy.
Spotify’s engineering team worked in sprints. During two weeks, they paused all product development work and focused primarily on migrating services to Google Cloud. When working on data migration, teams were either responsible for “forklifting” or rewriting. The result of this effort was a significant increase in scalability and development time.
90 Seconds is a video creation platform, which helps brands to plan, shoot, and edit video content from any location in the world. They manage a marketplace of 12,000 video professionals located in more than 160 countries across 70 categories, and their 3,000-plus clients include many global enterprise brands.
90 Seconds started using new technologies to scale the business after receiving millions of dollars in venture capital several years ago. The company received additional funds later on, and continued to scale the business.
They started operations in a colocated data center in the U.S. and as the business began to rapidly expand, they ran into scalability and cost issues, particularly around storage and content delivery. To solve these issues, they moved to another cloud service—Google Cloud.
90 Seconds leveraged Google Cloud services to support their growing marketplace and they are now adding machine learning (ML) capabilities through cloud AutoML, so they could provide deeper and more relevant analysis for brands. This type of service can help the company to easily train ML models for video analysis. Google Cloud provides the Video AI service that enables automated analysis of video content and extraction of metadata at shot or frame level.
The company now uses ML models to extract content from videos, such as footage of people, and analyze how footage contributes to performance in terms of attracting viewers and driving engagement. They plan to use Google-powered ML, and possibly other video APIs, to power a chatbot that provides customers with answers to frequently asked questions.
DocuSign helps organizations automate their agreements. Customers can automate the preparation, signing, and management of their agreements. DocuSign’s cloud-based platform includes eSignature features that let partners electronically sign agreements on any device.
DocuSign developed their own business intelligence (BI) infrastructure and analytic solutions. The company relies on this technology to make data-driven decisions. However, as the company continued to grow, its legacy infrastructure could not meet the increasing scalability and performance demands. Eventually, their infrastructure became too expensive to maintain on premises.
DocuSign created a data modernization strategy. The first step was to select a cloud data warehouse (CDW), and they needed a solution that could help them:
- Move their data from the existing SQL Server into the chosen CDW.
- Quickly process massive amounts of data.
- Ensure that all data in transit remains secure.
After evaluating several solutions, DocuSign chose Snowflake as its CDW vendor and Matillion ETL for Snowflake. Matillion was deployed from within the AWS Marketplace, launched directly into DocuSign’s EC2 instance, and was then integrated with Snowflake. The solution helped aggregate multiple data sources. Matillion was also responsible for creating dimensional models, which were needed to achieve downstream consumption.
Snowflake and Matillion helped DocuSign improve scalability, performance, improve security, and reduce costs on AWS via resource optimization. Using this strategy, DocuSign was successful in reducing the time needed for long-running jobs from approx. 22 hours to only 6 hours.
In this article I reviewed three successful cloud migration stories:
- Spotify decommissioned their bloated on-premise data center and moved their entire operation to Google Cloud Platform.
- 90 seconds moved their video processing and delivery system from a colocated data center to Google Cloud.
- DocuSign moved their enterprise data warehouse to a cloud solution based on Snowflake and Matillion.
The common denominator of these three stories is that the three companies took a phased approach, worked closely with their cloud provider, and were not afraid to go all out—moving an entire production system to the cloud and rebuilding it to take full advantage of the new platform. Like many things in life, avoiding shortcuts and building their operation from the ground up was the key to long term success.