AWS Migration

Registry Case Study

GoDaddy, the domain registration and management giant, is now migrating their majority of their infrastructure and data warehouse to AWS. GoDaddy group decided to go with AWS due to its deep experience in delivering a highly reliable global infrastructure, as well as an unmatched track record of technology innovation, to support their rapidly expanding business.

The Challenge

GoDaddy completed its $218 million acquisition of Neustar’s domain name registry business in the year 2018, marking a big entry into the wholesale part of the domain business.

Neustar registry was hosted in Greenplum Data warehouse server and was processing almost two TBs of data every day. The existing database had performance issues and the cost of ownership was too high. Hence, GoDaddy needed a new database solution, which is scalable, easier to maintain and is able to process the huge amount of data.

Considering how critical and busy this data warehouse is, migrating the data warehouse to a new environment seamlessly without any downtime was a challenge for the GoDadddy team.

Why Amazon Web Services

After evaluating a number of solutions, including other cloud service providers, VirtueTech suggested Amazon Web Services (AWS) because of AWS’ highly scalable, fully managed and cost-effective storage and compute services, as well as its ability to accommodate open-source software and hybrid architecture.

Amazon Redshift delivers fast query performance on gigabytes to petabytes of data, and from a few users to thousands. The ability to connect directly from Tableau to a Redshift cluster means letting anyone easily access your cloud data warehouse and quickly perform visual analysis via Tableau’s intuitive drag and drop interface.

Running Critical Applications on AWS

While there were plenty of ways to migrate the workloads to cloud, the cheapest and easiest method was be the lift-and-shift migration. It enabled GoDaddy to enjoy the full benefits of AWS, such as enhanced speed and lower cost of operation

GoDaddy is using Amazon Simple Storage Service (Amazon S3) to store the cloud-based portion of its data lake, AWS Redshift to handle data warehouse operations and Apache Airflow to programmatically author and schedule their workflows and monitor them via the built-in Airflow user interface. End user is notified of failure via slack with the help of AWS managed airflow. Tableau is connected to the AWS Redshift tables through Redshift connector, which is used for report visualization.

The Benefits

By replacing its existing on-premises database with the Amazon Redshift cloud data warehouse, GoDaddy now has a scalable, powerful data solution that is easier to maintain than its previous solution.

Using the AWS Redshift data warehouse has dramatically reduced the time required to obtain registry analysis reports. Analysis performance has also improved since shifting to Amazon Redshift, with faster response and better usability for heavy users.

The AWS cloud has also met GoDaddy’s requirement to maintain operating costs at the same level as with the Greenplum data center.

Advertising Channel Data Mart Migration Case Study

The Challenge

Advertising is important for every aspect of a business, as it lets the business gain more customers, thereby increasing business turnaround. It can be achieved using various media like television, social media, websites etc. 

GoDaddy being an online business, used to collect Data from these sources and store into Apache hive, which was then loaded into GCP Big Query for the transformations and Amazon S3, Amazon Redshift for generating reports. 

Too many hops in this data pipeline was creating dependencies on other platforms like Hadoop and GCP, and also this approach lacked the ability to manage the upscaling of the application. Thus, Virtue Tech suggested a way to decommission the GCP, Hadoop and manage AWS for both transforming and analytical purposes.

Why Amazon Web Services

Lack of capability to use columnar storage for storing metric data in GCP was increasing the storage space and the costs compared to using S3 for storage by and EMR for processing the data.  Amazon EMR makes it easy to set up, operate, and scale your big data environments by automating time-consuming tasks like provisioning capacity and tuning clusters.  We also saved 50-80% on the cost of the instances by selecting Amazon EC2 Spot for transient workloads and Reserved Instances for long-running workloads. Three R5.4 nodes used within EMR cluster costs 3 x $0.25/hour equates to less than a dollar for both tasks during a batch run

Running Critical Applications on AWS

Data from multiple sources such as Google Analytics, Salesforce etc. is pulled into Amazon Simple Storgae Service (S3) in CSV format through APIs. Amazon EMR runs a generic configuaration based Pyspark framework, which has automated the entire process of transforming, cleaning and loading the data into S3 ADS Layer.  Amazon Redshift table is created on top of this S3 ADS Layer. Tableau is connected to this redshift table through Redshift connector, which lets the analysts analyze the data and generate useful business insights

The Benefits

Using AWS, GoDaddy now has access to the scale it requires to deliver a reliable and insightful service to its customers. With its earlier setup comprising of GCP and on-premise, the availability of its system ran to 98 percent, but on its AWS cloud infrastructure, this has risen to 99.965 percent. Running the pipelines on AWS infrastructure has also resulted in saving time as well as money. While, the previous system took almost more than 50 minutes to run the end-to-end flow, however on AWS it hardly takes 20 minutes to complete. Company is also saving almost 3000 USD per month since previous infrastructure included extra storage cost of google cloud storage and on-premise hadoop cluster.