Now-a-days lots of data gets generated in every organization. To preserve data, organizations invest huge amount of money on infrastructure. To reduce infrastructure cost, organizations has come up with data migration i.e., moving of data from their on-premises to cloud. One such cloud service is Amazon AWS and its storage services are in the form of block, file and object.
Fig: Storages Services from Amazon AWS
Some of the online & offline services in Amazon’s AWS data migration, they are listed below in a figure
The basic question for every organization is
- What kind of data is it, and where is it going?
- 1-time or continuous movement?
- 1-way or bi-directional?
- How much data & time do you have?
- How about the WAN?
Fig: Few question in general while Data migration
Let’s discuss about some data migration services of Amazon AWS.
AWS Direct Connect:
- Establish private network between AWS and Data center, office and co-location environment.
- Dedicated connection can be partitioned into multiple virtual interfaces.
- Maintains network separation between public and private environments.
- Public resources such as S3 using public IP address or Private resource such as using EC2 within VPC with private IP address.
Fig: AWS Direct Connect
- Reduces bandwidth cost.
- Consistent network performance.
- Compatible with all AWS Services
- Private connectivity to AWS VPC.
- Elasticity (scale up/down the connections)
- Pay only for what you use. No minimum fee.
- Physically transport petabytes of data in batches to the cloud.
- Transfer hundreds of terabytes or petabytes of data between on-premises data centers and AWS.
- Creation of jobs, transfer and track the status makes the user to transfer data easily and can move more than 250TB in a week.
- Snowball uses 256-bit encryption and is protected by AWS KMS (Key Management System)
- Snowball is capable of handling 80TB capacity or 10 GB network at time.
- Snowball is available in 50 and 80 terabyte models, which it depends upon region.
- S3 is used in final stage after moving into cloud.
Fig: Snowball working Steps
Fig: Snowball encryption and rugged 8.5G case
- High Speed
- Extremely scalable
- Tamper resistant and secure
- Simple and compatible
- Low cost
- Easy data retrieval
Amazon EFS File Sync
- Efficiently and securely sync files from on-premises or in-cloud file systems to Amazon EFS with 5x faster than standard Linux copy tools.
- EFS-EFS file systems between AWS Regions.
- EFS basically resides on agent which runs on VMware EXSi or on an EC2 instance and access the file system via NFS.
Fig: EFS sync agent
- Dynamic Elasticity
- Scalable performance
- Shared file storage
- Fully managed
- Cos effective
- Security and compliance
AWS Storage Gateway
- A hybrid storage service that enables on-premises applications use the AWS storage gateway services.
- There are 3 types of Gateway services.
- File Gateway
- Volume Gateway
- Tape Gateway
- On-premises file storage maintained as objects in Amazon S3.
- One-to-one mapping from files-to-objects.
- Buckets are accessed through IAM role you own and manage.
Fig: Storage Gateway for Files
- On-premises volume storage backed by Amazon S3 with EBS snapshots.
- Creating and backing up on-premises volumes from and to with EBS.
- Data compressed in-rest and at-rest.
Fig: Storage Gateway for Volumes
- Virtual tape storage in Amazon S3 and Glacier with VTL (Virtual Tape Library) management.
- The name Tape represents, AWS supports Tape drives also.
Fig: Storage Gateway for Tapes
- Seamless integration across standard storage protocols
- Low-latency access
- Durability, cost and elasticity of AWS storage services.
- Efficient data transfer
- Data encryption
- Integrated with AWS monitoring, management and security
Basically, data migration involves 3 steps, as ETL (E-Extract, T-Transform and L-Load). The migration of data is defined in 3 types where each type follows ETL only, those 3 types are Storage migration, Cloud migration and Application migration. Based on the customer requirement, the action starts and by migrating huge amounts of money spent on infrastructure reduces and they pay for the virtual environment only. So we, VT suggest you take any one of the storage service from Amazon AWS.