Welcome to this third and final post in our series on Migrating to AWS. This blog post series will continue from Parts 1 and 2 and cover a number of areas you should look out for when migrating to AWS, and provide methods of working to ensure you can successfully move an existing on-premise or other provider-hosted workloads to AWS with a minimal amount of issues.
Overview
As a reminder, these posts will cover a number of topics, split into 3 parts. Each part is split up as follows:
- Why Migrate to AWS?
- On-Premise Constraints
- AWS Tools to Help with Migration
- Methods of Migration (The 6 Rs or is it 7?)
- Document Everything
- Phases of Migration
- Prepare / Discovery Phase
- AWS Account Setup
- CloudEndure
- Things to watch out for when using CloudEndure
- Application Migration Service (AWS MGN)
- Database Migration Service (DMS)
- Moving Complex Data
- Migrating – What to Do and When
- After Migration
Database Migration Service (DMS)
When transferring a Database to AWS, there are a few different ways you can do this. You can use native tools such as MySQLDump, or pg_dump, but AWS also offers their Database Migration Service.
You can use the DMS tool to move data from your source database to an AWS hosted one in RDS. You can also use it to convert between one Database and another, such as moving from Postgres to SQL Server.
There are some caveats with DMS though and it doesn’t always work with complex databases. AWS DMS doesn’t automatically create secondary indexes, foreign keys, user accounts, and so on, in the target database, so you will need to do some additional work to get everything running smoothly.
There are also some restrictions on source data, including MyISAM tables in MySQL, so if you are using those they need to be converted to InnoDB or migrated using another method.
It’s recommended to do a test migration to see if any of your data is not supported and to discover what else you might need to do to get your database up to production standards.
Moving Complex Data
In some cases, the AWS services can’t quite migrate all of the data you need, especially if you use Network Attached Storage or have a large number of small files.
AWS DataSync is a useful service if you can get permission to run it. It’s deployed as a virtual machine attached directly to your NAS server and syncs data directly to AWS, but this can be tricky to get permission to install from the on-premise provider.
If you can’t use one of the provided tools, you can use open-source data transfer tools such as rsync. This is especially useful with file systems that have lots of small files, as you can split up the folders being synced and run multiple rsyncs in parallel. If you have customer files split out in folders per customer, you can run an rsync for each one (in a screen command to stop interruptions) and migrate your folders in parallel.Â
The right strategy for moving your data (CloudEndure, DataSync, Storage Gateway, Rsync etc) depends on how much data you have and the distribution of files and file sizes. You can also use multiple approaches for different parts of your data.
If you have a lot of data, then the Snow Family of devices can be useful, a Snowball for example can be used to physically transfer 80 TB of data, or if you really have a lot, you can rent a Snowmobile truck, and transfer up to 100 Petabytes of data per truck.
Migrating – What to Do and When
When it comes to the actual migration (whether that is a single day, or moving different parts of the application at different times) there are some useful things you can do to help the migration process run smoothly.
- Follow the plan you set out in the earlier stage, and don’t skip any steps.
- Document Everything. It’s important to capture what has been done and how it will be supported in the future.
- Make sure everyone involved in the migration process is kept informed of what has happened so far, and any issues that crop up.
- Communicate with others in the Migration team continuously. Don’t go away and do one part of the plan and forget to tell anyone else, as that will just lead to duplication or confusion.
- If you do have to fix something that wasn’t picked up in the discovery phase, make sure it’s documented and fit for production once you have completed the migration. It’s tempting to fix the current blocker by a temporary workaround (such as opening a port to allow access), but if it’s not documented you might as well not have done it, because it will disappear or get forgotten about at some point in the future.
After Migration
If you get to this point in your Migration Journey, Congratulations! You’ve completed a difficult task in moving an existing platform to AWS.
However, this is not the end of the road, but a new beginning. You can now concentrate on running your workload on AWS, optimizing it where you can and settling into the new environment and supporting your workload.
Once that has settled down, you can look at further architectural changes and splitting up any monolithic elements you might still have and using more AWS services, and cloud-native ways of working such as serverless.
Conclusion
We hope you’ve found some useful info here that can help you in your migration journey. Migrating to AWS is a large undertaking with a lot of potential pitfalls along the way, and we hope we’ve provided useful information on some of these to make your own Migration smoother.
Please feel free to get in touch with us to see how we can help you in your Migration journey to AWS: https://www.cirrushq.com/contact/#contact-form
