[Estimated read time: 9 minutes]
I know you would never change a URL without identifying where to 301-redirect it and making sure that the links, XML sitemaps, and/or canonical tags are also updated. But if you've been doing SEO for a while, I bet you've also had a few clients — even big ones — coming to you after they've tried to do structural web changes or migrations of any type without taking SEO best practices into consideration.
Whenever this happens, your new client comes to you for help in an "emergency" type of situation in which there are two characteristics when doing the required SEO analysis:
- You need to prioritize:
Your client is likely very nervous about the situation. You don't have a lot of time to invest at the beginning to do a full audit right away. You'll need to focus on identifying what hasn't been done during the migration to make sure that the fundamental causes of the traffic loss are fixed — then you can move on with the rest.
- You might not have all the data:
You might have only the basics — like Google Analytics & Google Search Console — and the information that the client shares with you about the steps they took when doing the changes. There are usually no previous rankings, crawls, or access to logs. You'll need to make the most out of these two fairly easy-to-get data sources, new crawls that you can do yourself, and third-party "historical" ranking data. In this analysis we'll work from this existing situation as a "worst-case scenario," so anything extra that you can get will be an added benefit.
How can you make the most out of your time and basic data access to identify what went wrong and fix it — ASAP?
Let's go through the steps for a "minimum viable" web migration validation to identify the critical issues to fix:
1. Verify that the web migration is the cause of the traffic loss.
To start, it's key to:
- Obtain from the client the specific changes that were done and actions taken during the migration, so you can identify those that had been likely missed and prioritize their validations when doing the analysis.
- Check that the time of the traffic loss coincides with that of the migration to validate that it was actually the cause, or if there were different or coinciding factors that might have affected at the same time that you can later take into consideration when doing the full audit and analysis.
To identify this, compare the before and after with other traffic sources, per device & the migrated areas of your site (if not all of them changed), etc.
Use the "Why My Web Traffic Dropped" checklist to quickly verify that the loss has nothing to do with, for example, incorrect Google Analytics settings after the migration or a Google update happening at the same time.
I've had situations where the organic search traffic loss had coincided not only with the web migration but also with the date of a Phantom update (and they had the type of characteristics that were targeted).
If this is the case, you can't expect to regain all the traffic after fixing the migration-related issues. There will be further analysis and implementations needed to fix the other causes of traffic loss.
2. Identify the pages that dropped the most in traffic, conversions, & rankings.
Once you verify that the traffic loss is due (completely or partially) to the web migration, then the next step is to focus your efforts on analyzing and identifying the issues in those areas that were hit the most from a traffic, conversions, & rankings perspective. You can do this by comparing organic search traffic per page before and after the migration in Google Analytics:
Select those that previously had the highest levels of traffic & conversions and that lost the highest percentages of traffic.
You can also do something similar with those pages with the highest impressions, clicks, & positions that have also had the greatest negative changes from the Google Search Console "Search Analytics" report:
After gathering this data, consolidate all of these pages (and related metrics) in an Excel spreadsheet. Here you'll have the most critical pages that have lost the most from the migration.
3. Identify the keywords for which these pages were ranking for and start monitoring them.
In most cases the issues will be technical (though sometimes they may be due to structural content issues). However, it's important to identify the keywords for which these pages had been ranking in the past that lost visibility post-migration, start tracking them, and be able to verify their improvement after the issues are fixed.
This can be done by gathering data from tools with historical keyword ranking features — like SEMrush, Sistrix, or SearchMetrics — that also show you which pages have lost rankings during a specific period of time.
This can be a bit time-consuming, so you can also use URLProfiler to discover those keywords that were ranking in the past. It easily connects with your Google Search Console "Search Analytics" data via API to obtain their queries from the last 3 months.
As a result, you'll have your keyword data and selected critical pages to assess in one spreadsheet:
Now you can start tracking these keywords with your favorite keyword monitoring tool. You can even track the entire SERPs for your keywords with a tool like SERPwoo.
4. Crawl both the list of pages with traffic drops & the full website to identify issues and gaps.
Now you can crawl the list of pages you've identified using the "list mode" of an SEO crawler like Screaming Frog, then crawl your site with the "crawler mode," comparing the issues in the pages that lost traffic versus the new, currently linked ones.
You can also integrate your site crawl with Google Analytics to identify gaps (ScreamingFrog and Deepcrawl have this feature) and verify crawling, indexation, and even structural content-related issues that might have been caused by the migration. The following are some of the fundamentals that I recommend you take a look at, answering these questions:
A.) Which pages aren't in the web crawl (because they're not linked anymore) but were receiving organic search traffic?
Do these coincide with the pages that have lost traffic, rankings, & conversions? Have these pages been replaced? If so, why they haven't been 301-redirected towards their new versions? Do it.
B.) Is the protocol inconsistent in the crawls?
Especially if the migration was from one version to the other (like HTTP to HTTPS), verify whether there are pages still being crawled with their HTTP version because links or XML sitemaps were not updated... then make sure to fix it.
C.) Are canonicalized pages pointing towards non-relevant URLs?
Check whether the canonical tags of the migrated pages are still pointing to the old URLs, or if the canonical tags were changed and are suddenly pointing to non-relevant URLs (such as the home page, as in the example below). Make sure to update them to point to their relevant, original URL if this is the case.
D.) Are the pages with traffic loss now blocked via robots.txt or are non-indexable?
If so, why? Unblock all pages that should be crawled, indexed, and ranking as well as they were before.
E.) Verify whether the redirects logic is correct.
Just because the pages were redirected doesn't mean that those redirects were correct. Identify these type of issues by asking the following questions:- Are the redirects going to relevant new page-versions of the old ones?
Verify if the redirects are going to the correct page destination that features similar content and has the same purpose as the one redirected. If they're not, make sure to update the redirects. - Are there any 302 redirects that should become 301s (as they are permanent and not temporary)
Update them. - Are there any redirect loops that might be interfering with search crawlers reaching the final page destination?
Update those as well.
Especially if you have an independent mobile site version (under an "m" subdomain, for example), you'll want to verify their redirect logic specifically versus the desktop one.
- Are there redirects going towards non-indexable, canonicalized, redirected or error pages?
Prioritize their fixing.
To facilitate this analysis, you can use OnPage.org's "Redirects by Status Code" report.
- Why are these redirected pages still being crawled?
Update the links and XML sitemaps still pointing to the pages that are now redirecting to others, so that they go directly to the final page to crawl, index, and rank. - Are there duplicate content issues among the lost traffic pages?
The configuration of redirects, canonicals, noindexation, or pagination might have changed and therefore these pages might now be featuring content that's identified as duplicated and should be fixed.
5. It's time to implement fixes for the identified issues.
Once you ask these questions and update the configuration of your lost traffic pages as mentioned above, it's important to:
- Update all of your internal links to go to the final URL destinations directly.
- Update all of your XML sitemaps to eliminate the inclusion of the old URLs, only leaving the new ones and resubmitting them to the Google Search Console
- Verify whether there are any external links still going to non-existent pages that should now redirect. This way, in the future and with more time, you can perform outreach to the most authoritative sites linking to them so they can be updated.
- Submit your lost traffic pages to be recrawled with the Google Search Console "Fetch as Google" section.
After resubmitting, start monitoring the search crawlers' behavior through your web logs (you can use the Screaming Frog Log Analyzer), as well as your pages' indexation, rankings, & traffic trends. You should start seeing a positive move after a few days:
Remember that if the migration required drastic changes (like if you've migrated over another domain, for example), it's natural to see a short-term rankings and traffic loss. This can be true even if it's now correctly implemented and the new domain has a higher authority. You should take this into consideration; however, if the change has improved the former optimization status, the mid- to long-term results should be positive.
As you can see above, you can recover from this type of situation if you make sure to prioritize and fix the issues with negative effects before moving on to change anything else that's not directly related. Once you've done this and see a positive trend, you can then begin a full SEO audit and process to improve what you've migrated, maximizing the optimization and results of the new web structure.
I hope this helps you have a quicker, easier web migration disaster recovery!
Great post (as always) Aleyda
I have had several clients asking for help after a site migration has gone wrong. Far better than trying to repair the damage after the event, we should look at how to avoid this happening in the first place.
From my experience, the biggest single cause for disastrous site migrations are design agencies who know almost nothing about SEO, but pretend that they do.
I have several clients who dropped their agency (and in some cases took legal action) because their expectation was that the agency should know how to migrate a site without causing a significant loss of organic traffic. The agencies argue that technical SEO is not part of the service they provide and that the client should have hired an external SEO consultant.
Clients need to educate themselves better and carry out proper due diligence before hiring an agency. Agencies should be more transparent about the limitations of their technical skill-set.
Completely agree Danny! A few years ago I actually created an SEO checklist for Web migrations, so it was easier to plan them ahead. The reality is that sometimes companies don't think that they require this and is just a quick validation right at the end... others don't even take it into consideration at all, and then disasters end-up happening, and therefore is important to always be prepared :)
Hey Danny,
"The agencies argue that technical SEO is not part of the service they provide" that's a very poor escape tactic for poor service. Agencies that take that angle need to be slapped very hard indeed. However, if indeed they don't provide "advanced SEO" then any agency worth a damn will advise the client up front and very clearly on that. We can't rely on the client having enough tech savvy to ask the question.
Nicely covered Aleyda. Bookmarked!
"Agencies that take that angle need to be slapped very hard indeed"
- Couldn't agree more!
Hi Aleyda,
Thanks for this post, I am late to the party but it's a good guide for doing a good SEO migration of websites. This can very well be converted into a checklist to make sure you don't lose any traffic due to migration.
Regards,
Vijay
Danny, - We deal with stuff like that all the time. It is no fun cleaning up someone else's mess.
Aleyda, - Love detailed posts like this, thanks! You nailed the exact process for sure.
Great post Aleyda!
I have seen some domain migrations and they all seem to always have their issues.
I love this: "Update all of your internal links to go to the final URL destinations directly" because it is so easy yet people always forget it. Often my team will find these broken links and want to redirect them when I tell them, why not just fix the link if this is the only linking trace to that page?
I saw a website that didn't go through a website migration but changed the structure of their URLs. Two of the main page's (with back-links and ranking well) structure was changed and no redirects were made, and so the website lost 50% of there traffic!
Hi Aleyda, Great Post. We went through a large migration and one of the challenge we had was Screaming frog doesnt handle large dynamic sites well. Do you have any experience with any other crawling tools that can handle a dynamic site with 10mm plus pages? Any guidance is appreciated.
Hi Aleyda
Who he said migrating a website was simple? It is customary to miss traffic (more if not done right ...)
But with the steps you indicate perfectly we can return to gradually recover all that volume of organic searches.
Congratulations for the post !!
Thanks Ivan :)
Really Great Post Aleyda ! and migrating a website is not a easy task. most of folks think that we can migrate a website through 301 redirect, but if we not done proper 301 redirect then we loss our traffic as you said.
So your suggestion is really helpful for mine and also others.You really explain deeply about migration of website without losing traffic.
I want to say thanks for sharing such kind of ideas.
Glad is helpful Dilip!
Thanks, Aleyda! It’s great when we talk about the simplest case- clean URL migration when the platform stays the same, same code, same set-up.
However, it becomes more complicated when the URLs migration is dictated by the need of merging with other websites or for example upgrading the platform or when moving to a completely different code. Those changes may affect SEO performance enormously even though the basic website search optimisation is well carried. In such cases, the thorough analysis of page templates and their content structure before and after migration is crucial; also the analysis of the quality of the html code is important - it may not only strongly influence the way Google reads pages, but may also influence factors like page load speed etc.
Great post!
You mentioned "Remember that if the migration required drastic changes (like if you've migrated over another domain, for example), it's natural to see a short-term rankings and traffic loss." at the end of the post.
What do you view as short term? 3-4 weeks? or 2-3 months?
Thanks!
It is urgent to carry out at all costs organic search traffic from and what you plan to have an outlet prevent bad web .
Does anyone have an idea how long recovery can take if all things are done properly? There is a loss of traffic associated, I just wonder if the short term is 3-4 weeks or if on average it is a longer period.
nice post Aledya, I had major issues with a wordpress after a forced migration to a new host site last month that caused a high proportion of 404 errors broken links and missing images etc, luckily i was already familar with the screaming frog tool, and I installed a plugin to handle 301s. It was still time consuming to resolve all the issues :-(
Nice Post Aleyda. Transfer a website (especially the big one )from one to another server is very risky in terms of SEO. If anybody don't follow major terms then their ranking will decrease in no time. So, It is very important to choose a web host wisely.
It shocks me that some web agencies still don't understand how to migrate an old website structure to a new structure.
I deal with this type of stuff ever so often when businesses have lost rankings due to their web designer not having a clue about basic SEO techniques never mind retaining rankings during a migration!
Thank you for this video and article. I learned alot!
Nice article have made this mistake myself. Any recommendations when changing to ssl?
Another great post Aleyda, thank you so much for the comprehensive guide having all the pointers you need to check about a site migration.
Great post as always!!
On Page.org's is an excellent tool for this type of analysis, thank you very much.
Thanks for the information, this can save big headaches in the future
Thanks for this. Really interesting.
I faced 2 years ago a case in which I was asked to help a friend who had migrated website. Their traffic dropped drastically. They did not follow any of the items you indicate so no wonder....
A vey interesting learning from that was related to pages with not huge traffic, but very important traffic. Sales people were sending out links to pages with pdfs that contained technical details... After the 1st round of redirect worked went through these pages were not considered due to low volume of traffic. But they were really important!
Though an SEO at a search engine machinery-level requires a website in its regard, but then the decision towards the complete shift from the former to the latter, on a sole basis afterward, taking a decision regarding it is a tough one. Since, a company which has gained a good traffic at the former level, for its prospective traffic conversion into business at the latter stage, can lose out on what it had earned primarily. Therefore, the final say in this regard rests with an individual or company whose SEO it is basically, whether one is ready to forego the positives of the first step, for the unsure, would be chances are could be or could be not, profits which it may receive at the second step?
Thanks Aleyda! How far back is too far? We had this problem 2 years ago and I figured that since those urls didn't get traffic anymore it was not worth worrying about. Am I wrong? Thanks!
This is an awesome post for so many reasons, and the biggest one for me is your breakdown of how to read the data via screaming frog and GA to identify where and why the drop happened. A migration is something that most of us just have to deal with at some point and time and knowing not only the ramifications but what to look for is key. I really like how you mentioned finding the keywords that dropped during the migration time and monitoring them, but not to expect an immediate recovery. Information like this is what a client needs to hear or they think a few redirects and boom all is well again.
Great job putting this together and who knows maybe Moz will come up with a migration tool as well.
Thanks Tim! Glad to hear you liked it and found it actionable :)
Good morning Aleyda.
When we do a migration, pages with a low number of visits. Are we migrate or redesign?
Hi there! A migration is a migration if you're changing the Web structure whether is a small or big site :)
Dear Aleyda,
Thank you for such a nice post. This particular case has never happened to me or any of the clients that I've had, but in case I'd happen to me soon, I know this post is gonna be really helpful when the time arrives! it's great to get this help in moments of crisis, specially if you're a newbie in the scene.
Yes, it is sadly a not so uncommon situation! I hope this helps :)
Thanks, for such informational content. It will surely help a lot.
Hi Aleyda, thanks for sharing the epic guide! Site migration related SEO issues are scary and one should not just migrate a site without consulting an SEO. I have migrated a bunch of sites in the past several months and thankfully, they haven't see any drop in traffic.
Great !
One of our main client is migrating his website from 1 domain to another, the client have specifically mentioned that he doesn't want even 1 visit to go down due to this migration, hence we gave him migration plan with note saying incase if their is drop in traffic we will fix it in month or so..I think this post is somewhat perfect incase it happens (thou I am sure it will not)
Thanks Aleyda
Regards
Pulkit Thakur
Awesome article! We also use screaming frog and it is a tool that I don't think enough people know about yet.
A great article. Pity I had not these tips when I migrated I manage the web. Those two months I lost many positions and also errors were flooded. 3 months later remedied.
Thank you Aleyda for that epic post, I have never read something better about that topic. From time to time I am also faced with that kind of problem and it feels like being a Fire Dept. instead of an analytic seo company. At least it is good for clients to find help. I totally agree with Danny.I have worked for several agencies in the past as technical SEO (but only if they realized that they need one). Lots of agencies are scared to hire another agency because they fear to lose the client. Those agencies have those fears right away because no client should chose them without being informed that they need a seo specialist as well.
@Ankita Sharma - You have to understand that dear .
Thanks, Aleyda! Love your post , waiting for next one