How To Maintain Google Rankings and Traffic During a Site Upgrade or Redesign

Site redesigns can be both a fun and daunting experience.

Fun in the sense that it’s often an enjoyable experience from a design point of view. That being of course, looking at new colours, new layouts, styles and perhaps even adding new features or elements to make your website perform better or look more visually appealing.

Daunting in the sense that if you get it wrong, you could end up ruining everything.

When it comes to redesigning your site, there’s an awful lot to consider, especially for larger, well established sites that receive a lot of traffic from Google. Let’s face it, the last thing you want to do is lose your rankings along with all of your traffic.

This is where the thought of changing anything can raise concerns such as –

  • If I upgrade my website, will I lose my rankings in Google?
  • We’re first page position 1 for the keyword “XYZ” in Google, if we upgrade our website, will we lose that spot?
  • How can I upgrade my website and NOT lose all of our rankings and Google traffic?

These are all great questions, and certainly concerns that any site owner should have ahead of performing a major site redesign.

What happens when you get it wrong

Take a look at the graphic below. This is a screenshot of a site redesign that went terribly wrong.

Bit worrying isn’t it?

That’s close to 6,000 visitors per day down to nothing. Ouch.

So what can cause this to happen?

Typically a huge drop in traffic like this can be the result of –

  • An incorrectly set robots.txt file
  • It could be a buggy htaccess file
  • It could be an incorrectly set DNS entry
  • An incorrect setting within a WordPress plugin, OR
  • …it could be the simple fact that whoever built and launched the new site did absolutely no planning whatsoever

In most cases however, the main reason I see sites tank like this, is simply because 301 redirects weren’t implemented.

Sidenote – If you’re just changing the design of your website, and the actual site structure (along with all page names etc) aren’t being changed, then there’s no need to implement 301 redirects, or follow any of the advice provided below. However if you are making considerable changes to your website and everything is changing including page names and your site structure, then it is absolutely imperative that you follow the advice I’ve outlined in this article.

What are 301 redirects?

Google provides the following explanation

If you need to change the URL of a page as it is shown in search engine results, we recommend that you use a server-side 301 redirect. This is the best way to ensure that users and search engines are directed to the correct page. The 301 status code means that a page has permanently moved to a new location.

In other words, you’re telling Google, “Hey, this page no longer lives here anymore, here is it’s new location”

This is important because if anyone clicks on your page in the search results, and you’ve just gone about changing your entire website – chances are they’re going to land on a “404 page not found” error page and leave. That’s not good.

So by implementing 301 redirects, we are both preserving our traffic, and ensuring we take every possible precaution in an effort to NOT lose our rankings.

Phase 1 – New site build considerations

Understand exactly WHY you’ve decided to rebuild your site in the first place

I see a lot of site owners decide they want to upgrade or rebuild their websites for the wrong reasons. I’ve heard –

  • We just want a change, we’re bored with it
  • Our competitors site looks much nicer
  • We don’t like the blue

None of these reasons really mean much. Instead, as a site owner, you should be pushing ahead with a site redesign for some of the following reasons –

  • Our site doesn’t work on mobile, we need to fix that
  • Our site isn’t optimised for search engines
  • Our hosting is awful, we need to move and we may as well fix our site too as its very old
  • Our site is extremely user unfriendly, and its costing us in revenue

In other words, make sure you’re changing your site for the right reasons.

SEO before “web design”

When it comes to building your new site, your SEO guy (or agency) should be directing the build – NOT your web guy. This is an all too common mistake. A web developer builds out a site that looks absolutely brilliant, then when its looked at by an SEO company, they’ll say, “This things a mess, you need to rebuild your site”.

Always be sure to put SEO ahead of web design (aesthetics) Most important of all is your site structure. Your site structure is critical, and should be based around a) solid keyword research b) reverse engineering your competitors.

Make changes to your site inline with what your data tells you

Always have a good understanding of how your site is performing before you go ahead with a site rebuild. By this I mean,  You should know –

  • How much traffic you’re getting
  • How much the site is generating in monthly revenue
  • What your conversion rates are
  • The cost of customer aquisition
  • Overall search positioning, and
  • Other metrics of interest such as bounce rates, time on site, return visits, abandonment rates etc

You’ll want to know your numbers before you start pulling everything apart.

Take a look at Google Analytics, assess the data, take notes. Consider gathering additional data using Crazy Egg or Clicktale to get a better idea of which design elements are working and which aren’t. The more information you have, the easier it will be to upgrade the website, because you will know exactly what’s working, what’s broken and what needs to be improved upon.

Always make sure the rebuild means making an improvement on your sites performance. Don’t go backwards. There’s little point making your site “look pretty” if it kills your traffic, conversion rates or engagement times.

Your existing website should NEVER be offline

One of the most important parts of any site upgrade or redesign, is to ensure that you minimize any potential outages or downtime. Of course there are bound to be moments during the changeover that the site might be inaccessible, but that’s to be expected. However, your site should not be down for any lengthy periods of time that could result in a loss of business. The last thing you want to do is to have your site down for a week while you “finalise a few design elements”. Oh, and never ever take the site down and use a “website under construction” splash page – especially if you have well established rankings and traffic.

Build your new site in a live test environment

The most effective and efficient ways of building out a new site, without taking down your existing site, is to build it within a test environment. What works really well is to just register any old domain name – something like – xyztestwebsites.com.au or something – and then go about building your new site there. You may even want to consider separate hosting as well. Infact I have done just this. I have a dummy domain setup and use Digital Pacific to host my test sites, while we build them for clients.

The advantages of building your new site in a test environment include –

  • No disruptions to your business
  • No outages or downtime to your existing website
  • You can test features and design elements “live” and see how they perform/look
  • You can have the new site 100% ready to go, before you flip the switch
  • You’re not pushed to crazy deadlines and getting stressed about “getting the site up” because you’re losing business

Set a “block all” robots.txt file during the development of the new site

This is REALLY important. Especially if the content on the new site is EXACTLY the same as the old site. You DO NOT want Google crawling two instances of your site across two different domains. If this happens, Google may decide to drop one or both of the domains from its index, because it might flag your site with a “duplicate” content penalty.

To do this, you’ll want to set the following snippet of code in your robots.txt file

User-agent: *
Disallow: /

The “User-agent: *” means this section applies to all robots. The “Disallow: /” tells the robot that it should not visit any pages on the site.

For more information on robots.txt files, and how to set them up, go here.

Be clear about the desired outcome

Lastly, have a good idea of what you’re trying to achieve as part of the outcome. Is it to increase conversion rates? Is it to better emphasize your call to action? Is it to make your site responsive? Or is it just to “freshen up the sites outdated look”. How you measure your efforts should tie in with what your initial objections were.

Phase 2 – Preparing the 301 redirects

Okay, by now the new site should be completed, tested and you’re getting ready to go live with the new site. From here, you’ll need to start preparing your 301’s.

For this process, I use the following tools and software –

To begin with, we will want to prepare a master spreadsheet. This will help us prepare our 301’s for the changeover. The purpose behind this spreadsheet is simple – we want to be able to go through our URLs one at a time, be able to see them visually, (double check them) and ensure we have them set correctly before we launch.

Download this one I prepared for you. It will save you some time. When you open it, it should look like this.

What do the columns mean?

  • Redirect – Don’t worry about this for now, that’s just the syntax that’s required as part of the command. We’ll come back to this later
  • Old URL – The URL of the existing website (the old one)
  • New URL – The URL of the new site (the one that is in the test environment)
  • Transitional URL – This is where we will be pulling data from, into the new URL (you’ll see in just a moment)
  • Notes – For making annotations where necessary

Ok, so now in order to fill this spreadsheet in, we will need a complete snapshot of every single URL on both the old site, and the new site. The fastest and most efficient way of collecting this data is to run it through Screaming Frog.

To do this, we just enter in the web address of the old site (and the new site) and click on START. Screaming Frog will then scan our site and return all of the sites URL’s as follows.

Once Screaming Frog has finished scanning the entire site, we can then EXPORT the list of URLs, as follows.

This then allows us to view our listing of URLs in one nice clean spreadsheet like this.

From here it’s a no brainer. Just copy the first column that has all the URLs in it from this spreadsheet and put it into the master spreadsheet.

Be sure to put the old URLs under old URLs and put the new URLs under ‘transitional URLs’.

Once you’ve done that, your master spreadsheet should look like this. You’ll have the old URLs, the new URLs (which will be blank at this stage) and the ‘transitional URLs’

Lastly, you’ll want to go through your spreadsheet and copy the transitional URLs over to the new URLs column WHERE they match. This means, you’ve looked at the old URL and matched it against where the new page now lives.

Okay from here, we will want to focus our attention just on the old URLs and the new URLs. Before going any further its important that you understand how to format your redirect. Have a look below.

The correct way to format our request is as follows –

Redirect 301 /category_108/Wedding-Cermony.htm http://favourperfect.com.au/category/weddings/wedding-cermony/

What’s happening here?

  • Firstly, the redirect command which is “Redirect 301”. This tells Google it’s a permanent redirect.
  • Secondly, the old URL. You can see in this case, the old URL has the domain stripped out. So we’re actually redirecting http://favourperfect.com.au/category_108/Wedding-Cermony.htm, but it’s displayed only as /category_108/Wedding-Cermony.htm
  • Lastly, the full path of the new URL, which is http://favourperfect.com.au/category/weddings/wedding-cermony/

Got it? Great, let’s move on.

The easiest way to then make this change to our spreadsheet is to highlight the old URL column and do a “find and replace”, where we will remove the domain.

Once we’ve stripped out the domain name from that column, we should be left with something that looks like this.

Now the fun part.

Copy all of the data out of your spreadsheet and paste it straight into a notepad file.

Your notepad file should look like this.

In order to ensure we don’t have any “funky” spacing or irregularities in our text, we’ll head over to Text mechanic to clean it up. Text mechanic is a great online tool that will help us ensure our spacing is correct.

Copy the text file and paste it into this tool (remove extra spaces)

You should now see your 301 redirects all tidied up without any additional spacing. Copy the output and paste it back into your notepad file and SAVE it AS A CSV FILE. This now completes the preparation of your 301 redirects.

Phase 4 – Launching the new site

Scheduling the changeover

Scheduling the changeover is vital. Never ever, try and change a site during a busy time. Always, always schedule the changeover during a Saturday night or Sunday lunchtime or a time where the site is at it’s least busiest. For most of my clients, I perform the changeover on a Friday or Saturday around midnight. Doing this will ensure that you minimise any potential disruptions to normal business operations.

Having everyone on standby

Something else to be mindful is to ensure that you have everyone on standby while the change is being performed. This might include the site owner, any IT staff, and of course your team and anyone else that may be needed to assist. Typically if something is going to go wrong, it will go wrong when you least expect it, so have a few people around to give you a hand. Plus it’s good to have a few sets of eyeballs looking at things as you go. You will want to have others around to help you with initial testing.

Don’t forget email

Email is usually always something that is overlooked during a site upgrade. This may not be something you’ll need to think about, but if you’re changing hosts as part of the upgrade, then chances are, you’ll need to check to make sure your email is working.

Preparing for rollback

Always pull the old site down and back it up. Always ensure that you can (if necessary) rollback. That is, put the old site back in place, should anything go wrong. Don’t ever just delete the old site and think everything is going to go smoothly. Infact, you should keep a backup of your old site indefinitely, anyway.

Speed of changeover

I remember someone asking me once, “Should we just do small bits and pieces of the changeover, or should we do the whole lot in one hit?”

The answer to this is, get the changeover done as quickly as possible in one hit. You don’t want to cause any level of confusion for Google or your visitors as to whats going on. You want the process to be fast and clean. Pull the old site down, get the new site in place, then implement your 301 redirects and test.

Implementing the redirects

As soon as the new site is up, you’ll want to activate your 301 redirects immediately. Take your 301 redirect file and do the following.

Log into the new site, go down to SEO (Yoast plugin) and select ‘Tools’

Then select ‘File Editor’

Then paste your 301’s in here.

WARNING – MAKE SURE YOU BACK UP YOUR HTACCESS FILE BEFORE YOU CHANGE ANYTHING IN HERE. IF YOU GET THIS WRONG, YOU COULD SCREW UP THE ENTIRE SITE. To back it up, just copy the contents of what’s in this text area and save it to a notepad file. Then paste in your 301 redirects as I have shown below.

It should then look something like this.

Phase 5 – Site monitoring and ongoing refinements

Okay, so by now the new site is up, the 301’s are in place and everything appears to be going well. So what should we be looking at to ensure nothing is broken? Here’s a few things you should be keeping an eye on and taking care of.

Implement tracking codes

Be sure not to forget to get your tracking codes in place on the new site. This might include Google Analytics, and other software that you use to monitor site performance. You might also want to think about setting up or checking your goals (conversion tracking)

Test in Google directly

This is something that I do, because I am VERY particular about testing and ensuring that everything is RIGHT. I don’t care if a site has 10 pages, or 5,000, I spend the time to do this because you’ll want to find out about it now, and have the opportunity to fix it, rather than be kicking yourself 3 months later for NOT fixing it.

Go to Google and type in site:domainname.com.au

Of course change “domainname.com.au” with the URL of the website you’ve just upgraded. You should get a complete listing of all the sites indexed pages in Google. Go through each one of those URLs, click on them and make sure you are successfully redirected to the new page. If you’re not, then double check your redirects – there is obviously a problem.

Keep an eye on Webmaster tools

Under “Search Analytics”, you’ll want to keep an eye on both “avg position” and “impressions”. Look for any noticeable “dips” and take corrective measures where necessary.

Conclusion

When you change a site, you should always take precautions in order to prevent any losses in traffic or rankings. However, there will always be the potential for fluctuations. Infact, fluctuations are normal when upgrading or revamping a site. That’s pretty common. This happens because Google is going through your site and trying to “reorganise its index” to reflect the changes. The time it takes for this process to complete varies depending upon a) the size of your site, and b) the complexity and nature of your site.

For most sites that I work on, the process is usually completed within 2-3 weeks, sometimes sooner – especially for smaller sites.

But note ….

If you launch a site, and things go bad, you have just a few days to salvage things. Once the site falls off a cliff and goes beyond a certain point, then the damage is done. You’ve lost traffic, you’ve lost rankings – and it will take some effort to get that back. This is why it is SO important to plan out the redesign ahead of making the change.

As always if you’ve found this information useful I would really appreciate some social love. Please leave any comments you might have below, and I’ll respond personally.

2 thoughts on “How To Maintain Google Rankings and Traffic During a Site Upgrade or Redesign”

  1. Very nice and in-depth description for someone like me who had been wondering for some time how to handle this. Thank you for that. However, I am left with one specific question: Do I have to remove the redirects after a certain time or do I have to leave them in place forever?

Leave a Comment

Your email address will not be published. Required fields are marked *