Introduction
Handling expired content can be an overwhelming experience for any SEO in charge of a dynamic website, whether it be an e-commerce, a classified (example: job search, real estate listings), or a seasonal/promotional (example: New York Fashion Week) site. Even something as fundamental as glancing at the Google Webmaster Tools account for the site can evoke gut-wrenching emotions, especially if the site has amassed tens of thousands of 404 errors. How are you supposed to come up with a process to manage this? What should the process even look like?
What Qualifies as Expired Content?
There are a number of examples that would be considered "expired" content. Expired content is content on a website that is only relevant for a limited period of time. Below are examples of different scenarios that would need to be considered expired content.
Job Search/Real Estate Listings: Job listings routinely expire, especially when positions become filled. The same is true for real estate when property is sold.
- What is the best way to handle expired listings, especially if the content is only available for a very limited amount of time?
E-commerce: Expired products can occur when products that are sold on the site routinely change for one reason or another, such as:
- What happens when the site no longer sells a product?
- What happens if the product becomes temporarily out-of-stock?
- What about seasonal products that are only sold during limited times of the year?
Perhaps most importantly, sites that have to worry about expired content tend to be enormous - often comprised of hundreds of thousands of pages. Thus, recommendations need to be manageable and clear. Taking an individual look at all expired and out-of-stock products is unrealistic. Start thinking, is there a way we can build a process for these type of changes?
The Options
Like most SEO solutions, there isn't necessarily one right answer. We need to take a look at each individual situation on a case-by-case basis and take into consideration the current back-end of the site, as well as the resources and the technological capabilities of the site's team. There is a time and a place to use each of these options for expired content. Identifying the right scenario for each situation is very powerful.
I. The 404 Error
It makes sense for webmasters to think that 404ing expired content on the site is the approach to take. After all, isn't that the very definition of a 404 page?
(Distilled's 404 Page)
In most situations, a page on the site should not be 404ed. Why?
Disadvantages of 404 pages
404ing pages that used to be live on the site is just not beneficial for SEO because it alerts search engines that there are errors on the site. Essentially, you're wasting the site's crawl allowance on crawling/indexing pages that no longer exist.
Also another issue with 404 pages is that they tend to bounce - users land on the page, see that the page no longer exists, and quickly leave. Users are vital to the site and our goal as SEOs is to not only ensure that the site gains organic traffic, but that the users stay, browse through the site, and ultimately, convert.
Custom 404 Page
If you must 404 pages for one reason or another, consider creating a custom 404 page, so that in the chance that a visitor lands on the page, there is an opportunity for them to convert. A custom 404 page can also include keyword-rich links to other pages on the site (for instance: see Crate and Barrel's 404 page).
Determining the Right Approach for Expired Content
Now that we know the disadvantages of 404ing pages, what is the right approach in dealing with expired content? To determine this, multiple considerations need to be taken into account, such as:
- Was there significant traffic (and not just organic, but also consider direct) coming to this page?
- How can we provide the best user experience?
- Has this page received external links? How is this page currently internally linked to?
- Is there content/resources on the page that users would still find useful?
II. The 301 Permanent Redirect
Advantages of 301 Redirects
For the vast majority of scenarios, I'd suggest 301 redirecting your expired content to another page. This is usually the best option for SEO and can also be customized to enhance the user experience via dynamically-generated messages. For instance, if a product page had garnered external links, you're able to retain most of the link equity from those links via a 301 redirect (whereas with a 404, that link equity is lost). Why would you want to lose the link equity that you had worked so hard to obtain? Furthermore, it demonstrates to search engines that your site is well-maintained and up-to-date or "fresh".
(screenshot of infographic from Dr. Pete's epic status code post)
Where should you 301 redirect these pages?
Consider what would result in the best user experience. You want to redirect these pages to the most relevant page. A suggestion is to take a look at the breadcrumbs and redirect the page based on the internal navigation of the site. For instance, the product page can be redirected to the most relevant sub-category page. You want to be careful that you're redirecting the page to another page that is likely to stay on the site in the foreseeable future, otherwise you run the risk of having to deal with this issue again (not to mention that having a 301 redirect lead to another 301 redirect to another 301 redirect is not considered good SEO practice). A safe bet is to redirect these pages to the most relevant category page, as these are pages on the site that are least likely to change.
Dynamically-Generated Messages
You can customize and improve the user experience by implementing a dynamically-generated message via cookies during a 301 redirect. This would result in users who have landed on expired products receiving a message letting them know that the original product they were seeking is no longer available. This enhances the user experience because it informs users on why they are being redirected.
Disadvantages of 301 Redirects
For some sites, implementing multiple 301 redirects might affect server performance (though for a well-designed site, this should not be an issue). However, if it is true for your site, knowing that site speed is a search engine ranking factor, we want to be wary of the impact we may have by implementing this strategy. If this is the case for your site, consider only 301 redirecting the pages that have gained external links or have received significant amounts of traffic and directing the remaining pages to a customized 404 page. Please bear in mind that this is not an ideal scenario and is just a workaround.
III. Leave the Page on the Site
Advantages of Leaving the Page As Is
Sometimes product pages still garner significant amounts of traffic or are rich in unique content and contain information that is still useful to visitors. It would be worth leaving the original product there, especially if the page has unique, high-quality, evergreen content, but have a message that the product has been discontinued. This will likely provide the best user experience as it will provide a strong call-to-action.
How Could You Set Up the Message?
Implement a JS overlay that would include similar products as the one that has been discontinued and drive users to those new products. Consider incorporating keyword-targeted internal links to drive traffic to those sites. This provides a positive user experience and is especially important for repeat customers.
Example: Real Estate
For this niche, expired listings bring tons of traffic since people are curious about what has been sold and what the market looks like. Thus, consider leaving these pages on the site, but include additional information on the top of the page, such as "contact us to see similar listings" or "here are some other houses in the area that have similar selling prices."
Disadvantages of Leaving the Page As Is
You want to be wary of the practice of leaving old pages, especially if they aren't enhancing the value of the site. Why? Because this will require more bandwidth from search engine bots to crawl your site as you continue to add new product pages to the site. You don't want to risk wasting your crawl allowance having bots crawl pages that are thin in unique content and value. Also, having search engines crawl such pages indicates to them that the site is not "fresh."
Also often times, new products contain the same content as an older variation of the product. For instance, the names of new products may vary only slightly to their previous version and the product description can be a close duplicate. Having all these pages live on the site can result in massive duplicate content issues.
How to Deal with Out-of-Stock Products
If a product is out-of-stock and is expected to be restocked, the page should remain on the site, but an out-of-stock notice should be implemented on the page. However, please bear in mind that out-of-stock pages do tend to generate high bounce rates. To confront high bounce rates issues and improve the overall user experience, consider ensuring that users know similar products are still sold on the site or have users sign-up to be notified when the product becomes available again.
How to Deal with Seasonal Products - at the Category/Sub-Category Level
If a product is seasonal, such as the case for fashion products (example: swimsuits), you might want to leave the page on the site permanently. Why? Because overtime, these pages can retain their link equity year-after-year. If the swimsuit page garnered 3 links this year and 5 links the next, you can continue to accumulate those links. Overtime, you've developed a page that has retained a significant amount of link equity making it much more difficult for competitors to keep up. Thus, giving your site a huge advantage.
And if you don't want the page to be indexed in the off-season, add a meta tag to noindex/follow the page. Users will no longer be able to get to that link from search results (and hopefully from internal results as well), but only through direct links or bookmarks. Once the season starts again, remove the noindex/follow meta tag to an index/follow.
Building Processes/Checklists
Based on the specific needs of your site, it would be helpful to develop a checklist for your technical team. For example, if my site had seasonal products, I would compile a checklist that would include the following:
- Remove noindex/follow tag from the [products] page in [month]
- Update and resubmit XML site map
- Submit this page to "Fetch as Googlebot" in Google Webmaster Tools
Consider creating separate checklists for the steps that you, as an SEO, would need to take to determine which pages to 301 redirect, which ones you need to 404 (if you absolutely need to), and which ones to leave as is. Checklists should also be created to help develop the framework for how your technical team would implement these changes. After awhile, an overall framework should emerge on how your site handles its expired content, which will help make the entire process run much more smoothly.
Nice post. As an e-commerce site this is something I deal with on almost a weekly basis. I usually use a 301 to handle discontinued/expired products. If the product is just out of stock for an extended period of time then I will keep the page live and add an Out of Stock message. My Add to Cart button changes to and Out of Stock button and the item cannot be added to the cart at all. When it is back in stock I just change the button back to Add to Cart...done.
I try very, very hard to not use a 404.
For me my challenge lately has been what to do with a category/sub-category page when I no longer have products but that category received good traffic and it had good, relevant content. If it is a sub-category page then I will review the content and work to implement it, if I can, at the category level and then 301 the sub-category to the category.
However, if it is a top-level category page then I will modify the page away from a traditional e-commerce category page (with a grid of products) to a content-only page (e.g., blog post, article) about the category. I feel like someone might still value the content I have and I will work in related products that are outside the category to work to keep the visitor on the site and to explore other areas of the site.
I prefer this over a 301 to my home page. When I use a 301, I try to bring my visitor to a page that is related to what they expected. If a person clicks on a link and they are thinking they will go to a category page with products and then just get redirected to the home page that is not a great experience in my opinion. At least if I keep the content for my previous category page I feel like this is a better "user experience" for my visitor.
Cajohnson, I agree with your solution. I think this is the best possible solution for all :-)
Hi Stephanie, great post and - especially - an actionable one.
But even though your post is perfect especially for eCommerce sites, I think that something should be told also about news sites, which - obviously - tend to have gazillions outdated content. And maybe this should need a different post.
But, said that, in those cases one option I usually consider is the opportunity of reformatting those outdated contents (news, articles, posts, infographics, specials...) in better organized on topics Archive sections. Then, 301 the old URLs to the new topical section. For instance, let say an online magazine every year create contents related to the Cannes Movie Festival and that this content is originally distributed under different sections of the magazine (People, Entertainment, Culture...). Once the event is over, I would simply create a section following a procedure like this: Archive > Culture > Cannes Movie Festival > 2012 Edition (obviously this is an idea that can be bettered :)).
I found this methodology also a better and more (SEO/Content Marketing) effective way to deal with outdated content, which can still offer huge opportunities in terms of link building, referrals and so on.
Thanks Gianluca- that means a lot!
Definitely agree, for my publishing client, they did not redirect any of their old content, since it is unique and great content. Instead, they archived these articles by month/year of publication. I believe there have been a couple cases of publications bringing back old articles onto the homepage and those older articles generating even more links (especially surrounding historical events or an article with a great high-resolution photo). Might be a great linkbuilding strategy!
I personally believe in leaving the content on the site forever if possible, with a note on it that details a useful alternative on the site - this could be dynamically created based on the topic of the page - or just a set list of links to the various sections of the site.
Very insightful material, Stephanie. I wasn't expecting such an in-depth post, but am pleasantly surprised at the sheer amount of information given here. I'm not yet experiencing these types of issues because I am pretty OCD about keeping everything as neat and tidy as possible, but I also know the headache this would all be if there weren't articles like this to help me along the way!
I feel concerned by the topic of this article since I'm doing seo mostly for ecommerce brands.
The big challenge is really the speed of "corrections". In most cases we have to deal with a technical team with devs who are already overloaded by tons and tons of work.
So i found the last part of the post (about the porcess) very relevant here : create a process to deal with old content requires continuous improvement and what we try to do is to get everyone involve : So every friday i send a mail to the technical team with all the issue of the site and their potential impact on rankings/traffic and sales.
I think you're right - you need to decide what that process will be (and why). And it might be very different for each site, depending on the type of old content.
Hi Stephanie,
Great post.I have a question regarding a deleted page. In case a product is sold and the page has been deleted. But it is still in Google's index and it shows a 404 error. Would it be a good idea to add a no index/follow tag so that it tells the bot to crawls it but not to index it and once it is removed from google's index than add the no follow tag.
thanks
Really great post and wonderful information! This post couldn't have come at a better time for me as well! I'm working with a client looking to delete probably 2-3K products from the site because they are old and not driving towards the new direction of the brand. I'm encouraging them to first look at the traffic each product page still receives, and the backlinks to each of the pages before recommending next steps for each. The product pages themselves do not have that great of content - the pages are actually very thin on content what-so-ever and mostly flash. Because of this, I'm recommending permanently redirecting the content with links to closely related content (either a very related product or the category page the product lived under). The products without links (the vast majority) I was going to recommend the custom 404 - but now I'm thinking about the 410 from the comments above ('this content used to exist but is never coming back').
Really helpful - thanks again and great work!
For pages that wont return why not send the appropriate header of 410 - "Gone"?
https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#410
"Upon receiving a 410 status code, the client should not request the resource again in the future"
Not sure of the SEO value of a 410 status code - how would you retain link equity? what if the page receives tons of traffic?
I implement 410 code some months ago in a old site where thousand of users wasn't active anymore. So, after a database cleanup we add a 410 to no longer active profile page. They had some traffic, but only from search. So I hope they will be replaced by active and more recent profile pages.
Ok, I'm going to reference Ian's comment above:
"Only time a 404 is good for SEO is if content is remaining indexed and you really, really want it removed forever. For example, if you've got spammy content on your site, or had a complete change in your product line and can't productively redirect."
Ian's very right, but a 410 might be better for the spam content scenario - the advantage being that Google will usually stop telling you it can't find the page in WMT, and also stop trying to crawl it. If the page or content never existed, or has moved somewhere uncertain, 404 is a more appropriate response.
Awesome post on the topic.
301 redirects will be better given that you have close related stuff to that link, or may be redirecting to homepage can be a good option, will keep the juice flow internally.
On the other hand, a custom 404 design is a good idea as well, if the 404 page is well designed and well managed and proper internal linking is done then in that case it can reduce the bounce rate as well.
Very nice article, Stephanie!
10 instances of 'Link Equity', 1 instance of 'link juice' - I think Mike King is getting his way :)
I would pick the top "linked to" expired pages and set up URL mapping to 301 them to relevant pages which are still live. Some times it is hard to re direct 20k expired pages on a site, you can set up rules for 301's, you can 404 the lot. Overall it usually it ends up with discussions with the dev team and every one has to decide on an actionable direction to take with the project.
Definitely have to loop in the dev team. Agree with the top "linked to" pages, so long as the expired page is being redirected to a page that makes sense from a user perpsective.
So glad that I found this informative post.
I have a classified advertising website, people can register and add their own advert on a page. I have some free advert for a year, say a car or other item.
Every advert on my site gets at least 30 to 100 monthly hits, even old ones. Do I remove these old adverts or not as it clutters the site if there are too many. For adverts that I have added myself it is no problem as I use them for new material or adverts.
After reading this post it seems I should leave as much material on my site as possible as they are all getting page hits.
Great work Stephanie! Out of all the stuff i've read out there, you definitely inspire the most confidence. I have one quesiton, that I think I know the answer too but am in denial about. For obsolete product pages that I keep on my website, is it important to have good page titles? (ie the ones that are in all caps). (hoping not).
I have an automotive classified ad site and my solution over years has been for the sold cars to be examined for continued traffic and quality links. If the sold URLs are determined to bring low traffic or if they lack good links I will 404 the pages. This can be a very painstaking process so I'm thinking of doing a 301 to a higher category for all of my sold content. I know it will carry the link juice but my fear is that I have hundreds of expired pages that rank well that i will lose traffic on. Will the 301 SEO benefit to a new category override the traffic loss of well ranking but expired pages?
Hi Stephanie Chang,
Google have different suggestion about not found errors. You can read one overview sentence to know more about it.
Generally, 404 errors don’t impact your site’s ranking in Google, and you can safely ignore them.
I was not dealing with 404 errors behalf of this Google's official suggestion. But, you have defined few factors which inspired me to drill down more on this section.
You can get more ideas by following Flickr photo.
https://www.flickr.com/photos/commercepundit/8070125347/in/photostream
I am going to fix 404 errors via 301 redirect. I will redirect discontinue products to associated category pages. Do you think, It will help us to improve my Crawl Stat. Because, I have set custom crawl stat via Google webmaster tools. And, it's creating traffic issue on server.
Will Google consider healthy website as a fresh website? Because, I am focusing more to add unique and fresh product description on product pages rather than fixing 404 errors.
By the way, this blog post make my day and thanks for sharing such a fabulous information!
How can I 301 redirect via PHP programming or JavaScript?
Hi Everyone:
I have read through all of this but it doesn't answer my specific situation (though someone did mention coupons above) so I am hoping you guys can give me some guidance.
The site I am working is a monitized site which basically is coupons and special deals. The average blogger in this niche can and does post 2k - 4k posts within 3-4 months, which most, if not all, are useless after the deal is over. I'm talking Kroger, CVS, Walmart, Amazon... small ticket items; short window for the deal. I can't see a reason to keep content that is useless and outdated and comprises thousands of posts per year. It just seems crazy to me. Using a 404 or a 301 on thousands of old posts seems like clutter for the server, search engines, the db...everything. The 404 is especially concerning to me...a visitor hitting a 404 several times is not a good thing.
My question is about expiring the posts and the impact on SEO for this business model. My belief is that the site owner/blogger should put most of the effort on the static pages for SEO and expire the posts with the deals when the deal expires. It seems logical to me and of course the deal posts would pay attention to SEO, but wouldn't count on that to be "the" SEO for the site.
I know it's a debatable issue :-) Any thoughts?
Thanks,
Terri
Thank you for the great pros and cons of the different options. I am running a coupon site in WordPress and am struggling with the best way to handle the expired content. It sounds like the best option is to balance a 404 custom page and 301 redirects to balance traffic. I will have to experiment and see what works best.
Hi Stephanie,
I've been doing seo for 10+ years and just made a mistake I've never made and not sure how to handle it :) While developing a site I accidently removed the no-crawl and was not finished with the site. Some of the pages I was working on had content that spanned into multiple pages so there are ...page1.htm ..page2.htm etc.. I was not done handeling the logic, there was an error with the logic and even tho the last page for example should have been page5.html, pages were being rendered from the first record in the database, meaning page6.html was being rendered (which shouldn't have) and was displaying page1.htm content. When the no-crawl was not in effect google came in and indexed massive amounts of pages (which now look like duplicate content) so instead of the last page being ..page5.html now in google's index there is ..page253.html. In this senario do you think a 301 redirect to the 1st page would be better or a 404 for all of those pages that shouldn't exisit and are now duplicate content?
Thanks!
Awesome post and very timely. How about 302 redirecting temporarily sold out products to the most relevant category page, which is changed to 200 as soon as it gets back in stock? Would that be a valid option?
I really don't like dealing with 302s. Also redirecting risks confusing users. Why don't you just include text on there saying the item is temporarily out of stock and that you'd notify them via email as soon as it becomes available?
We seem to be having an issue with over 1+MM 'Not Found'. The bulk of these are from search/no-results pages that show a 410 response code. Is there someway to clean this up? What is recommended?
We're a simple blog built on wordpress.
I use 301 redirects on the expires pages of GuyManningham.com and they work great. It's much better than a user or robot ending on an error. Your website is often your intitial handshake with the user. If a user sees an error, it makes the entire business look bad.
This is one of the best posts I have ever read on Re-directs. It is much more actionable then most. Excellent info.
We have a big site that used the .de Domain, but now we bought the .com domain with the same name, but GOOGLE is still giving the OLD SITE all the LINK priority and ranking. If we were to 301-rewrite our 'Old-Site-URLs.de/product.html to 'New-Site-URLs.com/product we should be keeping all the LINK juice and get our .com ranked one day exactly where the old .de is right now? The boss is very nervous about touching the old site, as it ranks and the new one doesn't yet, but I honestly think the best way to go is to 301 rewrite each and every link from the .de to the new .com site. Would you agree and concur that this would be the best way to go? Of do you think there is a lot of RISK involved, in that we could lose both .de and .com Rankings?
Just discovered a video from Matt Cutts here wich gives some more insights about the way Google treats 404 vs 410 https response codes.
But still, there is no clear way to keep content alive, indexed, but not prioritized below new / active / relevant content.
We can't use 301 redirects and 404 would be the wrong way to go, we get too many 404 errors and the content is still super related to productowners wich bought products that have gone EOL years ago.
Darn :(.
Great article....
nicely scripted : )
But i wanted to know whether i should use the 301 redirect or should use the 410 error page for my coupons website. Because it has many coupons expiring every day.
They have some link juice some are relevant to the existing categories and some are not.
Surely for the relevant ones i will use 301, but what should i do for the links that are no more relevant. Should redirect or should erase completely from site ??
Please Help !
Hi Stephanie,
I'm a bit late to the party, but thanks all the same for the very useful info.
We took over a website recently that had nine spammy articles targeting three cities the client does not serve. For the purposes of this example, let's say my client is a family lawyer in Dallas. Dallas is the target city, but there are three (3) articles targeting Plano (divorce, child custody, alimony); three (3) articles targeting Fort Worth (divorce, child custody, alimony); three (3) articles targeting Arlington (divorce, child custody, alimony).
THE ISSUE
The client doesn't want to travel as far as these three cities, and each article basically is a rewrite of the one targeting Dallas, with its own localized spammy key phrases. All content is unique enough to pass Copyscape, but it's poor at best. We just rebuilt their website in WP. The old URLs were html; the new site is extensionless. That has forced me to a crossroad with these nine articles, but each option seems to have disadvantages:
OPTIONS
Canonical: I don't want to canonicalize because there are no current relevant pages. Also, if visitors somehow access these pages (through bookmarks, for example), they'll get the false impression that my client practices there. Neither party wants that.
301 Redirect: The site's Dallas pages targeting those three areas of practice rank very well. I don't want to taint their rankings by associating them with something Google considered junk.
404: Going from zero (0) broken links to nine (9) concerns me for the short term. That's 30% of the site's content.
Given the above, I'm thinking 404 is the proper course of action for all nine pages. I'm banking on the 404 hit from Google being less severe than the hit I could take from doubling down on spammy content via 301 or canonical.
Any thoughts on this would be welcome, especially a different viewpoint on why you'd recommend one of the other techniques.
Thanks a lot.
Glen
I stay with the option of 301 redirect, I think it is definitely the best choice and is what I use in my web
Hello.
Thanks for such good info.
I have a scenario.
When I 301 re-direct less traffic page(old) to a more traffic page(new), should i delete the old page.
What about the canonical url. To which page it should link to.
Just came across 4 years after you wrote it but I found it still refreshing. Great article!
Hi Stephanie,
These are highly relevant information. It seems that in handling expired content, 200’s or 301’s temporarily provides SEO advantage. Eventually, however, 410 is the most ethical method to ensure better user experience. Thanks for the enlightenment.
Great! now I will create magento extension to handle expired product urls and make it available through my github account 'dbashyal'. I am gonna 301 redirect expired product urls to parent category.
update: Created extension - https://github.com/dbashyal/Technooze_T404
So I have a specific example and would like your opinions. I write real estate market activity posts. You know the kind like, "Beaver Cove Subdivision Real Estate Sales June 2010", and then another one "Beaver Cove, Texas Home Sales 2011", etc. The first article for an area usually has a lot of photos (a dozen or more, some in a gallery, some as regular images with good SEO tags). Most of the first articles are ranking near the top for the name of the subdivision.
Seems like I should add a paragraph to the first (and subsequent) posts that suggest that the reader look at the most recent report? And then, should I also reduce the content on older posts, to avoid duplicate content issues? An example would be that each subdivision gets described at the beginning of each post, and each post also has an embedded Google Map and a list of schools (with outbound links).
A very nice article to read
I have come up with the following options to assist my clients
before the delete a product page for example
1 is the product only out of stock now - wont do anything leave the page as is and not delete it
2 Automatic out of stock for long time would be sent back to clients as users would be coming in ( a list can be generated)
3 Delete permanently a) have it be on 404 and remove all links from the site. b) redirect to a category page (automatically) - Or rediect to a similar product page with script to let users know that they have been redirected and offer them this page as suggestion product.
Thanks
Let's say your stock was going to come back...but you didn't know when. Then, you decided to use the 301 solution. What are the implications of browsers caching the HTTP status code? That's our team's greatest concern right now.
If I 301 and expired product today...then the browser caches the 301...then the product comes back...how do you mitigate the risk of the browser using an unwanted 301 instead of the 200?
Fantastic article.
With this article being over a year old. Are there any updates on the latest techniques to use on a job-search site "webjobz.com" where we have 6,000 jobs a day being added and 6,000 jobs a day expiring and the average life of each job being around 7 days?
At the moment we are issuing 410 errors on a custom 410 page for each expired job. which sounds like the "wrong" thing to do.
Hi, did you ever find a solution for this issue? We are 301 redirecting and get roughly the same amount of job changes per day.
Michael
Hey Stephanie,
I think that the 301 redirection is not an option anymore.
I recive this message from google webmaster : 'Your server is redirecting requests for a non-existent page, instead of returning a 404 response code. This creates a poor experience for searchers and search engines.'
I have a question regarding the option of 301 Redirects, would you still want to remove expired content from sitemaps or keep?
Expired content/pages can be redirected. How about Expired Google Content Removal submissions. How are these handled. And are they reindexed?
Hi Everyone:
I have read through all of this but it doesn't answer my specific situation (though someone did mention coupons above) so I am hoping you guys can give me some guidance.
The site I am working is a monitized site which basically is coupons and special deals. The average blogger in this niche can and does post 2k - 4k posts within 3-4 months, which most, if not all, are useless after the deal is over. I'm talking Kroger, CVS, Walmart, Amazon... small ticket items; short window for the deal. I can't see a reason to keep content that is useless and outdated and comprises thousands of posts per year. It just seems crazy to me. Using a 404 or a 301 on thousands of old posts seems like clutter for the server, search engines, the db...everything. The 404 is especially concerning to me...a visitor hitting a 404 several times is not a good thing.
My question is about expiring the posts and the impact on SEO for this business model. My belief is that the site owner/blogger should put most of the effort on the static pages for SEO and expire the posts with the deals when the deal expires. It seems logical to me and of course the deal posts would pay attention to SEO, but wouldn't count on that to be "the" SEO for the site.
I know it's a debatable issue :-) Any thoughts?
Thanks,
Terri
I'm also curious about this...does deleting a page that is no longer useful have a negative effect. If Google sees a site shrinking regularly could there be a potential penalty?
Excellent refresher on 301 redirects and the additional insight on custom 404's that I had not considered.
Great Post! And very timely for our up and coming relaunch.
Nicely put togetter. I got some nice new user experience ideas to implement.
Cheers.
I have an e-commerce website related to automotive. Sometime if any product got expired so i put an information to user like "this product is sold out" only. And, user can simply go through other products which listed under the featured and latest added products. Can you suggest how can i make my website more searchable?
Thanks,
Praveen Singh
Hi Praveen
If the products are not coming back, it would be better for usability to remove links to the expired pages from your website (at least primary navigation). Users seeing a lot of expired products mat get tired with the website.
Stephanie wonderful post, great way to explain about the content optimization. Overall you have cover up very vital points because normally we used only once the content and ended up with the promotion. These are something great points you have discuss here specially custom 404 approch. I will surely create the Checklists from now onwards. Thumbs up for sure.
Good post Stephanie
These things are often dealt with on an as needed basis. It is nice to see the options all pulled together in an organized fashion.
Thanks for putting it all together for us.
Nice post. I do think there is a way to still leverage expired content to build links or reference back to within new content.
Great post Stephanie - really useful - thanks.
Great post Stephanie, lots of thoughts there simliar to mine..
I'm working on the development of a site at the moment which is similar to job finding websites.
The idea is to always have internal links to the category page(s), the companies profile page, recruitment company and perhaps similar listings(live & expired pages). There will be a very vivid, "this posting has now expired" message visible, so that new visitors to the page clearly understand the page is expired. We are considering providing an option for new visitors to click to the category page or a similar listing.
The category pages will rank stronger due to the internal linkage of expired pages, so too will those company profile pages. Categories,Profile pages & company pages will form pillars of our site, and these pillars are expected to be strong, have lots of internal and external links pointing to them.
Each of these sets of pages will list the live job offers only, this will enable them to distibute the link equity to the most recent offers. Not the expired offers.
A hurdle i have yet to clear is to no index the expired pages on a large scale, and perhaps reindex them if they have the opportunity to be recycled. This should be an automatic process. My Php developers say this can be done.. So, hopefully we can set in place a good system to rank live pages highly and demote expired pages.
I had a discussion with Wil Reynolds during a seer hangout about this process, he likes the idea too. So hopefully it has legs. Feel free to give your opinions, positive or negative :)
Sounds like the right approach! It's hard to say without knowing the specific details or being familiar with the site, and Wil Reynolds is always a great resource :)
indeed he is, good guy.
Ta for the reply :)
I've done agency SEO work for a couple of job websites, and the challenge is intereseting. Individual job offers expire extremely fast.
Consider to have faceted navigation and/or tag pages to create multiple entries to the jobs. Thenmake sure that category/tag pages have snippets for e.g. top 20 most relevant jobs in e.g. "secretary jobs in london". As you say, such pages should be pillars in the website.
Now, you might consider to test the following: Put NOINDEX, FOLLOW on all individual jobs, to make sure that it is your pillar-pages that rank in Google, which are always updated with recent jobs, and would give a low bounce rate. But make sure to have plenty of pillar pages to catch the long-tail searches of your market.
I use the term "test", because the strategy is not guaranteed to give the best ROI, just perhaps.
Also consider to use XML sitemaps to speed up indexing of new jobs.
Haha I love that HTTP status codes image!
:) Can't take credit for that - it was from one of my favorite posts by Dr. Pete (among many, many others)
Expert said that 404 pages have various SEO benefits so, it is right or wrong because I would suggest my friend so, he doesn't view it and rejected my idea. Why?
Hi Rameez,
Not sure if I understand your question correctly, but I don't see the value of 404 pages from an SEO perspective because eventually these pages will be noindexed by search engines, since they don't provide value for users.
Only time a 404 is good for SEO is if content is remaining indexed and you really, really want it removed forever. For example, if you've got spammy content on your site, or had a complete change in your product line and can't productively redirect.
Hello,the article written in very good.
Thanks lot for this useful information about "The 404 Error"-
What I can do in case of expired content?
For example I have written an article for product of E-commerce store and then after few days product is out of stock or sold out?
Can I de index same content and publish it for some other product by rewriting article?
Thanks in advance.
Depends on whether or not the product will be restocked. However, you might want to consider writing articles on broader topics instead of just specific products and I wouldn't recommend writing the same article for different products, mostly because the content would seem too generic to be helpful.
Hi!
Completely agree with Stephanie on the content element of this... if your article is broad enough that it could be applied to two different products, then the content isn't informative enough!
RE whether to 404, the main consideration should be whether it will be restocked. If it will be restocked, then just leave it live with a notice to users about the fact it's out of stock and maybe give them an option to sign up to be notified when it's back in stock and make it easy for them to navigate to other closely related products that are in stock.
If the product will never be restocked, then consider 301 redirecting to the nearest relevant product, if applicable. For example, I work with a number of people who sell bikes. Many of these bikes get new models every single year. If an old model's page has links, but will never be available again, we don't want to 'waste' the existing links and as the newer model is the most relevant page, we adopt a policy of 301ing those to the new models.
If you do decide you don't want to redirect a page, that it will never be back in stock and there's nothing relevant and you ultimately opt for the 404, at least make your 404 helpful :)
Thanks Stephanie - a very timely post. i am in the early stages of designing a system for discontinued / out of stock products on our site. At the moment some of our old products are 301'd to new products, some are 301'd to the parent category, and others removed from the site navigation, but still accessible if you know the URL, or you find the page in the SERPS.
But it's a bit haphazard, and we need some rules in place. At the moment we also have a lot of discontinued products clogging up our database for no good reason, and I'd like to get shot of them without any adverse SEO side effects.
Food for thought. Thanks once again.
If you know a product is not likely to get back in stock, it makes most sence to delete the product from the database and 301 the traffic.That has no adverse SEO effect (for whitehats)
Keeping the page but removing it from navigation is also a fine solution, that I use myself. With no internal links, Google should devalue it with time. However it is a bit spammy if you don't think the product will return.
Totally agreed that 301 redirects is of utmost importance, when you you have close related stuff to that link, or it may also be possible that redirecting to homepage is a great way to go for. All these events help keep your juice flow internally.
This could not have come at a better time. Thank you Stephanie, excellent excellent post!
''And if you don't want the page to be indexed in the off-season, add a meta tag to noindex/follow the page.''
Does that mean that later on when you turn the noindex/follow off you will bounce right back to the position you had before? Is there no penalty for this?
Leaving the page there is more ideal, but not always possible. However, this keeps the page active from a linking standpoint - all links that were going there before, still go there). Each year with more links, the page gets stronger and you're not cloaking or doing anything suspicious (not to mention the content is constantly changing).
Great article and great timing! We had this exact issue come up for a real estate client of ours and "sold" listings. We made the decision to 301 listings and save the crawl bandwith. Glad to see reassurance on the matter!
One problem I've faced when I've developed websites for a clients who used to be with another agency is moving hosting provider when relaunching their site. It seems a lot of people use terrible DNS servers which take forever to update meaning they still get sent to the old location (old IP) and thus the old website. I'm not sure how to fix this. I now use OpenDNS rather than relying on the local ones and I find updates much faster. I can move a site and see if nearly instantly when for the client it may take several hours.
Does anyone know how to deal with this type of expired content?
1) Make an exact copy of their current DNS records, and add those to your DNS server.
2) Update the registrar to use your new nameservers about 1 week from launch.
3) A few days from launch, update the TTL on your DNS to something low like 15m.
4) Launch the site.
5) A day later, up their TTL back to something normal like 12h.
Quick, painless DNS and hosting transfer. Of course, this requires that the client is on good terms with their existing host and can get a copy of the DNS. If not, use a tool like DNSStuff.com to help. Good Luck!
Hi Stephanie,
nice article and infographic ( 301 & 302 Redirection, 404 Error page & 200 OK ). But i love "How to Deal with Out-of-Stock Products" section and surely i will work on this.
Thanks,
Great info. We use a "no longer active" page on our classifieds site for expired listings, with it being marked noindex. This prevents a 404 but I see now at the cost of any link equity the page might have earned. Redirecting to the related category makes more SEO sense. Thanks!
Second thoughts on redirecting expired content vs de-indexing it. Seems risky to me to have dozens of 301s on a site; not sure how the spam algo would view that.
If you use NOINDEX, FOLLOW, you are not loosing the link equity. Just don't make the mistake of using NOFOLLOW
I think it would be best to keep the urls but just change the content on the page. It would require a custom code that allows you to change the content once it is expired.
How would you handle EOL (End-of-life) products? We currently, automatically, remove any internal backlinks it has (category, sitemap) and make sure you can't find the product anymore with the internal search function. The product then clearly states it's no longer available.
Steph,
Great post. This is a very common question I get from clients. My initial hunch: never remove content from a website! Sure, the product is no longer offered by Xbrand's website, BUT if I am searching for that exact match product title, and you've left a page with high link equity that happens to rank on page 1 for my longtail (conversion ready) search, then landing on your page will make me happy (and might still lead to a conversion) if it does the following:
Just my 2 cents! Rank on Steph, rank on....
</rant>
This is a really informative post, but to be honest I was totally distracted by the AWESOME infographic that you have!!!!!!
The HTTP Status Codes infographic is GREAT.
Totally Making a poster out of it.....I know I'm an SEO Nerd.
To give credit where credit is due, Dr. Pete created the infographic for a previous blog post
Very nicely explained. Expired content is always been a great headache. Many times, we do not even know that our website or link is not working and Google may take it wrong. This is really a great post about how to handle this expired content.
Good information, Stephanie.Depending on how your ecommerce site handles these you might want to just leave your inactive products live. My site ranks really well for some products I no longer sell but still attract traffic to the site. Surely they bounce at a higher rate but I'd rather have the chance to attract the customer than to never have them visit at all.An Excellent Feature - if your ecommerce solution supports it having a "Notify Me When Product is Back In Stock" is an outstanding way to drive sales and keep in touch with your customers. I know that InstanteStore.com has this feature and when an out of stock product is returned to stock this always generates immediate sales.Thanks - Mark
Hi Stephanie,
This post couldn't have came at a better time.
We have currently build a new website for a client and they wanted to just simply remove their old webpages and replace them on the same domain with the new webpages instead of moving domain (which is fair enough). However, the old website had a LOT of pages as it was an ecommerce website and all of these pages were indexed through Google.
What we are now finding is that the bounce rate is increasing rapidly due to the fact that we have put a custom 404 page on the website. The approach that I have started to take to sort this out is to go through and manually ask Google to remove these old URLs via webmaster tools, but to say this is time-consuming is an understatement (lucky we have a work-experience kid in the office at the moment, bless him!).
Before we start removing them all though, I wanted to check this would be a valid course of action. Would you instead recommend setting up 301 redirect rules instead? This is the first time I have came across this problem (on this scale anyway) so would really like your opinion (or anyone elses!).
Thanks in advance.
P.s, great blog post!
Hi Matthew
Returning 404 in such cases are really not good. 410 as mentioned in the comments will get the pages deindexed faster, as you tell Google "The page will never ever return". 301 redirecting is also better than 404 in this case (and my preferred strategi)
Notice, that in order to get the pages deindexed faster, a recent blogpost actually showed that deindexation will happen faster if you still link to the pages e.g. from an html sitemap. The more incoming links to a page, the more likely Google is to crawl it (and thereby notice that it should be deindexed).
With lots of pages it can easily take 3-6 months to get the lot deindexed.
I definitely agree that 301 redirects are the best option. You can control what page links where (as opposed to just sending everyone to the homepage), so your internal pages get to hold onto their link juice. You never want to leave you visitors up against a wall (404 error), but keep them moving through your site.
I'm dealing with this for a client right now, but get this. They chose to have their 404 error page be a rendering of thier primary home page, so for every piece of expired content (8000 pages) they have a duplicate of thier home page.
Also interesting to note that the top 250 of these pages account for over 4000 links.
I'm thinking that I have to get them to use 301's, would anybody rebut against that?
Great post. I always used to use 404 not found for expired content but moved to 301 permanent redirect for SEO benefits.
Thanks for sharing.
Lovely content. It's good eo read and useful.
Thanks
AuroIN