Subdomains! The bane of many SEO-conscious organizations, there is an easy solution right under your nose.
By using subfolders in place of subdomains, you can unite your content under one domain. But What if two sites exist on two different servers? A reverse proxy can make the technical implementation quite simple. With that one change, the URL https://blog.slingshotseo.com/ becomes https://www.slingshotseo.com/blog/. By acting as a content broker, your master server acts as a proxy to the content of your other servers. Multiple CMSs, databases and even platforms can be used, while all appearing to come from the same domain.
Check out this infographic to see how reverse proxy technology can relieve your subdomain woes.
Embed code:
Nice! I love a proper technical read on the Moz from time to time. Spidey senses tingling ;-)
Promote!
Agree, I think? Richard, please translate: "Spidey senses tingling" :-)
I have one more question, how did the other sites run which has every page by there sub-domain like softonic.com and toptenreviews.com, every post or point they made are simple shown as there subdomain did they also using the reverse proxy method to do that. thank.
We do this at SEOmoz. /qa, for example, is proxy_passed through the www servers.
Many of the higher-end load balancers will do this for you as well.
There are some technical challenges to be aware of. If the application being proxied has a hierarchy that references a / or a subdirectory other then the one being proxied, it will break and may need some herioc rewrites to make it work.
David
Another good use of reverse proxy services - hiding tracking phone numbers. Flightdeck, our best of breed marketing dashboard, uses tracking phone numbers to monitor the effectiveness of marketing campaigns. One potential problem with this is that if Google finds phone numbers that differ from your real business number, it can hurt your Google Pages listings. Phone numbers on reverse proxy servers are not found by Google and therefore don't create a problem.
A very detailed step-by-step for IIS7 and Apache would be a really good follow up to this, showing a screen at a time of what to do.
Rackspace cloud uses this technique(proxies) for their dedicated pool for SSL in the cloud sites platform. In that case I wouldn't suggest this because on a site that steadily has 20 to 40 ACTIVE visitors it can be sluggish and sometimes you will get a message like this:
Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request GET /.
Reason: Error reading from remote server
Apache/2.2 Server at .... Port 80
How does performance hold up in other cases? I don't know.
"Sub-domain pass very little authority to root domain"
where is that coming from?
Let's say you have one main domain, and one subdomain containing your blog. If you interlink the sub-domain on every-post, as well as side menu to the main domain, is that really different than having the blog hosted on a sub-folder? I thought that as long as interlinking is strong and you don't have hundreds of subdomains, it wouldn't matter
btw nice infographic and info :)
It does matter, a subdomain is almost a different site with no authority
I can testify on behalf of my team that this infographic DOES help you in cilent meetings to explain a reverse proxy without too much 'geek-speak'
Excellent post - Great, detailed information and engaging graphics. This is what an infographic post should be.
Rethorical question: Jen don't you move this awesome post to the main blog?
REALLY appreciate the infographic approach to writing a blog post!
This infographic is one of the few that I actually read through and absorbed. Mission accomplished. 100 Thumbs Up!
Hopefully not a silly question: what are the performance issues, if any, when using reverse proxies /w Apache? Especially if the content is on physically separate servers? Is there any impact on Apache serving the non-proxied pages? Thanks peeps :)
Not a silly question at all, Ackthpt. If anything, this should improve performance of the proxy server. Instead of having to serve the content itself, the proxy server simply asks the separate server for the content.
We use reverse proxies at Compendium.com to serve blog hosted on our servers in subdirectories instead of subdomains. Because you're hopping services, especially when pphysically separate, there is a small performance hit. But i'd trade a few extra milliseconds for better domain/page-authority control anytime.
No embed code? Thought that would have been aloud with attribution to Youmoz + your site :)
Sure! You can embed the image using the following HTML:
<a href="https://www.seomoz.org/posts/view/what-is-a-reverse-proxy-and-how-can-it-help-my-seo" target="_blank"><img src="https://www.slingshotseo.com/wp-content/uploads/2011/09/ReverseProxy_9_6-e1315407839540.png" alt="Reverse Proxy Infographic" /></a>
Extremely beneficial for me and nice infographic. Good stuff Slingshot SEO!
Slingshot SEO nice infographic and thanks to choose infographic medium to explain such a concept as it really gives me very hard time to understand such a technical stuff by text article.Well I had understand the importance of reverse proxy and it sounds really very helpful I will definitely give it a try.
This technique can be useful for rewriting content on the fly e.g. appending additional robots meta tags where a CMS does not support it.
Short, sweet, and a proper use of an infographic! I actually get something USEFUL from it.
Well done Slingshot guys!
Ironically - we just did this the other day for a client. Woulda been nice to have this infographic to show them beforehand. Great job!
Good stuff!
I'm just wondering: what SEO implications does it really have? I mean, how hard is it for a SE to find out a reverse proxy is used? And is this considered 'cheating'? The content actually is on a subdomain/different domain and not physically on the main domain.
I have two question here - please assume I have rather no technical skills:
1) After setting up that reverse proxy the search engine only sees the result domain? They can't spider that setted proxy?
2) You only wrote about subdomains. I guess you can set the same proxy from one domain to another, right?
1) The search engine will be able to crawl both, but one will be proxied through the other. The separate server will have to have redirects in place that redirect all traffic, with the exception of our proxy server. This will prevent it from appearing as duplicate content.
2) The content server can be located at a completely different domain, or simply an IP address. As long as it can be found on the network, the proxy server will work.
Hi, really liked your post and I am very interested for this implementation. However some things are not very clear to me.
When you say that "The separate server will have to have redirects in place that redirect all traffic" which kind of redirects do you refer to? 3xx redirects such us 305 use Proxy or 301 ones?
For instance, if I have blog.domain1.com/ and want to revers proxy it from domain2.com/blog/, does it mean I have to 305/301 redirect content from domain1 to domain2.com/blog/ and that this second URL (domain2.com/blog/) will be the one indexed by Google without any duplicate content issue?
I have no experience yet with reverse proxy configurations so I wonder if if will read the content hosted in blog.domain1.com/ or will follow the 305/301 directive ending in a never-end loop.
I am quite intrigued about the SEO/Panda effect of this immplementation.
Thanks
Excellent post, we actually use this technique today and it works very well for us.
Great infographic.
Quick question. What ramifications are there if the blog content has been indexed prior to this reverse proxy initialialization? What duplicate content problems could the be, and where would canonicalization come in?
Also, what about the the URLs of the blog conent, and absolute or relative references, wouldn't they have to be groomed?
Very good question, Strategics. In most cases, the content server (https://blog.slingshotseo.com/, for example) would need redirect rules set up, creating a 301 to the appropriate page. This would let search engines know where the new content lies. If it has already been indexed, it might cause some confusion in the short term, but yield long term benefits.
As for the existing URLs on the content server, if the redirect rules are set up properly, it will translate to the proxied version.
For example, https://blog.slingshotseo.com/trust-in-me-just-in-me/ should redirect to https://www.slingshotseo.com/blog/trust-in-me-just-in-me/
The problem strategic points out is as follows:
https://blog.domain.com/page.html contains a relative link href = /page2.html. if you were to click the link from that URL, you would end up at https://blog.domain.com/page2.html and see the content.
When pulling it rewritten from the proxy server, the page is served as:
https://domain.com/blog/page.html
If the proxy server does not rewrite all the link within the content of the page, you end up clicking the link and 404 on https://domain.com/page2.html, because the content is actually at: https://domain.com/blog/page2.html
If the proxy server does rewrite all of the content of the page, then this ties back to the performance concern stated above (also, there are perf related concerns regarding the fact that you end up with 2 pairs of request/response instead of just one, which increases response time especially if blog.domain.com and domain.com are not physically on the same network (ie 2 pairs of request/response over the internet))
Dont get me wrong, reverse proxy is great, just pointing out a few things you should be aware of :)
(dbl post sry)
Loved it... the reverse proxy is much efficient way to make our contents seo friendly, and most of all... I liked the visual representation. It was worth to see it for just some moments and easy to understand!
Simple, clear, informative and on a topic which impacts many sites. The infographic is a great bonus which just makes this article all the more useful.
Hi, I am working in a multinational company and unfortunatelly company structure is based on hosting all the websites in one server located at USA because of the content management application.. The major issue is in terms of SEO for a brandnew platform that I will develop I need to localize it or i will be lacking of SEO. They do not letme do this in terms of investment costs from the head quarter.
The question is, If we set a reverse proxy in the other country will it help to solve the issue ? So we will be running the CMA in USA server and it will mirror it in local server? What will google understand that this server is hosted in Local or USA and how will it optimize our future web site?
Thank You! I love it when I learn something really useful, and I did from this infographic, and the excellent follow up questions and answers in the comments.
interesting post, I hope it will help really :)
What if i have an existing blog on a subdomain and have been building links for a couple years on it. If we decide to go the reverse proxy route now, is that going to set us back? Does a simple 301 remedy this?
i was not aware of reverse proxy and its use this post is helpful and the explaination with diagram make it easy to understand thanks for the post
I am interested in doing this for an e-commerce client - I don't want to risk having theirsite.com/blog with a Wordpress installation because of the security risks Wordpress brings. The main shop site cannot ever go down, so no matter how much I am told/blog as a subfolder has maximum juice, I am keen to keep the two worlds apart.
So - is the upshot that Google truly sees it as subfolder?
And am I the only person who has these concerns over Wordpress blog subfolders on mission-critical sites?
(Note to SEOmoz - the text input box on here still shows text in a tiny font on Win 7 Chrome plus carriage returns are interpreted as two carriage returns)
Hi BlueLightInternet,
One of the benefits of using reverse proxies is the security. Should the blog fail, the e-commerce site will remain active, and any .../blog requests will simply receive a 503. By having the blog on a completely different server, with a separate CMS, you can avoid interference with the e-commerce site and its data.
And, yes, Google will truly see this as a subfolder.
Thanks for the great post. I am trying to set this up for a Wordpress.com website which has been mapped to blog.domain.com. Does reverse proxy work in this instance to move the blog from blog.domain.com to domain.com/blog?
We have a client that is moving to a new blog that will be using a reverse proxy to make it look like it's being hosted at www.domain.com/blogi nstead of blog.domain.com.We can't get a clear answer from our provider on how 301s should be handled. We actually managed to make 90% of the URLs (wordpress-hosted) stay the "same", but a select few had to be rewritten due to the redesign. Here's the deal:Ex: www.domain.com/oldblogurl needs to 301 to its new home on the wordpress server: www.domain.com/newblogurl HOWEVER: the "actual" home of the URL is really https://blog.domain.com/newblogurl, but the reverse proxy makes it look likehttps://www.brownwoodacres.com/newblogurl Can we still configure a 301 redirect to point www.domain.com/oldblogurl to www.domain.com/newblogurl, or does the reverese proxy add an extra layer of DNS complexity to make this impossible? What we DO know is that it won't work to 301 www.domain.com/oldblogurl to blog.domain.com/newblogurl, since the entire point is to prevent discovery of the subdomain.
I setup a blog for a client and they were going to use a reverse proxy, but for some reason only the first page of the blog goes to the primary domain, everything else routes to the subdomain. is there any reason that the entire subdomain would not be automatically displayed on the primary domain?
Thanks for the super information.
2 questions about using reverse proxy to maximise SEO juice from subdomains.
1- How do you handle it when there are 2 protocols involved. My site is: https://www.mysite.com and the 3rd party site which hosts the subdomains = https://www.mysite.com
2- Also from the security perspective are we not exposing our top level domain by doing this proxy recommendation, what would happen if the 3rd party hosting the subdomain has a JS injection attack?
Thanks for your comments.
Regards.
Lakeside
Nice infographic! I do love infographics since the data presented in there are much more appreciated and understood. Just like this one. I thought, I couldn’t find a way to help my SEO working. But with this, I’ve found out that it is possible to have two different pages (hosted in different servers) to appear the same. Google will crawl and find these two as authority subdomains. But of course, this will not be successful without the help of reliable proxy server that I am using right now. It’s called Hidester. Have you checked that one? I hope you do.
This is a little tricky to pull off but it works. If you can avoid this kind of architecture you will probably have less headaches, but as a last result it can have dramatic impact. It seems to work with Wordpress...but take your estimated time and multiply it by a factor of 4.
Good read, got it dude. But how this affects your website in terms of its rankings? Would Google allowed this one? -as content is the highlight on this.
Very technical stuff. Hope I don't have to do this.
OK, this was possibly the most valuable post I've ever seen on SEOmoz...I have 3 very big clients for whom this is a total game-changer. Thanks for helping make me a hero with them! Cheers!
Very useful tip!
You say using subfolders is preferable from an SEO perspective - can you give some insights as to why that is better than using subdomains?
Thanks!
Really detailed with great information and engaging graphics. This is what an infographic post should always be.
Awesome, This is a fantastic information and thank for embed the image using this html code.
<a href="https://www.seomoz.org/posts/view/what-is-a-reverse-proxy-and-how-can-it-help-my-seo" target="_blank"><img src="https://www.slingshotseo.com/wp-content/uploads/2011/09/ReverseProxy_9_6-e1315407839540.png" alt="Reverse Proxy Infographic"/></a>
Great and beautiful infographic. I can't think of a way anyone can explain in fewer words than those.
I have a question around this statement -
Subdomains pass very little authority to root domain
I saw Matt Cutts's video sometime back on this very subject and he was ok with any of the structures. Are there any tests conducted that can prove this statement?
Hi AnkurJ,
I'm unsure if this is the same article you were referencing: https://www.mattcutts.com/blog/subdomains-and-subdirectories/
Although this is from 2007, Matt discusses a trend that leads Google to not give subdomains preferential treatment. He goes on to say that he would prefer subdirectories to subdomains.
To clarify, it's entirely possible for a subdomain to have a large enough authority on its own, that it passes it to the root. However, this wouldn't be any more remarkable than a completely different domain passing authority.
The statement should read: Subdomains pass no extra authority by simply being a subdomain.
Due to the nature of this issue, it is very hard to test. That being said, if anyone from the SEOMoz community has two high authority domain and subdomain sites that they would like to use in an experiment, we would be more than happy to hear from you.
The problem strategic points out is as follows:
https://blog.domain.com/page.html contains a relative link href = /page2.html. if you were to click the link from that URL, you would end up at https://blog.domain.com/page2.html and see the content.
When pulling it rewritten from the proxy server, the page is served as:
https://domain.com/blog/page.html
If the proxy server does not rewrite all the link within the content of the page, you end up clicking the link and 404 on https://domain.com/page2.html, because the content is actually at: https://domain.com/blog/page2.html
If the proxy server does rewrite all of the content of the page, then this ties back to the performance concern stated above (also, there are perf related concerns regarding the fact that you end up with 2 pairs of request/response instead of just one, which increases response time especially if blog.domain.com and domain.com are not physically on the same network (ie 2 pairs of request/response over the internet))
Dont get me wrong, reverse proxy is great, just pointing out a few things you should be aware of :)
timely infographic at the moment when we are trialling a reverse proxy solution for corporate use in the organisation where I work. We will however be using Akamai to perform rerouting from a single Drupal CMS install over a large network of sites. We are currently testing speed levels, bots behaviour and/or any other events. I'll be reporting shortly on this via a blog post if I am given thumbs up for it. Great infographic, Slingshot, thanks for sharing, and no doubt I will be sharing it internally
This will be really useful in future when dealing with subdomained blogs and other weirdness,
Thanks for making it so clear.
Excellent! Nice and clear infographic review. It is perfect for us, dummies, who liked picture books when we were kids :)Thumbs up for this. Very informative and detailed explained step by step.
Thanks for the post. Do you know if this works if you use Wordpress?
Because our blog uses Wordpress, and we tried this, and ran into problems with Wordpress URLs/Rewrites.
We were therefore forced to use a subdomain.
If you have a solution using this for wordpress would love to know.
Yes, Croozie, this will indeed work with Wordpress. In fact, Wordpress was the CMS of choice while we were implementing.
The ability to customize the paths to be "domain.com/blog" or "domain.com/whatever" via configuration settings eliminated the problem of having to change a large number of navigation links.
That being said, any links that exist in the content of documents will have to be updated. Is this the problem you are facing?
Hey Slingshot,
Here's what we tried.
Subdomains pass no extra authority by simply being a subdomain.
I thaught authority will be passed by links. And on a Site this will be realized by crosslinking the content. Is there a real differnence between subdomains and subdirectories, if they were crossconnected with the same number of links? Is there a real "extra" and why?
Or are there other seo-reasons to use subdirectories instead of subdomains?
I'm hoping someone can help me out here. I've got our blog running on a subdomain on lets say "server 1" and then the main website on "server 2". We're on IIS7 and was wondering if it's possible to do this and if so, which server should I install the plugin on? Thanks!
Sure thing, jbs325is.
To do this with an IIS 7 server, you will need to install the Application Request Routing plugin on "server 2": https://www.iis.net/download/ApplicationRequestRouting You shouldn't need to install any additional software on "server 1."
There are a lot of details in the setup. Ruslan Yakushev has a great article on the IIS web site: https://learn.iis.net/page.aspx/659/reverse-proxy-with-url-rewrite-v2-and-application-request-routing/
Hope this helps.
Thank you for the post Slingshot SEO, this has really got me thinking about some of the bigger websites I run, which involve a lot of subdomains.
My level of SEO understanding is well above average, but when it comes to server related matters that involve htaccess and such, I seem to struggle a little. Have I understood correctly if I say that you would add the following 3 lines of "code" to the htaccess file:
Does this apply to the example of blog.slingshotseo.com becoming www.slighshotseo.com/blog?
Hi ChristopherM,
In this scenario, we are proxying to an existing blog. The code in the www.slingshotseo.com's htaccess file should be:
If you are creating a new blog, "blog.slingshotseo.com" could also be an IP address, "176.295.43.193"
This assumes that your server has the Apache module mod_proxy installed and enabled. mod_proxy is included with most Apache distributions, but might not be enabled.
Thank you so much for your response and confirmation on my thoughts Slingshot SEO, didn't know if you'd have a chance. Amazed more people didn't ask!
My last question, I'm going to implement the htaccess tweak, how do I test if it's implemented correctly?
I best go read up on proxypass, proxyrequests and proxypreservehost as I like to understand what I'm doing, rather than just asking, implementing and praying.
If you don't have a staging/development server, it makes testing a bit more difficult. I would highly recommend testing this on an identical server, before trying to implement on your production server. In either case, here is what I would do.
Answer the following questions:
Then I would follow these steps:
Hopefully, this can get you started.
Thank you :)
I understand how to go about doing it all, that's pretty easy, I should have phrased my question differently - knowing that visiting /blog without errors means it works is what I was looking for.
Wicked, let me get going!
Cheers!
Nice POST!! I enjoyed the technical read out and explanation of why this works better than subdomains. thank you
I think a lot of SEOs are changing their minds about the choice between a subdomain and a subdirectory when it comes to blogs and other content-portions of eCommerce, corporate and affiliate websites since Panda came around. Moving content to a subdomain isn't a "magic bullet" but it has helped several sites get away from their affliction recently. Conventional wisdom is great because it acts as a hueristic to experience for many. But sometimes paradigms start shifting before the infographics do.
Personally I don't recommend migrating content off of a main domain over to a sub-domain for Panda reasons. If the content moved does have value, what I've found is the only time to move it to a sub-domain for Panda is if it's information that's radically different in topical focus than the entire rest of the site.
What I have been recommending to clients instead, with great success in Panda recovery, is to take the time to properly organize the content for more refined topical focus, consolidate thin content as well as perceived duplicate content, (beefing up unique content within individual pages), and removing the confusing over-used "related" content boxes that so many sites have where it's "sort of" related but not enough to be of laser-focus topical relevance. For many of those clients I also help them thin out the over-use of on-page advertising that further wraps around the main content areas, and finally, in some cases, have them improve page speed.
All of these techniques have helped clients rebound, though some have also then needed new social media and inbound link work done to reinforce the "we've cleaned up our act" perspective.
I take this route because of long-term reality.
By moving to sub-domains, you end up having to essentially build their authority up from the ground; also, unless the content is truly unique, you're not doing site visitors any favors by isolating that content. And finally, because if the content really is worthy, and can be consolidated as I've described, you retain that much more depth for the main site, which is the most beneficial aspect of choosing this path from an SEO perspective.
Just my experience, involving several web sites comprising millions of pages ranked and millions of visits.
Nice visual representaion of the technique. Just a heads up though that you'll either need to be able to reconfigure your subdomained blog to accept and spit out the new subfoldered addresses or put some redirects in place otherwise you'll end up with a load or either 401s or duplidcate content. I don't see this being a viable way to subfolder a tumblr blog for example.
Also nginx and cherokee webservers make very capable reverse proxies.
Not something I knew before, so handy info.
However that info would have been more understandable in a normal written format. The infographic takes away from my understanding rather than enhancing it.
I agree that some of it could have been moved to a written format, especially the URLs for finding more information -- would be helpful if those were clickable.
I completely agree. The infographic is a little distracting.
Pretty much agreed on this one! The infographics make this boring topic (hosting and domain are always boring for me) become interesting for me…
I use to advice clients in our company to move there blog from blog.domainname.com to domainname.com/blog but obviously its bit difficult for them to understand as this is more like a geek talk.
I think this info-graphics actually make it little easier for a client to understand this recommendations.
Congrats to Slingshot for such an awesome post!