This week we are thrilled to have a special guest joining us for Whiteboard Friday: Duane Forrester, a top-of-the-line SEO who went over to the other side of the fence and now works at Bing's Webmaster Project as their Senior Project Manager. Duane's the one you'll see throughout their blog, and if you have a feature request or any questions about Bing's Webmaster tools, he's your man. Duane joins Rand to discuss a multitude of search topics, including Bing's Webmaster Tools suite, the metrics used and displayed in their tools, and some exciting and extremely important news about Bing's use of quality thresholds for sitemaps. Check it out, and let us know what you think in the comments below!
Video Transcription
Rand: Howdy, SEOmoz fans. Welcome to another edition of Whiteboard Friday. This week I have another very special guest for you. Duane Forrester used to be one of the chief SEOs for Microsoft, architecting all the things that Microsoft had to do about SEO. Duane, you felt the webmasters' pain, you felt the SEOs' pain, and now they handed you the keys to the Ferrari and you are running Outreach for Webmasters for Bing, the search side of things. You switched sides. It's amazing.
Duane: Yeah, totally.
Rand: First off, thanks for being here. I really appreciate having you.
Duane: Thanks for having me over.
Rand: Second, tell me what this change has been like for you, to go from SEO guy to search engine guy.
Duane: Yeah, it's been kind of intense. I spent over a decade of my life as an in-house search SEO, and this is a dramatic change. Seeing things from the other side gives me an entirely new perspective. Some things are very validating. Other things, I'm slightly embarrassed by.
Rand: That's a good combo.
Duane: Yeah, and it's humbling is what it is. So, part of what I'm keeping in my mind as I go into work every day and I work on Webmaster tools and I work on bringing new ideas and content forward is this idea that I was there, and the people that are there now are fulfilling a very important role for their business. They need as much help as they can get. So I truly am the eyes and ears of it. It's amazingly exciting.
Rand: It's great for you to have that background rather than . . .
Duane: Completely.
Rand: . . . an attitude of antagonism potentially towards people doing SEO. You've got one of empathy. I think that's wonderful.
Duane: No, my antagonism is directed internally toward everybody I work with because I want all this stuff and they keep bringing me in.
Rand: So on that front, let's talk about some of the stuff that you've already gotten into the product. The last six months have been phenomenal. I just logged in for the first time in about 30 days, maybe 20, 30 days, and there's tons of sweet new stuff. Talk to me first about this Index Explorer thing. This is kind of spiffy.
Duane: So, Index Explorer for you Mozzers who have been watching, you'll know this. Index explorer is an area, if you come into Webmaster tools and you actually look at the tab called Index across the top, on the left-hand side at the top of the navigator, you'll see Index Explorer there. I've known about Index Explorer for a few years now because it's something that I can use as an internal employee to kind of peer into Bing's index and see what's there.
It's a really handy thing for us, because when I was SEO for MSN, it was at vast scale. So, if I had an indexation problem, it was a magnitude of orders. Now, actually everybody who has access to Bing Webmaster tools sees a version of that, applicable to their own domains.
Rand: So this is like the internal tool pointed externally and you can see the stuff that applies to you.
Duane: Right.
Rand: So, I can go in, I mean we just did this with SEOmoz.
Duane: Yeah, exactly.
Rand: We looked at, here's SEOmoz.org and you click this little drop down and it's got all the folders in there. So now I can see slash blog. Then I can see any individual page, for example, post.html. That's not actually our structure. Then if I click on this, it pops up a little box and I've got links and I have anchor text.
Duane: Exactly, yep.
Rand: I can export that.
Duane: There are actually a couple of really neat features in here. So, what we do is in this area we give you the ability to come in and take a look at something. Maybe you're looking at and you're saying, "Jeez, you know, I had a duplicate content issue there, and I've since installed a rel=canonical to solve my issue. But that's still in the index, so I need to remove it." You can actually click a button right here and block that. That tells us, "Hey, you know what? You shouldn't bother with this anymore." Thank you very much. We move on from there.
Rand: How does that act? Does that act like a robots.txt blocking?
Duane: Essentially, it basically tells us that, okay, if you, the site owner, are saying don't go there, then we're going to honor that, because, quite frankly, if everyone took that kind of care and attitude toward maintaining the website, it lessens the resource load on us.
Rand: Cool.
Duane: By allowing this little button, this little button . . .
Rand: That little button right there.
Duane: Now orange. It makes a big difference for us. And we give you the option, you can block an individual URL, or you can block a folder if you want. You may look at that and say, "Oh, you know what? I don't need anything in this print folder. Block that."
Rand: And now I don't have to write to robots.txt if I don't have access?
Duane: You still should be writing to robots.txt. However, there are cases where people won't have access to it, and this is an opportunity for them to still communicate the need.
Rand: Sure, absolutely.
Duane: Another area that's in here that's really cool is we're going to show you how many links there are to this particular URL. You'll see a little number here. The one we looked at was 198. And then if you actually click on where it says "pages linking to this page," click on that, it opens up this really nice, beautiful pop-up, and that actually contains all of the URLs, the domains that are linking to you, the URL that it's on, and the anchor text that's being used to provide all those 198 links to this individual URL on your website.
Rand: And you're showing these, if I recall here, you're showing them in sort of a date order of crawling . . .
Duane: Right.
Rand: . . . and you're showing up to 20,000 per URL.
Duane: Exactly, exactly.
Rand: That's a lot of link data.
Duane: That is a lot. For most small, medium, and even a lot of large websites, that is going to be far in excess of what they're going to need. That is on a per URL basis, so it's not 20,000 overall. It is 20,000 applied to this individual URL. The next URL gets another 20,000, and another 20,000 and so on.
Rand: This is potentially a lot more data than I can see from any existing source at least about my own site.
Duane: Exactly.
Rand: Because Webmaster tools in Google has a sort of a limit on, a smaller limit and it's across the whole site.
Duane: Right, exactly. This right here, I am super excited about Index Explorer simply because of the detail it lets you get into. So, now let's say, you're really trying to figure out how to build out a proper link building program, and you're looking at this going, "All right. We have good links. How do we maintain this? How do we optimize these things?" This is a blueprint for who's linking to you and what they're saying about you. This is an opportunity now for you to contact the websites and say, "Hey, I notice you're using this anchor text to link to me. Instead of click here, can you actually put my product name in there? Here's a sentence I've written for you that incorporates that. All you have to do is copy and paste it in place for me. Thank you very much." If people are wondering how you tracked it on the website . . .
Rand: How much do you have to pay these people to have them do that for you?
Duane: Oh, dude, that's between you and them, right? Personally, I don't pay anybody anything. My own website? If you're going to do it, I love you for it, and I will show you the love later on. If you're going to ask for a handout for it, it's not really how it works.
Rand: Right, yeah. I was going to say, and Bing has been pretty good about penalizing a lot of the links that look manipulative on the Web too.
Duane: Yeah. It's a natural part of keeping things clean, right? At Bing, we are very keen on having a quality driven index. So, the main focus we have is making sure that everything that gets in is a good resource, when someone makes a query they get a realistic answer that is actually an answer to their query. Not, here's some shallow depth data. I'm going to click on it, and then oh, it's not really what I want. I go back and I try it again. We're trying to shorten that number of searches to get to the final answer.
Rand: So, a question that a lot of people have around Bing is often, I've launched my site and I've seen maybe some other search engines pick it up, but I haven't yet been crawled as deeply, or I've been crawled but I haven't indexed. Can this help to answer that question of why that's happening?
Duane: Well, what it will do is, over time, first off you've got to get in the index. If you're not in the index, nothing's going to show here. So what you want to do is make use of the submit sitemap feature we have in here. You know what, I was going to do a blog post on this on the Webmaster Blog but I'll just tell Mozzers, you guys are hearing it here first. We have a quality threshold on our sitemaps. When you build a site map for us, we want it to be clean. When you put a URL into our site map, what I don't want to see in there is any URL that's a 404, 302, 301, anything at all. I want the end state URL only.
Rand: You don't want rel=canonicals.
Duane: Only end state URL. That's the only thing I want in a sitemap.xml. We have a very tight threshold on how clean your sitemap needs to be. When people are learning about how to build sitemaps, it's really critical that they understand that this isn't something that you do once and forget about. This is an ongoing maintenance item, and it has a big impact on how Bing views your website. What we want is end state URLs and we want hyper-clean. We want only a couple of percentage points of error.
Rand: The best of the best 200s.
Duane: Right, because if you start showing me 301s in here, rel=canonicals, 404 errors, all of that, I'm going to start distrusting your sitemap and I'm just not going to bother with it anymore. If the way that you're communicating to me that you have new content is to submit the sitemap through the functionality in Webmaster tools, instantly you're submitting me something that I've learned not to trust because its cleanliness is in question. It's very important that people take that seriously. It's not a fire and forget. Don't just go and grab some random tool when you do a quick search and saw, oh, here's a sitemap generator. It will go crawl my site. Blah, there you are. Forget how many links that actually misses. Seriously, be thoughtful when you build your sitemap. We don't want every single page from every single website. We want your best quality pages and content. So, you as a site owner . . .
Rand: Are there content thresholds that I should be thinking about around this too?
Duane: There are probably are, but those are buried so deep in so many different layers and there's a lot of other influences too. I mean, you can have variations and orders of magnitude and still rank well.
Rand: Okay.
Duane: It's not something that I would really say to folks, look you need to lose sleep over that. This, you should be thinking about and investing in.
Rand: So any of these codes, you really need to worry about.
Duane: Right, exactly. I love this idea, Rand, this whole pick your top 200, whatever the number happens to be for you, pick it and run with it. You don't need everything indexed. Pick your best stuff and make sure that's in there. Make sure your quality content is in there, right? Be sure that you look at the site and say, "What's the goal of this page? Is it to monetize ads? Is it to convert somehow? What is the goal of it? Is it optimized properly to do that? If it is, I want that indexed in the search engine ranking well."
Rand: Duane, are you telling me that we need SEOs?
Duane: Funnily enough, it works.
Rand: Along with this, there's another really cool tab and we're not going to go all through it, but I want to talk about the traffic tab because there's cool stuff in there right now. It will show the rankings data for Bing.
Duane: Exactly.
Rand: It shows something called average rank and average impression rank.
Duane: Exactly.
Rand: So per-click rank and impression rank, what are the difference between the two?
Duane: The way it works out is, because of the volumes that a search engine deals with, like millions of things every minute, essentially, what we're really showing you here is we're saying, when someone did query, the average impression -- where you showed -- is this position. It could be that the average works out to 3.2. So you were roughly the third position for all these queries. The next line over will show you what the click-through rate was. The average click-through rate for you may have been 9.8%, which pretty much falls in line with what we kind of all know and understand.
It's a realistic thing, right? Then the next column over is actually going to show you the average position you were in when that click happened. Now, here's where it gets really exciting for an SEO. If you know that you're in third place most of the time and your click-through rate was 10%, we'll say, but the clicks happen when you're in the first position, how does that 10% that you're getting being in the first position potentially compare with other people who may be in the first position? Now, we don't show you the competitive data, but it should get you thinking about this. Is 10% for me in the first position on this query a realistic amount?
Rand: So, I can look at those numbers, and then I can say to myself, "Wait a minute. This doesn't match up with some of my better click-through rates.
Duane: Right, exactly.
Rand: I should be thinking better title . . .
Duane: Meta description, title, all that, exactly.
Rand: URL string, rich snippets.
Duane: Totally. Speaking of rich snippets . . .
Rand: The stuff that Stefan was showing us, and it just rolled out is pretty slick. You've got that nice sort of box, and there's the guy and he's cooking up some piaya in there, whatever he's got. Then there's the title listing and the meta here.
Duane: Yep.
Rand: I mean that looks really slick.
Duane: It is. I don't know what to say, other than if you're thinking about rich snippets, do it. It's very valuable to a search engine, which means it's very valuable to you and your content.
Rand: What's the how of this? Is there stuff in Webmaster tools now or will there be?
Duane: Not yet. Just as we were prepping for the session, we were talking about future looking things. Rich snippets is one of those still kind of floating for me ideas. I have to find the space for it. I have to understand what the value prop is, what the likely cost is to you, as a business owner. It's one thing if somebody has . . . if there's anybody out there who does plug-ins for WordPress and wants to make a rich snippet plug-in, might be useful to people.
Rand: All right. Rich snippet plug-in . . .
Duane: Because, I mean, you think about the plug-ins we have for WordPress already . . .
Rand: Is Joseph watching this?
Duane: I'm guessing there's already one out there, because rich snippets is a concept that's been around for a while now. So my money is on there's already one there.
Rand: The question is, is the format that you're using the same format that I would use when I am submitting rich snippets to Google, or is this slightly different?
Duane: It's a little bit different, but we're essentially looking for the same kind of points, right? We want this meta markup data to be able to help us understand exactly what this piece of content is. In a lot of cases, it's not going to matter because it's a regular content page . . .
Rand: What is this image? How does it related to this page?
Duane: Right, for that kind of thing, that's really important to us, because, yet still, even though it's 2011 and we are technology advanced, although we're still flying the space shuttle today, search engines still can't actually look at a picture and say, "Ah, that is a fat chef with piaya."
Rand: Must be good.
Duane: Right, exactly. Eats well.
Rand: Awesome. Before we wrap up, there are three questions that I know webmasters should be curious about. You may not be able to give us a ton data about what's happening with Yahoo Site Explorer.
Duane: I'm actually going to not quite pull a no comment on this one, but Yahoo Site Explorer is a Yahoo product, and we don't comment on their product pipeline or things they're working on. What I will tell you is, if you're looking for external link data, it's right here. So we have it for you, right? It's a similar product. We power it.
Rand: Is Bing thinking about some way of showing competitive linking, so like I can go see the links to anyone? Or is that not a product that you're interested in at this point?
Duane: I, personally, have a great deal of interest in it. Bubbling that up to what we can actually bring forward, there's a lot of layers to get through on that. Conversations like that, very nascent at this stage, meaning it's on my whiteboard. However, I am influenced by the greater user population. So, as people have feedback, I'm going to be at SMX in a couple of weeks, I'll be at South by Southwest, SES New York, specifically looking for feedback from people.
Rand: Fantastic and in the blog comments.
Duane: Exactly. If we have like . . . you mean I'm supposed to come to the blog. Oh, you didn't tell me that.
Rand: What? You give up SEO and now you don't come and visit anymore. What's up with that?
Duane: No. Seriously, if people do have feedback for it, right, I capture all of that data because that is a tool that I can use with our product planning folks to say, hey, look, we reached x number of thousands of people with this and the percentage that got back to us was like 18% said they want "A"'.
Rand: So, you're telling me if I throw up a little survey on the blog and I ask people what they most want from Bing's Webmaster tools people . . .
Duane: I will buy you a martini.
Rand: All right. It's done. It's going to happen.
Duane: There you go. There we go.
Rand: Then, static rank, you know people . . .
Duane: Yeah, it's pretty cool.
Rand: Well, MSN and then Bing had this for years, this static rank rate, which my understanding is it has some of the logic of page rank, but a bit more advanced in terms of what it analyzes and what it looks at.
Duane: There are a lot of signals that goes into calculating that, as I am sure there are for page rank. Now that I've kind of come inside . . .
Rand: You can see those.
Duane: . . . I get it now.
Rand: So static rank, we sort of talk about like, hey, Bing wants more webmasters paying attention. They want more people downloading and installing their toolbar. What do you think about putting static rank in a toolbar or in Webmaster tools?
Duane: Personally, I see a great deal of value for people in it. There's even the entire conversation about gaming and all that. We're kind of at the stage now where we're kind of really getting beyond all those things. Like the ability to deal with people trying to game something has evolved immensely. It's . . .
Rand: Showing them a little one through twenty radiated bar isn't going to make them more . . .
Duane: From the company's perspective, however, there's a lot of personal or I should say proprietary investment in this concept. So, what would make it out is got to go through a lot of layers again. However, in your poll . . .
Rand: Put in static rank.
Duane: Why not?
Rand: All right. It's in there.
Duane: I mean, seriously, if there's any tools people think they might want, let us know. I'd be surprised if we didn't already have a version of it floating around the ecosystem somewhere. Part of my job is to bring that all to one spot that's useful.
Rand: I was going to say, you've done some cool stuff to bring Index Explorer into the public facing tools.
Duane: Exactly.
Rand: Last question, and this is around less Bing focused stuff or tool focused stuff, but we have this big content farms update, the farmer update that's coming out from Google, and there's been this discussion ongoing for a while around, hey what's up with people like Mahalo and eHow and Answers.com, who are essentially producing a page for every search term they can possibly ever find. Sometimes of okay quality, other times of really low quality. What's your opinion of how, if you were in charge of the algorithm of Bing and how it got handled, what would you do?
Duane: I'd be looking, and this is the toughest part of it, is separating the wheat from the chaff. These guys seriously have some depth in areas that are legitimate answers to questions. How do you, you can't look at a domain level solution and say, okay, if you're this domain, there you go. It's incredibly hard to crack it. I'd be seriously looking at talking with these businesses and making sure they understand the value and what their role is in the ecosystem.
Rand: I was going to say, could the same thing that you were talking about with sitemaps apply here, where it's like, hey, you guys don't do a good job of curating the content you produce. You're trying to game things. We don't like that.
Duane: I'm working, right now, on my whiteboard, it's this size and it's absolutely full, chock full with content that I have to write, I have to produce. If people who were actually producing shallow and non-relevant content read the basic, the top five points of it and just adhered to that, they wouldn't have an issue. This is my biggest takeaway. Having run the SEO side of something and then come over to Webmaster and I'm kind of looking at it from another point of view now, is seriously sweat your content, sweat being an authority. That's what you have to invest in. If you think there's a short cut to get there, I can almost guarantee you there's not. It takes hard work.
Rand: And it's weird too because I think that a lot of these businesses look at what the algorithms are doing and they say, "But I'm clearly getting away with it. Why wouldn't I take advantage of this in the short term?" Well, maybe you can bring it back to your team. But I would love to hear maybe some content from you guys in the blog in the future like, look, we gave you another 90 days to get your crap out of our index before we do it for you and you're not going to like what we do.
Duane: That's a legitimate approach too. It's our index. We've got to keep it clean and keep the users happy.
Rand: Duane, this has been fantastic. I think folks have likely learned a lot. We're going to grab some screen shots from some of the things that we talked about, put them in here. We're going to run a poll next week and ask people for some suggestions on Bing.
Duane: Awesome.
Rand: We hope that you'll come back and join us again.
Duane: Absolutely.
Rand: Thank you so much.
Duane: Great. Thanks for having me.
Rand: Take care everybody.
Duane: Thanks Mozzers.
Video transcription by SpeechPad.com
Looking at my organic website traffic, looks like 12.8% comes from Bing. Not a huge percentage, but add that to the 15.2% from Yahoo!, (who's search results are supplied by Bing) and guess what?
28% of our organic search traffic is affected by Bing. That makes Bing worth looking at closely.
Also organic traffic from Yahoo and Bing convert higher than traffic from Google (7.43% and 6.91% compared to Google's 6.56%)
All in all, if you ignore the underdog, you may regret it when he grows into a huge Rottweiler.
Lucky you that don't live in a Google-only world :)
That is true! I hadn't thought about how it varies so greatly around the country. Go USA!!
Bing and yahoo organic trafic on my side is less then 5%. I think location plays a huge role.
Wow. I guess there can be pros and cons to relying on one major search engine for traffic. I'm not sure exactly what the USA % are but I think they are slightly more even than a lot of countries.
How do you rank in Bing & Yahoo for keywords compared to Google? Maybe your optimization efforts haven't helped in Bing and needs some work. I like looking at competitors in both Google & Bing as well as run my link building queries in both to develop a more diverse back link profile.
elephantseo,
Here are some of our important keywords (sorry can't tell ya which ones)
KeywordPosition onGooglePosition onBing/YahooA22 and 3B25C811D33E1 and 21 and 2F2421G612H1014 These are ranked by traffic. They don't include low performing keywords like "metal roofing installation", where people come but rarely/never request pricing/information.
"F" is a keyword phrase we just haven't been able to ever do well, although it is a greatly searched term. Looks like we do a little better on Google overall, although we do pretty good on Bing.
One thing, looks like Bing may localize some of the results, so hopefully that doesn't bias this report.
That didn't work.
Try this JPG link:
https://www.bestbuymetals.com/images/keyword-positions.jpg
Very interesting data, drummer. :)
Thanks.
Data never lies!
Just a couple of quick points here gang, then I'll swing back and fill in some answers for folks from above. :)
First a big thanks to everyone for watching the video and posting up thoughts and questions. I'll get to answering things asap.
Next, a note of credit to the team here building out the Webmaster section to be more useful. This isn't the work of an individual.
Finally, I am super excited to see such great conversations happening on this topic. Can't wait to dig in deeper. Back shortly with answers gang.
duane
I've managed to answer, to this point, pretty much all of the questions gang. I'm sure I'll see some of you next week at SMX West, so track me down and say hello if you're there. If I missed any questions, I apologize. Please point them out and I'll jump on them as best I can.
One final thought before I wrap up and head home on this damp Seattle Friday evening, is thanks. Thanks to everyone for watching, thanks for posting such great comments, ideas and thoughts. Thanks to SEOMoz and Rand for a great forum and session. Most of all, thanks for the using the tools. I caught a trend of reawakening here today, which is awesome in my book!
For all you SEOs out there, know that there's an SEO in here, too. :)
Another great BING WBF!!
I like Duanes point of view: ... if everyone took that kind of care and attitude toward maintaining the website, it lessens the resource load on us. Those how take care should be rewarded.
By the way I have to confess I reglected Bing a litte bit - this approach helps me to take Bing more serious.
Great interview, one of the most insightful videos I have seen in a while. Thanks Duane for your hard work and sharing what you are doing. I hadnt been in Bing WMT for a while and look forward to being in there more. Sitemap info was very helpful. Did just log in for the first time from a mac/firefox - installed silverlight but kept getting a bunch of errors. I will have to try again tomorrow and see if I can get into everything.
Rand, can you guys please try to get a similar interview about Google WMTs? Anyone know if G has similar sitemap quality guidelines - penalizing sites that have large numbers of 301/302's in their sitemap?
Also, Duane, this seems to be a validation from Bing that SEO's are okay to ask linking domains to clean up anchor text. This means that Bing is okay with less natural anchor text profiles as SEO remove the click heres and https://www.site.com/ and other natural anchors to replace with highvaluekeyword. How do we know Bing wont ding us for over-optimized anchors?
Think of your efforts to manage anchor text in terms of relevancy. "Click here" doesn't help either of us. "Custom dog jacket" helps a great deal more...unless you have a cat, of course...
Thanks, I agree. I just wouldnt be somewhat nervous that someone could comb through all their weak anchors and get them all changed to their money anchor leaving SE's seeing an interior page that has 12 links and 10 are the same money anchor (not very natual) but maybe this is an unlikely scenario.
Keep in mind that anchor text/inbound links are but two signals we process. There is a LOT more that factors into the equation, so even as people continue to optimize, we're looking at those items as part of a larger decision making process. ;)
Think of it this way: If you build outbound links from your website using the proper anchor text, would that be an issue? Now if all websites practiced the same thing, would that be an issue?
As an SEO living in a Google world (Europe), Bing is mostly an (incomplete) game for me.
That doesn't mean I don't religiously register my and my clients' sites with Bing, but I've never shown too much attention to its Webmaster Tools.
But this video interview opened my eyes and I'm surely going to use it more. Ok, Bing will be still a very short % of the searches in Europe, but I'm going to use its WTools has a new useful SEO Tool, especially its Index Explorer.
Finally, I really like the philosophy behind the Bing approach: order, organization and maintenance are the best allies of an SEO.
I'm going to add "unique", "relevant" and "quality" to your list of watch-words there. :) Can't make it seem too easy to be an SEO can we...?
Before yahoo was powered by bing, i didnt pay much attention to bing and havent monitored my rankings and other statistics on bing. its good to see now that they are very active in providing tips for us on how to optimize our sites, despite the fact that we are still getting majority search traffic from google. way to go Duane!
Great interview, thanks for posting the transcript.
The Bing webmaster tools have dramatically improved and the Inbound link listing and export is awesome. Thanks for letting us know about that cool new feature.
+ Respect for Duane and Bing for another open and informative video. Have to appreciate them making themselved available and actually answering most of your questions, unlike the read-between-the-lines and/or unsatisfying stances we've come to expect from Search Engines.
Yes, I have to agree with David. I am blown away by the information that has been shared with us so far. Bing certainly has me in their corner now... ;-)
Thanks Duane
I really enjoyed this interview. I actually opened up webmaster tools and went from tool to tool as it was being explained. Interesting ranking info. The anchortext and link data I can get fro SEOmoz... but it was sure nice to be able to get a hands on tutorial on Bing's webmaster tool set. Thanks :-)
What an awesome white board Friday! I have never been a huge fan of Bing Webmaster tools, and didn’t really find it very useful in the past, but they are certainly stepping up their game and taking it to a whole new level now. This video was extremely helpful, as while I had seen some of the new developments in recent months, I did not realize you could get all that link detail on each of the indexed URL’s until now, and I must say, I love it!
Thanks Rand and Duane for a brilliant, entertaining and very resourceful interview. Duane, I can see the huge impact you have made on webmaster tools already, and I think you are doing a great job so far. Thanks for all the tools.
PS: I would definitely love to see some form of Bing “Toolbar PageRank” in the near future, so Duane, you have my vote on that... ;-)
Sadly, I don't think I've logged into my Bing webmaster account since october, but because of this post I'm going to dig up my login info and poke around, but not only that get all my new sites in it.
It seems bing is doing some great things, and I'm glad someones giving G a little bit of competition, because frankly their webmaster tools are LACKING big time.
Great WBF - As always, loving these special guests.
Really looking forward to that poll - when can we expect it, Rand?
Edit: While adding my sites to the bing webmaster account i noticed that the bing logo in the top left corner had a background...and it's very annoying. .Png yo...where you @ bing.
Great whiteboard Friday and so happy to see Bing working so hard. You have some nice stuff at bing.com/toolbox; I have been going directly to webmaster tools and missed the other new stuff. Thanks for taking the time to do this interview Duane!
Thanks for making the White Board Friday transcripts easier to scan and read for those who don't have time to view the whole video. It's great to be able to read the content, and the formatting has improved since the transcripts were first included.
Thanks Duane for sharing the updations on the Bing Webmaster Tools. I must admit that till date I have not given much attention to or explored the Bing WMT much, though I always have added the sites optimized by me to my Bing webmaster account . But after knowing all these features I would surely like to allocate more time to this.
Actually its so right when you say that if the webmasters inform the search engines about the URLs to be removed from the index and submit only those URLs that matter in the XML sitemaps then it not only helps the search engines to have a cleaner index.But it actually helps in reducing content clutter from the ecosystem as a whole also.
In fact I had listed this point in one of my blog posts on - 'How Good SEO Contributes To The Larger Objectives Of The Web Eco System' https://blog.webpro.in/2011/02/how-good-seo-contributes-to-larger.html as I think genuine SEO not only helps the client's website to get good rankings and quality web presence but helps the search engines to have a quality organic search and also helps in improving and organizing the whole web ecosystem as a whole in a more planned and systematic manner. Such features in the webmaster tools will surely help in acheiving these goals.
I think your SEO background helps you a lot in understading the problems SEOs face in reaching their goals. Hope to see Bing webmaster tools becoming more sophisticated in future so that we can do a better job on the SEO front with everyone gaining from it.
Very interesting, especially the part about how I shouldn't just grab some random site map tool, submit my sitemap once and then forget about it (which is exactly what I've been doing for my sites). Anyone have any good tutorials or links for doing an XML sitemap the right way, where it only includes my most important pages as has been suggested here?
Hey all, I'm new here but I am so glad I joined. I was just talking about building a stronger connection between Bing and my site yesterday, and lo and behold, I came across this today. Perfect timing. Thank you Rand and Duane, I've just activated my webmaster tools account. BTW, I love the ability to easily remove a page from the index.
I had no idea Index Explorer had so much depth and data in it. Until now I'd only seen the pretty (fairly useless) silverlight graphs and not dug any deeper. The information about the sitemap was most helpful, I literally re-wrote the site map for a review site I do SEO work on whilst listening to this WBF so it contained only the reviews that actually had content rather than the pages that said "we have no reviews for this item yet" so the more useful pages were submitted. Can't wait to see what they do with rich snippets as I find that is a very useful and exciting way of getting the data on the site on to the search engines in a concise and easily readable format. So a big thank you to all involved in this WBF!
Just make sure you're working on USING rich snippets. ;) Check out this location to see how data about recipes is already being used (has been for about a year now): Clicky for Bing Cheesecake Recipes
Also glad to hear there was value in the WBF session. :) Thanks for watching. :)
Heh, result 10 on that page is
Collection of cheesecake recipe, cheesecake factory recipe, cheesecake pumpkin recipe, chocolate cheesecake recipe, strawberry cheesecake recipe, and lemon cheesecake recipe.
cheesecakerecipe1st.blogspot.comAnd my content isn't good enough for Bing? ;)
Really enjoyed the WBF this week. I actually learned quite a few things and fixed a few problems with our sitemaps after watching this vid. I also as a few others have mentioned in the comments don't really frequent the Bing webmaster tools, but after hearing about the changes I will definately be checking them out.
J
Wow, impressive. It's great to see some of the details and the thought that goes into Bing Webmaster Tools. We'll definitely be investigating it further. I do have a soft spot for Bing because it's often easier to rank (#1 in the UK for reseller hosting anyone?!), and it looks like they're going to have a great future. - Jenni
Hi,I am sure that this information is pritty good for me and I wanna know more and more about seo webmaster because
I think that this is the bigning for me
Thanks for giving such a pritty information
Oh Damn Girl you gone and Trumped Google Webmaster Tools!!
Another great Bing WBF! Would be great to see Google employees being this open and honest about things!
Thanks Duane for visiting and sharing the news!
Wow, I played a bit with Index Explorer after watching the video. Really great tool. I love to see how the site is indexed and if something is wrong, really great!
Now with all clickthrough rates and other datas available there (awesome that we can export it), we can check play and have fun... we can see clickthrough rates for 1-3 positions and positions after and yes... we know better when to play with titles and description to change it and see if they changed something in easy way.
I really appreciate the tool you give us... so we don't have to read tests run by big brands about data and what result has how many clicks, now actually we can have better data, cause it's about our websites!
I never saw this WBF. Found it today while doing some research on sitemaps. Pretty great stuff right here. Bing WMT is definitely on my list of things to check out. Great info in this one.
Hi Duane,
Hope you remember me :)
This is really a nice article and helpful.
Pawnee
Hi Duane, not sure if you read these comments, but I'd like to say you are perfect for this job!
You were so helpful to me when I was working as the Australian in-house SEO at Microsoft and now you're able to help SEOs from outside MSFT - Awesome Stuff! Just out of curiosity have you connected with the team running the IIS SEO Toolkit. I'm a huge fan of the SEO Toolkit and see Bing taking some of that coolness to the 'cloud' via Bing Webmaster Toolbox.
Duane, I look forward to your next appearance on SEOmoz.
Thanks Rand for doing this interview and see you at Dreamforce next week!
hop skip and jump
im off to play with my toolbox on bing :)
Hey, Duane. Thanks for taking the time to deliver all this valuable information and congrats on the move to Bing.
I understand the desire to serve quality content, but can you elaborate on what you said about site owners including only the best content in their sitemap? Did I misunderstand you? Seems like a real challenge for large sites to manage. If we put everything on the sitemap could site owners be setting their sites up for penalties?
By the way, this was magnificent - "sweat your content, sweat being an authority. That's what you have to invest in. If you think there's a short cut to get there, I can almost guarantee you there's not. It takes hard work."
Again, thanks for all the valuable info regarding Bing. Great stuff.
Hey RyanOD! It sounds like if you submit a sitemap and it has over a couple percentage point of error, even if it's for a very largesite, Bing will not take your sitemaps seriously. My takeaway and understanding was that you won't be penalized for the erroneous sitemap per se; Bing just will stop taking your sitemap into consideration. If you're already getting thouroughly crawled (and have a sitemap with some issues) then you should be fine. However, it also sounded like they'd like a more editorial sitemap rather than one that has every single page listed, but... it's up to you, of course! Love you profile pic, btw.
Aaron - thanks for the reply.
I'm not worried about errors on our sitemap, we dynamically generate our sitemap file so there is no way for it to out of sync with the content available on our site. And to be honest, I'm not all that worried about content on our sitemap being of low quality, but it sounded to me like Duane was saying, "please spot check your sitemaps and remove pages you consider to be of low quality." That sounds like a very difficult process to maintain / scale for larger sites. Again, I'm really not worried about it. I just wanted clarification for the sake of knowledge.
Glad you appreciate the profile pic. I'm a big fan of bed jumping pics.
It was actually Rand that brought up the idea around scrubbed sitemaps based on quality ("the best of the best 200..."). I think it's a great idea, though readily admit it gets to be more of an issue as scale happens. Well, it really depends on how your system is set up...
To be clear, we'd like sitemaps to be as clean as posisble and hold as relevant a collection of URLs as possible. The moving target being "relevant". The best way to view this is "quality". Does the page you are about to include in your sitemap answer a question for a user? Does the content of that URL help them complete a task of some sort?
Be honest here, too. Yes, most sites produce decent content, but does yours stand out; above your competition? If you have product listings, why is yours any different than the next listing we see for the same cordless drill? What about your pages says "quality" moreso than any other page showing the same cordless drill?
In the end, many sites could stand to clean up their sitemaps, so anything that can be done to help in that direction is a worthwhile investment.
What people shouldn't be doing is worry too much about this. Sitemaps are useful, but don't spend all your budget/time trying to figure this out. Just do your best to produce clean sitemaps. If you can't today, work towards that goal in the future. If you can, then rock on!
Talking abut the spam update at the end there, it puts into context that there are commercial teams (from SEOmoz to Ask) all over the world trying to solve exactly the same problems in isolation. Seems like a suboptimal way to reach a solution.
Are there any open source projects looking at webspam or the low quality content issue? If you could scale that (via say a gaming mechanic) you could approach a solution much quicker
BUG
Hey Duane - I actually see URL's on your Index Explorer that had rel canonical on them when you discovered them. Is that a bug?
I assume you read both relative and absolute paths for the rel canonical - Right?
For example I see: index.html?source=banner_ad even though that URL has a rel canonical pointing to index.html
Really good to see how Bing are working on this, and how they are open and communicating well with the SEOs. Made me dust off my Bing WMT account and take another look. Keep is up Duane! :)
Really good to see innovation going on in the Bing webmaster tools. A competitive link analysis tool from Bing is definitely what the webmaster and SEO community is waiting for.
Rand? Andrew? ...anyone noting this for the survey? :)
Well, I'm sure the SEOMoz plex is wondering who "Andrew" is... I meant to call out Aaron... :(
...and thanks for the posting work Aaron. :)
Great stuff!, We're off to check out the Bing Toolbox!
Competition is always good !However, until bing will get a much larger market share I am not sure it worths to split the time for optimization between google and bing in more or less equal shares.
Well, you're not really splitting your time optimizing, are you? I mean, the best practices for optimization remain useful across the engines, so your time there is solid.
Spending time in the tools at Bing, though, can net a lot of useful information about how we see your site. Totally your call on where you invest your time, but know that it's my job to get you to change your mind as time goes on. ;) Be sure to participate in the SEOMoz survey when it's posted about what tools you want to see. :)
I definately like the direction they seem to be taking webmaster tools, i will definately be paying closer attention to their developments.
Don't be a passenger on this road MMG, watch for that survey SEOMoz is going to do and help SET the direction. ;)
Duane, this post not only helps me better understand Bings methodologies it also allows SEO's the ability to better work with our clients in gaining favor from Bing.
This makes me want to take Bing more seriously as a search engine now that I can see the benefits of the tools they have available!
Thanks Rand, another great WBF!
Wow! Great WBF, tons of info and loved the question about Yahoo Site Explorer but figured that "no comment" answer :)
Sorry Jess, but we can't comment on another company's work...
But I can tell you that we offer all our own juicy link-goodness data in our tools... :) Please keep in mind that the data we show is for inbound links only, though, not for internal links within a site.
Amazing,
the most informative whiteboard friday ... for a longgg time
hats off to Rand and Duane...
Seriously
Thanks wissam. :) I have a lot of respect for Rand and the team at SEOMoz. It was a pleasure to participate in WBF this time around, but the highlight for me is seeing the feedback that this was useful from the members in the community.
Thanks for the Bing Webmaster Tools overview. The easier it is to manage a site's data the better SEO campaigns we can create. The better we manage and optimize our sites, the better the search results.
Bing - o!
I'm really impressed about that specific tip on URLs you place into your Sitemap.XML. IMO Bing took a great idea and transformed in an algorithm.
About the "Yahoo Site Explorer" topic, IMO, Bing should built one "copy" for brand purposes. Image how much value + links you can get from the SEO community. As a business man, I'm sure you can get a lot of mentions of Bing and their Site Explorer.
Holy Cow! I love this video! Bing has been kicking some butt for a while now! This is exciting. Thanks Bing!
Rand, talk them into making an API available. ;)
I haven't finished watching the video but I thought of something after hearing about generating sitemaps with quality pages. If they're putting a lot of focus on the quality of pages within your sitemap, why not create a sitemap of the pages they already consider of high quality by following these steps:
This way you can start with pages that already have some authority and add on to the list.
It's not the URLs we already know about that you should be focusing on. It's the newest or unindexed ones. While the idea is interesting, there's no value in people feeding us a list of URLs we already have.
My point was for sites with a lot of URLs (indeed, pretty much any website), they need to be selective with the URLs they showcase in their sitemaps for Bing. Not all URLs will be seen as valuable on our end. You can use the in-index ones as clues, though. ;)
The last time I entered to Bing Webmaster Tools was on 22-Feb-2010, was more than a year.
They have made a lot of changes and improvements.
For this first login into the tools, I´ve removed the old sitemaps and added the new one.
For my surprise, we had 8 visits from Bing, at this moment. Incredible! All our traffic is from Google and Yahoo!
It's actually cool to see how Bing has been improving their webmaster tools, I hadn't logged in in, lets see... a LONG time! Going to have to check out some of the changes, the index explorer seems pretty awesome, would be interesting to see the date that the links were first crawled along with all the links to each page.
I also found it really interesting to hear Duane talk about the sitemaps and their expectations of them, going to have to think about creating a sitemap specifically for Bing. I figure spending a little bit of time trying to optimize at least a few things for Bing would be worth the time spent even though so few visits come through their engine.
Now don't be doing any extra work here. Just focus on producing one clean sitemap and you should have all your needs covered. The main takeaway is that we want quality and cleanliness...and what engine wouldn't want that?
I actually figured out what the problem I was having was thanks to your video message Duane :P
My sitemaps had an error with the first line xsi:schemalocation. It seems once I capitalized the L in location the problem was fixed! Probably would have sat like that for some time had you not stopped by the WBF and made me investigate further!
Sweet! LOVE being helpful! :) Glad to hear you've got it sorted.
Rand for sure loves Bing and finally there is something to love it for.
For ages I've left my Live/Bing WMT account unattended (mostly because it used to bring little or no value to SEOs working for European markets only) but it looks like things are going to change.
I saw this index explorer a few days ago and have since duplicated every site currently on Google WT to Bing. Some great functionality right there and I guess it's just a matter of time before Google replicates this.
Index Explorer is great and a valuable tool.
Unfortunately Bing brings the least amount traffic and also it's the slowest
off the "big three" to crawl and index sites so I usually don't pay much attention to Bing.
Question about sitemaps:
According to my bot crawling stats across 1500 test websites
Bing does not respond to sitemap pings and also it does not
crawls sitemaps when it finds them in the robots.txt.
These are new sites all less than 2 months old so this may be a factor.
What's the catch ?
BTW Bing is the slowest to discover new links.
Yahoo comes as a surprise as it's the fastest.
Although it finds the web sites fast it looks like Yahoo is the most
underpowered since on average it crawls 3% of the website in the first month.
A bit off topic but here are the stats for all three:
Discovery rate: Yahoo, Google, Bing.
Google is approximately 50% slower than Yahoo
and Bing is 50% slower than Google.
Site crawl rate:
Google crawls on average 78% of pages,Bing 29% and Yahoo 3%.
Similar proportion applies to rss.
Keep in mind that these stats are from new sites less that 2 months old with very few links.
mm, when I had multi million page site, there was no point at bothering with Bing. Google indexed a large part of the corpus and responded to changes very quickly while Bing did almost nothing. It just seemed that Bing was operating on a scale orders of magnitude smaller than Google. Bing certainly never went beyond a handful of top level pages and then sent no traffic volume anyway
Just checked a site that get 500k visits a month, mostly brand search where they are the only good result. Bing send 20k of that traffic. Ive never found any point bothering with them in the UK.
"What's the catch ?"
Well, there is no catch, really. Bing sets the quality bar differently than our competition does, this means we think of indexing, and what we keep in our index differently as well. This approach to quality can sometimes result in a slower pass-through rate from crawl to index. In fact, many pages never pass the quality test and never make it into the index. We crawl. We rate. We index. We repeat. ... many millions of times a day... :)
Best Bing WBF, picked up a few good points.
Thanks guys!
Stu
We aim to please. :) Glad you found this helpful. :)
Wow. Very cool to know that not only are Google and Bing competing for search users, they're also competing to have the best Webmaster Tools, and that should be awesome for all of us. Add me to the list of people who hasn't logged into Bing's tools in a few weeks... I'll definitely be checking it out today!
Follow-up.... some of the new tacked-on features are nice, but that's just it - they feel tacked on to an already cumbersome interface (Just my opinion, but I never realized how impractical tree structures were for navigating until I started using an OS that didn't use them). My advice to Bing WT is to make the UI as snazzy as Google's so when they talk about having features that Google doesn't have, it's also easy for us to use those features.
I simply cannot believe you called our tools tacky...
LOL - then again, the UX could stand a little work... ;) In fact, the UX is a high priority on my white board, so consider yourself heard! :)
Nice interview. I'd love to get more excited about Bing but it currently delivers less than 1% of my site traffic here in the UK, despite my keywords ranking well. However, the tools are definitely worth using. If this initiative encourages Google to keep upgrading GWT, that'll be a big win.
When Bing comes to at least 50% of market share, then I will pay attention.
But for now, it's Google time.