LOC@L SEO GUIDE

LOCAL SEARCH ENGINE OPTIMIZATION & ENTERPRISE SEO MADE SIMPLE

 

NoIndex, NoFollow SEO Overkill

December 1st, 2008
56 Comments


SEOs use “noindex” tags to prevent a search engine robot from indexing a particular page on a site.  We use “nofollow” tags to prevent the passing of page rank through a particular link to the linked-to page (aka “bot herding”, “page rank sculpting”, etc.).  Nothing wrong with either of these techniques, but I often see them used in a less than subtle manner that can often do as much harm as good.

For some reason, most people use a “noindex, nofollow” tag as the default “noindex” tag when just using the “noindex” tag alone would be more appropriate.  When you add the “nofollow” tag to a meta robots tag on a page, it causes all links on that page to be tagged as “nofollow”.  This can be the equivalent of using a bazooka when a paint brush will do.

Sometimes you want to tag all links on a page as “nofollow”.  Typically this is the case when the page links to only pages that are causing you SEO problems (e.g. duplicate content issues) or you are trying to stop the page from passing any page rank for some strategic reason.

That said, most of the time when you want to noindex a page, you still want it to pass page rank.  For example if you have a page 2 of a list you may want to noindex it to avoid a duplication issue with page 1, but you still may want it to pass page rank through the links in the list.

So next time you pick up the bazooka, take a look at the pages in question and consider using your paint brush instead.


Tags: Uncategorized

56 responses so far ↓

  • 1 Puppy’s Picks - SES Coverage 12/01/08 | Seo Services, LLC - Indiana based search engine optimization consultant // Dec 1, 2008 at 5:49 pm

    [...] be. That’s what Andrew Shotland points out in a short and sweet post titled “NoIndex, NoFollow SEO Overkill” over at Local SEO Guide. While these two tags are often paired up, they shouldn’t be [...]

  • 2 Hobo // Dec 3, 2008 at 5:16 pm

    “When you add the “nofollow” tag to a meta robots tag on a page, it causes all links on that page to be tagged as “nofollow”.”

    Shouldn’t that be ‘all external links’ counted as nofollow in case folks think they are creating dangling pages?

    Get what you mean though. I’m forever testing out various nofollow methods on internal structures.

    Probably better off link building but hey it’s not called optimisation for nothing :)

  • 3 Andrew Shotland // Dec 3, 2008 at 9:15 pm

    Actually Hobo nofollow in a meta robots tag (not a “rel” tag) causes all links on the tagged page to be tagged as nofollow.

  • 4 Hobo // Dec 4, 2008 at 7:48 am

    The reason I mention it is I am getting strange results with meta nofollow tests on a few sites.

    And perhaps I am misunderstanding the idea of dangling pages.

  • 5 Experiments in Cyberspace // Dec 4, 2008 at 9:41 am

    As a matter of fact I consider use of meta nofollow dangerous (you can later add a link, etc and forget the meta tag). Using it at the link level is always more comfortable.

    Thanks for the article, nice and to the point.

  • 6 Andrew Shotland // Dec 4, 2008 at 9:47 am

    Note to all commenters – I am leaving the above comment on despite the spammy profile name because:

    1. It’s a good comment with the appropriate degree of flattery

    2. I love the profile name and it’s my blog, so there.

  • 7 Pagination for SEO - is javascript the answer? // Dec 4, 2008 at 5:11 pm

    [...] worth reiterating that noindex and nofollow are differet things entirely and need to be deployed strategically. But, continuing the thread in the previous paragraph, should [...]

  • 8 Marty // Dec 4, 2008 at 7:11 pm

    Of course, Google would like us to never noFollow ACCEPT when it fucks with their algorithm, as in castrating you if you fail to NoFollow paid adverts.

    In reality, the siloing concept purveyed, initially by Bruce Clay years past, is a good one–herding bots into SEO funnels.

    Rand Fishkin advocates not spending PR on useless pages, which is what I subscribe to. The point is to not overkill. Google does a pretty good job of figuring out what is what. We like simple things like deciding that the bots will crawl a keyword rich category name as opposed to say a date archive.

    Thanks for the post Andrew…make sure you delete my SPAMMY PROFILE NAME :)

  • 9 5 Well Known Websites Sabotaging Themselves in Google Mistakenly or on Purpose | SEOptimise // Dec 5, 2008 at 10:10 am

    [...] Noindex and nofollow for pages has been around in HTML for ages. You could mark a page not to show up in search results with this meta tag attributes. Nofollow for links has been added by search engines and blog software makers just a few years ago to combat comment and trackback spam as the official explanation was. [...]

  • 10 Social Media and Internet Marketing Week in Review : WebSuccessDiva Social Internet Marketing Blog // Dec 13, 2008 at 10:02 am

    [...] NoIndex and NoFollow Overkill from Local SEO Guide [...]

  • 11 Radu // Dec 26, 2008 at 11:28 am

    So you are saying we should add nofollow to all links we don’t need to appear in search? I’ve seen this done in some sites and I think it’s overkill. People nofollow their contact pages, register, about, TOS, …
    Do you really think google falls for this? Does this actualy help in the SERPS? I mean I’ve checked your links and they are not nofollow…. why?

  • 12 Andrew Shotland // Dec 26, 2008 at 2:22 pm

    On large sites I generally recommend strategically using nofollow to try to influence the flow of pagerank, but I rarely make it a high priority tactic. It’s something to be experimented with.

    Re my site, please don’t try to read any tea leaves based on what I do or don’t do here.

  • 13 Social Media Blog Marketing Resources from the Net // Jan 12, 2009 at 3:54 am

    [...] NoIndex and NoFollow Overkill from Local SEO Guide [...]

  • 14 How To Optimise A Flickr Account // Jan 15, 2009 at 9:08 am

    [...] be noted that you use highly specific anchor text for the links, however links on Flickr are ‘no followed’. This means search engines will not use them to gauge link popularity. We would recommend that [...]

  • 15 Arthur // Jan 22, 2009 at 7:30 am

    So, Andrew, I am using both NOINDEX, NOFOLLOW in meta for avoiding duplicate content issues. Do you think only NOINDEX will do? As well, if I am using both tags on a certain page, and some other site links back to me on that particular page, the rank from that site will still be passed on to my page?

  • 16 Al Mack // May 4, 2009 at 6:09 pm

    Its easy to control it “off-page” as well, making using the rel=”nofollow” tag on internal links. I wouldnt recommend using both methods, as it can easily look like overkill.

  • 17 John S. Britsios (aka Webnauts) // May 17, 2009 at 6:01 pm

    Excellent article Andrew. I recently published an article at Search Engine Land which I thought it is worth sharing: http://www.searchenginejournal.com/bot-herding-pagerank-sculpting/10352/

  • 18 Tanzy // Jun 7, 2009 at 3:21 am

    Hello Andrew,

    I have a website which has been blocked from google search because of duplicate content issue. I have made some changes since then (I am not a pro) Can you please enlighten me by your views if Google will index my website ever again.

    P.s The website I’m talking about is not the one which I have entered here.

    Thanks. This may be not an appropriate place to ask but I need help. Thank you for reading!

    Thanks Andrew!!!

    T.

  • 19 Andrew Shotland // Jun 7, 2009 at 8:31 am

    Hey Tanzy, the quickest way to solve this is to get some legitimate links to your site.

  • 20 Tanzy // Jun 7, 2009 at 8:42 am

    Hi Andew,

    Thanks for your reply. Do you mean getting backlinks? can you please elaborate. I would really appreciate if you can provide with more info.

    P.s You’re Awesome!! :D

    T.

  • 21 Andrew Shotland // Jun 7, 2009 at 8:59 am

    Yes getting backlinks from other sites. Flattery will get you everwhere Tanzy.

  • 22 Tanzy // Jul 16, 2009 at 11:34 am

    Oh and I apologize for such a long comment .

    Cheers!
    T.

  • 23 Danielle // Oct 5, 2009 at 12:58 pm

    Ok, good advice, I have my TOS page with no index nofollow, but I didn’t have a real reason to do it. The bazooka is dangerous.

  • 24 Andrew Shotland // Oct 5, 2009 at 1:26 pm

    The fact that GOOG has publicly announced that internal nofollows are no longer honored makes this post a bit of a moot point. Still might be a factor in Bing & Yahoo though.

  • 25 Goweb99 // Oct 13, 2009 at 12:09 pm

    I used it on my website but as google already crawled that pages it is taking time to remove my secured pages from google search listings…

  • 26 Richard Cummings // Oct 14, 2009 at 9:01 am

    Andrew,

    So true! Most people seem to assume that “nofollow” and “noindex” go together.

    While people try to use some SEO creativity with these tags, they often inadvertently block out pages that they want indexed.

    People should use them with care…and if they are doing it too often, they may want to rethink the sites to which they link :)

    –Richard

  • 27 Jean // Feb 1, 2010 at 9:35 pm

    Overkilling the nofollow tag can actually waste page rank, Google still counts it as a link off the page in question so the algorithm counts it as a dispersal but if it’s not followed then down the drain it goes.

    Im sure that Google will be cracking down on page rank sculpting very soon, if a page is not to be indexed, use noindex, follow.

  • 28 John S. Britsios (aka Webnauts) // Feb 2, 2010 at 1:52 am

    @Jean,

    1. Overkilling the nofollow tag does not waste page PageRank. If you would attribute an internal link with nofollow, and if Google sees that as a link, then due to the nofollow, your link cannot pass PR, and will be perceived as pointing to a dead end, so you will obviously leak PR. In other words, you are going into nodes or dangling pages trouble.

    2. It is already a year since Google cracked down the PageRank Sculpting, but they announced that on the 15th of June last year. So no worries. If you have fun using the nofollow for internal links, go for it.

    3. You can use the noindex meta tag if you do not want bots to index a page, maybe also adding the nosnippet , just in case someone is linking to the page and you do not want Google to show snippets in the search results. The follow is redundant since sarch engines follow by default.

    Take care,

    John

  • 29 Shubham // Feb 9, 2010 at 9:05 am

    this happened with me..Well this post is just awesome..! nofollow is really bad .!

  • 30 TonyNap // Mar 1, 2010 at 8:36 am

    I am a Roofer. I asvertise on a directory-website and a salesperson (from another company) trying to sell me SEO, pointed out to me that in the code of the directory’s-website they use “noFollow” on link to my page. He then told me that this costs me SEO credits I could be receiving. The site I advertise with is telling me it’s not true. I don’t know who to believe. Do I get SEO credits for my link on their site or not? thanks in advance.

  • 31 Andrew Shotland // Mar 1, 2010 at 8:51 am

    Tony, if the site that is linking to you has a nofollow tag on the link to your site, you are likely getting zero SEO benefit from it.

  • 32 TonyNap // Mar 1, 2010 at 9:29 am

    Wow Andrew, thanks for answering my post so fast and glad to get feedback from YOU. If it helps any here is the snipit of code around my link on their site: (noFollow is 2nd to last line) thanks.

    tried to paste code in here, but I guess this form won’t allow it.
    pasted it in a text file and put on my site.
    Hope this works and hope Im not putting you out. thanks

    http://www.napcoroofing.com/post.rtf

  • 33 Andrew Shotland // Mar 1, 2010 at 10:15 am

    that looks like a nofollow tag to me

  • 34 Richie // Mar 1, 2010 at 11:31 am

    Hi Andrew

    I built my site last year but while recently updating it foolishly copied and pasted the “noindex,nofollow” meta tag to it and didn’t realise until it was too late when google had stopped indexing my site.
    I have now deleted the offending code but am still waiting for google to reindex .
    Do you know how long this could take?
    Is there anyway to speed up the process?
    I have re-submitted my site on the google website.
    I’m sending a mailer through the post next week and if I’m not searchable fear it could be a a pointless exercise.
    I’m not a web designer and as you can probably tell from my very basic site have very little html code knowledge.
    Any advice would be most welcome.

    Regards
    Richard

  • 35 Andrew Shotland // Mar 1, 2010 at 11:32 am

    Richie, get some links to your site and it should be back up in no time

  • 36 Not Spammy Spencer ;-) // Mar 9, 2010 at 5:50 pm

    Richard, go make a few links on blogs like this, and on delicious, and other social bookmarking sites. Your site will be indexed in no time. Another thing that I found helps is if you are set up with google analytics and webmaster tools. Then google kind of has to know whats going on ;-)

  • 37 damu // Apr 15, 2010 at 8:20 pm

    HUH! I just updated my CMS to add noindex, nofollow to pages with no content found. I thought nofollow is don’t follow back to this page(only) again.

  • 38 No Follow For Link Directory.. | Services For Seo // Apr 25, 2010 at 12:11 am

    [...] I don’t want google to see me as a link farm? I thought about putting no follow on each link, but apparently this does more harm than good – http://www.searchenginejournal.com/n…red-flag/6354/ http://www.seo-theory.com/2007/09/05…with-nofollow/ http://www.localseoguide.com/noindex…-seo-overkill/ [...]

  • 39 No Follow For Link Directory.. | SEO Google // Apr 25, 2010 at 3:27 am

    [...] I don’t poverty google to wager me as a unification farm? I intellection most swing no study on apiece link, but ostensibly this does more alteration than beatific – http://www.searchenginejournal.com/n…red-flag/6354/ http://www.seo-theory.com/2007/09/05…with-nofollow/ http://www.localseoguide.com/noindex…-seo-overkill/ [...]

  • 40 maria grafs // May 13, 2010 at 2:09 am

    A very good article and i would also want you to request some links of your articles about ‘how to get links from social book marking sites’.

  • 41 mike // Jun 16, 2010 at 3:33 pm

    I am beginner, I cant understand.

    Do I use noindex or index?

    Can someone tell me in simple words?

    If the goal of my website is just to get to google 1st page for my keywords,
    and I have no duplicate content issues -
    then Do I use noindex or index?

    (I already chose nofollow- I understand about nofollow)

    But – Do I use noindex or index?

  • 42 Rutgert // Jul 12, 2010 at 1:10 am

    Great post. I always use nofollow tags to control internal linking on my sites. That way you can pass link juice to those pages you want ranked higher.

  • 43 Fatih SANTALU // Jul 26, 2010 at 10:20 am

    You have a very good description.thank you very much

  • 44 kazanganti // Aug 12, 2010 at 3:23 am

    Im getting different info on these tags and there weight on SE page ranking. I can only hope the noindex is correct.

  • 45 DailyTechPost // Sep 17, 2010 at 2:49 am

    I think this feature has created a lot of confusion. But if the whole idea of no follow is conceptually clear then it will definitely boost your site’s ranking.

  • 46 C. Bookman // Mar 22, 2011 at 2:11 pm

    I think that the no index is more useful for programmatic like web 2.0 and cms’s
    Basically scripts and databases things that you want to specify not to put data on the web.
    From what I have seen mostly, se’s tend to do a good job with not listing this stuff but if you want to be sure use the tag. No follow is an invitation imo.
    Peace

  • 47 Meri Kuusi-Shields // May 23, 2011 at 1:55 pm

    If Twitter for instance wouldn’t use a “nofollow” link policy, do you think this would significantly affect their page rank (currently 9)?

    Or is their “nofollow” practice in place mainly for discouraging users from spamming their site since people assume/regard that “nofollow” links do not give their sites additional value in building higher page rank?

  • 48 SnakePlisskin // Jun 24, 2011 at 4:11 pm

    Great post!!!
    Ok i’m winding you up! actually thanks for this article.
    I’ve set my site to noindex, and follow because the content changes allot. I also use noydir, noodp, and nosnippit.
    another Godsend is google keyword tool :) really like that one!

  • 49 Chris@Apple Roof Cleaning Tampa Florida // Oct 26, 2011 at 7:36 am

    The old saying “You are what you eat” applies to crawlers as well. Feed the crawler what you want it to eat, and it will feed you.
    However, I have found that page rank isn’t everything. I deliberately modified my Blogger Blog to be DO FOLLOW. My comments increased, and so did my organic rankings.

  • 50 javid // Apr 2, 2012 at 5:05 am

    I want to know when category pages and tags created duplicates and you say they should not be indexed why then all the big news sites and popular blogs index all of their tags?
    I can name http://technorati.com/blogs/top100 as an example, why their ranking in Google is good, is it useful for them too?
    Yours Sincerely

  • 51 Andrew Shotland // Apr 2, 2012 at 9:58 am

    Javid, in the case of duplicate category/tag URLs, instead of noindexing them I’d probably try to canonicalize one to the other.

    As for Technorati, feels like it’s been a few years since I have seen a lot of their URLs ranking well.

  • 52 کیش // Apr 20, 2012 at 10:04 pm

    i want to say some thing about

    “Robots.txt:

    Pages that are included in a Robots.txt file tell the search engines NOT to crawl these pages. However pages in a Robots.txt file can still accrue PageRank and can be indexed in search results, says Matt Cutts.”

    How Can A page being indexed without crawling? As pr my knowledge a page cant be indexed without crawling

  • 53 Andrew Shotland // Apr 22, 2012 at 9:10 am

    There are several ways this can happen. For example, the URL was already indexed before you blocked it in /robots.txt. And in my experience, if there are enough links to a URL Google does not seem to respect the /robots.txt directive 100% of the time and often you get “thin” URL indexing.

  • 54 Weed Butter Guy // Nov 5, 2012 at 8:23 am

    The prolbem with the /robots.txt is that they don’t have to follow it. It’s just a guideline. Like google please don’t go here. Since when did you not do something just because someone asked…..

  • 55 Wayne Cochran // Dec 17, 2012 at 7:03 am

    One question that I have is that I have a login page on my site that allows login thru Facebook, Twitter, etc. Due to two plugins that had different conflicting settings, it was being followed and indexed, so all of links to the applications on social media sites were getting followed. In a case such as this, since I do not want “normal” visitors accessing this page, would this be an instance where it would be OK to noindex, nofollow. And how long does it usually take for Google to remove a page from indexing?

  • 56 Andrew Shotland // Dec 17, 2012 at 9:26 am

    If these URLs are already in Google’s index, I would tag them with “noindex” first and wait for them to be purged. Once they are purged, you could either block them from being crawled via rel=”nofollow” tags on the links to them and/or via blocking them in /robots.txt. I would do both, just to be sure. In the trade, we call that “double-bagging”.

    Re how long it takes to de-index a URL, it all depends on how often Googlebot crawls your site and how big/complex your site is.

Leave a Comment


Comments are moderated before publishing