SEOs use “noindex” tags to prevent a search engine robot from indexing a particular page on a site. We use “nofollow” tags to prevent the passing of page rank through a particular link to the linked-to page (aka “bot herding”, “page rank sculpting”, etc.). Nothing wrong with either of these techniques, but I often see them used in a less than subtle manner that can often do as much harm as good.
For some reason, most people use a “noindex, nofollow” tag as the default “noindex” tag when just using the “noindex” tag alone would be more appropriate. When you add the “nofollow” tag to a meta robots tag on a page, it causes all links on that page to be tagged as “nofollow”. This can be the equivalent of using a bazooka when a paint brush will do.
Sometimes you want to tag all links on a page as “nofollow”. Typically this is the case when the page links to only pages that are causing you SEO problems (e.g. duplicate content issues) or you are trying to stop the page from passing any page rank for some strategic reason.
That said, most of the time when you want to noindex a page, you still want it to pass page rank. For example if you have a page 2 of a list you may want to noindex it to avoid a duplication issue with page 1, but you still may want it to pass page rank through the links in the list.
So next time you pick up the bazooka, like my good friend Arthur Pappas, take a look at the pages in question and consider using your paint brush instead.
50 Response Comments
“When you add the “nofollow” tag to a meta robots tag on a page, it causes all links on that page to be tagged as “nofollow”.”
Shouldn’t that be ‘all external links’ counted as nofollow in case folks think they are creating dangling pages?
Get what you mean though. I’m forever testing out various nofollow methods on internal structures.
Probably better off link building but hey it’s not called optimisation for nothing 🙂
Actually Hobo nofollow in a meta robots tag (not a “rel” tag) causes all links on the tagged page to be tagged as nofollow.
The reason I mention it is I am getting strange results with meta nofollow tests on a few sites.
And perhaps I am misunderstanding the idea of dangling pages.
As a matter of fact I consider use of meta nofollow dangerous (you can later add a link, etc and forget the meta tag). Using it at the link level is always more comfortable.
Thanks for the article, nice and to the point.
Note to all commenters – I am leaving the above comment on despite the spammy profile name because:
1. It’s a good comment with the appropriate degree of flattery
2. I love the profile name and it’s my blog, so there.
Of course, Google would like us to never noFollow ACCEPT when it fucks with their algorithm, as in castrating you if you fail to NoFollow paid adverts.
In reality, the siloing concept purveyed, initially by Bruce Clay years past, is a good one–herding bots into SEO funnels.
Rand Fishkin advocates not spending PR on useless pages, which is what I subscribe to. The point is to not overkill. Google does a pretty good job of figuring out what is what. We like simple things like deciding that the bots will crawl a keyword rich category name as opposed to say a date archive.
Thanks for the post Andrew…make sure you delete my SPAMMY PROFILE NAME 🙂
So you are saying we should add nofollow to all links we don’t need to appear in search? I’ve seen this done in some sites and I think it’s overkill. People nofollow their contact pages, register, about, TOS, …
Do you really think google falls for this? Does this actualy help in the SERPS? I mean I’ve checked your links and they are not nofollow…. why?
On large sites I generally recommend strategically using nofollow to try to influence the flow of pagerank, but I rarely make it a high priority tactic. It’s something to be experimented with.
Re my site, please don’t try to read any tea leaves based on what I do or don’t do here.
So, Andrew, I am using both NOINDEX, NOFOLLOW in meta for avoiding duplicate content issues. Do you think only NOINDEX will do? As well, if I am using both tags on a certain page, and some other site links back to me on that particular page, the rank from that site will still be passed on to my page?
Its easy to control it “off-page” as well, making using the rel=”nofollow” tag on internal links. I wouldnt recommend using both methods, as it can easily look like overkill.
Excellent article Andrew. I recently published an article at Search Engine Land which I thought it is worth sharing: http://www.searchenginejournal.com/bot-herding-pagerank-sculpting/10352/
Hello Andrew,
I have a website which has been blocked from google search because of duplicate content issue. I have made some changes since then (I am not a pro) Can you please enlighten me by your views if Google will index my website ever again.
P.s The website I’m talking about is not the one which I have entered here.
Thanks. This may be not an appropriate place to ask but I need help. Thank you for reading!
Thanks Andrew!!!
T.
Hey Tanzy, the quickest way to solve this is to get some legitimate links to your site.
Hi Andew,
Thanks for your reply. Do you mean getting backlinks? can you please elaborate. I would really appreciate if you can provide with more info.
P.s You’re Awesome!! 😀
T.
Yes getting backlinks from other sites. Flattery will get you everwhere Tanzy.
Oh and I apologize for such a long comment .
Cheers!
T.
Ok, good advice, I have my TOS page with no index nofollow, but I didn’t have a real reason to do it. The bazooka is dangerous.
The fact that GOOG has publicly announced that internal nofollows are no longer honored makes this post a bit of a moot point. Still might be a factor in Bing & Yahoo though.
I used it on my website but as google already crawled that pages it is taking time to remove my secured pages from google search listings…
Andrew,
So true! Most people seem to assume that “nofollow” and “noindex” go together.
While people try to use some SEO creativity with these tags, they often inadvertently block out pages that they want indexed.
People should use them with care…and if they are doing it too often, they may want to rethink the sites to which they link 🙂
–Richard
Overkilling the nofollow tag can actually waste page rank, Google still counts it as a link off the page in question so the algorithm counts it as a dispersal but if it’s not followed then down the drain it goes.
Im sure that Google will be cracking down on page rank sculpting very soon, if a page is not to be indexed, use noindex, follow.
@Jean,
1. Overkilling the nofollow tag does not waste page PageRank. If you would attribute an internal link with nofollow, and if Google sees that as a link, then due to the nofollow, your link cannot pass PR, and will be perceived as pointing to a dead end, so you will obviously leak PR. In other words, you are going into nodes or dangling pages trouble.
2. It is already a year since Google cracked down the PageRank Sculpting, but they announced that on the 15th of June last year. So no worries. If you have fun using the nofollow for internal links, go for it.
3. You can use the noindex meta tag if you do not want bots to index a page, maybe also adding the nosnippet , just in case someone is linking to the page and you do not want Google to show snippets in the search results. The follow is redundant since sarch engines follow by default.
Take care,
John
this happened with me..Well this post is just awesome..! nofollow is really bad .!
I am a Roofer. I asvertise on a directory-website and a salesperson (from another company) trying to sell me SEO, pointed out to me that in the code of the directory’s-website they use “noFollow” on link to my page. He then told me that this costs me SEO credits I could be receiving. The site I advertise with is telling me it’s not true. I don’t know who to believe. Do I get SEO credits for my link on their site or not? thanks in advance.
Tony, if the site that is linking to you has a nofollow tag on the link to your site, you are likely getting zero SEO benefit from it.
Wow Andrew, thanks for answering my post so fast and glad to get feedback from YOU. If it helps any here is the snipit of code around my link on their site: (noFollow is 2nd to last line) thanks.
tried to paste code in here, but I guess this form won’t allow it.
pasted it in a text file and put on my site.
Hope this works and hope Im not putting you out. thanks
http://www.napcoroofing.com/post.rtf
that looks like a nofollow tag to me
Hi Andrew
I built my site last year but while recently updating it foolishly copied and pasted the “noindex,nofollow” meta tag to it and didn’t realise until it was too late when google had stopped indexing my site.
I have now deleted the offending code but am still waiting for google to reindex .
Do you know how long this could take?
Is there anyway to speed up the process?
I have re-submitted my site on the google website.
I’m sending a mailer through the post next week and if I’m not searchable fear it could be a a pointless exercise.
I’m not a web designer and as you can probably tell from my very basic site have very little html code knowledge.
Any advice would be most welcome.
Regards
Richard
Richie, get some links to your site and it should be back up in no time
Richard, go make a few links on blogs like this, and on delicious, and other social bookmarking sites. Your site will be indexed in no time. Another thing that I found helps is if you are set up with google analytics and webmaster tools. Then google kind of has to know whats going on 😉
HUH! I just updated my CMS to add noindex, nofollow to pages with no content found. I thought nofollow is don’t follow back to this page(only) again.
A very good article and i would also want you to request some links of your articles about ‘how to get links from social book marking sites’.
I am beginner, I cant understand.
Do I use noindex or index?
Can someone tell me in simple words?
If the goal of my website is just to get to google 1st page for my keywords,
and I have no duplicate content issues –
then Do I use noindex or index?
(I already chose nofollow- I understand about nofollow)
But – Do I use noindex or index?
Great post. I always use nofollow tags to control internal linking on my sites. That way you can pass link juice to those pages you want ranked higher.
You have a very good description.thank you very much
Im getting different info on these tags and there weight on SE page ranking. I can only hope the noindex is correct.
I think this feature has created a lot of confusion. But if the whole idea of no follow is conceptually clear then it will definitely boost your site’s ranking.
I think that the no index is more useful for programmatic like web 2.0 and cms’s
Basically scripts and databases things that you want to specify not to put data on the web.
From what I have seen mostly, se’s tend to do a good job with not listing this stuff but if you want to be sure use the tag. No follow is an invitation imo.
Peace
If Twitter for instance wouldn’t use a “nofollow” link policy, do you think this would significantly affect their page rank (currently 9)?
Or is their “nofollow” practice in place mainly for discouraging users from spamming their site since people assume/regard that “nofollow” links do not give their sites additional value in building higher page rank?
Great post!!!
Ok i’m winding you up! actually thanks for this article.
I’ve set my site to noindex, and follow because the content changes allot. I also use noydir, noodp, and nosnippit.
another Godsend is google keyword tool 🙂 really like that one!
The old saying “You are what you eat” applies to crawlers as well. Feed the crawler what you want it to eat, and it will feed you.
However, I have found that page rank isn’t everything. I deliberately modified my Blogger Blog to be DO FOLLOW. My comments increased, and so did my organic rankings.
I want to know when category pages and tags created duplicates and you say they should not be indexed why then all the big news sites and popular blogs index all of their tags?
I can name http://technorati.com/blogs/top100 as an example, why their ranking in Google is good, is it useful for them too?
Yours Sincerely
Javid, in the case of duplicate category/tag URLs, instead of noindexing them I’d probably try to canonicalize one to the other.
As for Technorati, feels like it’s been a few years since I have seen a lot of their URLs ranking well.
i want to say some thing about
“Robots.txt:
Pages that are included in a Robots.txt file tell the search engines NOT to crawl these pages. However pages in a Robots.txt file can still accrue PageRank and can be indexed in search results, says Matt Cutts.”
How Can A page being indexed without crawling? As pr my knowledge a page cant be indexed without crawling
There are several ways this can happen. For example, the URL was already indexed before you blocked it in /robots.txt. And in my experience, if there are enough links to a URL Google does not seem to respect the /robots.txt directive 100% of the time and often you get “thin” URL indexing.
The prolbem with the /robots.txt is that they don’t have to follow it. It’s just a guideline. Like google please don’t go here. Since when did you not do something just because someone asked…..
One question that I have is that I have a login page on my site that allows login thru Facebook, Twitter, etc. Due to two plugins that had different conflicting settings, it was being followed and indexed, so all of links to the applications on social media sites were getting followed. In a case such as this, since I do not want “normal” visitors accessing this page, would this be an instance where it would be OK to noindex, nofollow. And how long does it usually take for Google to remove a page from indexing?
If these URLs are already in Google’s index, I would tag them with “noindex” first and wait for them to be purged. Once they are purged, you could either block them from being crawled via rel=”nofollow” tags on the links to them and/or via blocking them in /robots.txt. I would do both, just to be sure. In the trade, we call that “double-bagging”.
Re how long it takes to de-index a URL, it all depends on how often Googlebot crawls your site and how big/complex your site is.
i want to say some thing about
“Robots.txt:
Pages that are included in a Robots.txt file tell the search engines NOT to crawl these pages. However pages in a Robots.txt file can still accrue PageRank and can be indexed in search results, says Matt Cutts.”
How Can A page being indexed without crawling? As pr my knowledge a page cant be indexed without crawling
I didn’t have a real reason to do it. The bazooka is dangerous.