In the olden days, as in before this week, you used to be able to get an idea of how many pages you had in Google’s index by searching “site:<yourdomain>”. The resulting page would say something like “results 1-10 of 1,390,000” which while not entirely accurate gave you a general idea of how well indexed your site was. Now with the official launch of Google Caffeine (update: I stand corrected, this is not a Caffeine issue but a new GOOG UI issue that I neglected to stay on top of – thanks Rhaghavan), the site: query no longer displays the number of total results (update: at least it doesn’t work for me but as you can see in the comments others have not experienced this yet).
While many people were unduly obsessed with this number, it did have its uses. For example, while big swings in the reported number say from 10,000,000 to 236,000 were scary but irrelevant, small changes in the reported number seemed to be more in sync with SEO problems or fixes.
So if you still want to find out how many pages your domain has in the index how do you do it?
- Sign up for Google Webmaster Tools and submit xml sitemaps for every URL on your domain. The Sitemaps report in GWT will then show the number of indexed URLs from your sitemaps (btw it’s not clear that this number is accurate either). My guess is getting more xml sitemaps submitted was one of the primary reasons that GOOG stopped reporting this number. That and maybe saving bandwidth from all of those site: queries that nervous site owners did all day long.
- If you don’t want to give GOOG your data via GWT, then you can still do a fake site: query by using “inurl:<yourdomain>”. Make sure you don’t use “www” in the query (e.g. inurl:localseoguide.com). This isn’t a perfect query – sites that incorporate your domain into their URLs will show up (e.g. www.alexa.com/siteinfo/localseoguide.com), but for most sites this shouldn’t be a huge number of URLs. It’s hard to judge how accurate this query is but I have tried it for several client sites and it seems to square up pretty well with how many pages they seem to have.If anyone has any other ideas feel free to add them to the comments and/or put them on your blog, link back here and it will show up in the trackbacks.
13 Response Comments
On my sample query I get just under 600k for “site:” and 16 million for “inurl:”
that’s not even close 🙂
why not just
site:*.localseoguide.com
Michael, maybe I am seeing a version of caffeine for SEOs but I don’t get any numbers when I do a site: query.
Jason, same thing with your query.
But as I said, the inurl: query is definitely not perfect.
maybe my info on which data center is caffeine is wrong…. I’m hitting 225.103
Andrew,
As a matter of fact Caffeine shows me 2x – 10x more results (which is comparable to GWT) for the websites. I am using 216.239.59.103
It isn’t caffine that you refer to instead it is the new interface where Google doesn’t display the number of results for the site: operator.
Duly noted up top Raghavan. Thanks.
Hi,
With the introduction of Google Caffeine active Twitter users will soon be seen on Google search pages. This is a promising new development that I believe will help small business owners get the absolute most out of search.
I am having the same issue as you were. I could not get a good list of indexed pages via the site: query. Upon suggestion I used the inurl, but that wasnt quite accurate either.
I then used inurl:site.com site:site.com and that seems to be accurate for me.
Does Google have a displayed results quality quantity formula working? If I search INURL +force a keyword I get some records back, some are missing, the missing ones can appear if I force extra words into the query? Google saying … just give them what they asked for even if “We” think its a bit rubbish?
Watching our own site: and inurl: results over the last couple weeks has been a roller coaster ride.
With a business directory site, indexed pages is important to us considering the millions of business profile pages we hope Google decides to deem worthy of being indexed.
About 2 months ago, we made a major site structure change and URL change to our site, and we took a big hit in our indexed pages. Since then, we have been steadily back on the rise, increasing by 5-10k pages every few days.
The last week we have seen many spikes, up to 250k, and then settling around 200k total pages indexed.
During those searches, the supplemental index continued to stay around the 800k-900k level.
Now, all of a sudden, the supplemental index shows less pages than what is actually in the index!
Instead of seeing 200k indexed and 900k indexed in the supplemental, We see 200k indexed – 150k in the supplemental!!!
Is Caffeine replacing the supplemental index? I can’t find many people talking about this at all. Very curious!
Watching our own site: and inurl: results over the last couple weeks has been a roller coaster ride.http://www.mbtgoods.com
Well you can just download seoquake and search in google: inurl: site name, youre done…
—
http://www.hotlib.com/