How to Have All Your Pages in Google Main Index
As you know Google and also all other main search engines are highly against duplicated content and spamming. They try to do their best to make their search result as relevant, unique and distinct as possible.
Google Supplemental Index was created in 2003 to fight with duplicated content. Websites that have duplicated pages and contents become penalized or banned by Google but even in good websites, there are always some pages that look so similar to each other and it doesn’t look necessary to display all of them in the search result. So only one of these pages were chosen to be displayed in the search result pages and all the other pages were kept in the Supplemental Index.
But Google always keeps on improving and it has become able to crawl more pages and URLs and two pages that were used to be looked so similar to Google, look different now and they will be displayed in Google search result under special queries. Google improves everyday and so the Supplemental Index loses its application and as Google engineers emphasized, the difference between supplemental and main index is getting narrowed. So they removed the “Supplemental Result” label from the search result pages.
Some webmasters and bloggers didn’t understand the Google engineers properly and have asked Google not to remove the “Supplemental Result” label from search results. Why? Because most webmasters and bloggers were used to check the Google index and see which of their pages were in the Supplemental Index and then tried to get them back to the main index through making some changes on them. Now they think they are not able to find these pages any more and these pages will remain in the Supplemental Index for good and searchers can not see them. This is not correct.
Instead of getting worried about having your pages in the Supplemental Index, try to make distinct and unique pages that Google knows them as the pages that deserve to be in the main index. How?
1. Make unique pages that their contents are different from other pages.
2. If the content of a new page looks so similar to an older page, try to link the new page to the older one to say Google that these pages are related to each other but are different and unique.
3. Avoid having strange URLs in your website. Try to have the URLs as small and static as possible.
4. Update your old pages by adding new paragraphs to them to send this signal to Google that these pages are not just created and abandoned and they are important and become updated regularly.
Bottom line:
Google is a search engine and its search result is its main product. If Google wants to keep its position and ranking as the best and most professional search engine, it has to improve its algorithm and system. Google loves to find and index more pages and information because it helps the searchers to find what they look for more easily. If you see that your websites and weblogs pages don’t receive any reasonable traffic from Google, it doesn’t mean that Google avoids to index your pages. It means that your pages have some problems that prevents Google to drive any traffic to them. It means that others’ pages, websites and weblogs are more eligible to be ranked than yours.
So what should you do? You have to follow Google guidelines and fix the problems. Then you will enjoy the traffic that Google drives to your pages.
Related topics from this website:
How to Start a “HIT” Website?
How to Improve and Keep Your Google Rankings
92 Reasons Why Your Website Sucks But Others Websites Make Real Money
What Is the Google’s Favorite Title?!
Description Meta Tag - Important But Not Critical
Related topics from other websites:
We love you, webmasters
More details about our webmaster guidelines
Nice article vahid.. keep it up.. it has cleared my idea about supplement results and how to get rid of that..
Hi Vahid,
Your blog is very helpful. I’m new in blog(started April 2007) and planning to get one or two domain soon. So I think i might need your advice before im going to my new web-site.
Wish Weboma and you all the best!
Great info, Vahid. I’m glad to subscribe to your feed. I can get to read brand new post and information every times you update your blog. Keep it coming.
Greetings and lotta loves from Malaysia.
Nice articles there Vahid. I just have one question:
I have a website (not a blog) and I have at least 50 pages on it. Some of the pages I forgot to add the meta description. Now I update itby adding some meta description but the indexed page in Google is still the same - no meta description. How can I let know Google that I added now a meta descrition of that pages?
A really well written article.I learn something new, I willd efinatly be back often to keep myself updated ! Thanks
Ramil, do you use a sitemap function? That would help Google more accurately register your pages.
Thanks to all my friends who posted their comment to this article so far: Bala Krishna, jamesforex, ArahMan7, Ramil, Jesse and Michael.
Dear James, Please let me know whenever you think I can be any help.
Dear Ramil, Google spider will come to your website and will update its index. You don’t have to inform Google. It will check the websites pages automatically. Also please keep in mind that the title and description of the pages should not be the same.
Great article Vahid. I’ll have to keep in mind the adding-a-picture-to-old-content strategy to make it look like it’s updated. I didn’t know about that.
@Michael,
Yes, I am using sitemap and I have already updated and submitted it last July 30. Google shows that it crawled the site last August 15. however, the indexed pages are still the same - the old one. It did not update it’s indexed correctly.
Thanks Patrick. I am glad that my article was helpful for you.
Ramil,
So you should not have any problem and you will see the new information in Google search result soon. It takes some time for Google to update its main index with the new information that its spider has collected. There are always some delay between what you see in the cache and the search result.
Please wait for a few more days and then check. If there was any problem, let me know and send the URLs to check it for you.
Thanks friend.
Earl
Hi Vahid,
This whole SEO issue gets very confusing I read things in your blog and in other sites that make sense but wonder how to apply them in a real world environment. I will focus on just duplicated content. I use the Joomla CMS. My site traffic is growing every month. I seem to have lots of search terms bringing my site up on page 1 for Google, MSN and Yahoo. Most of my search engine traffic (80%) comes from Google but search engine traffic only accounts for 17-18% of my total site traffic. Most of it I am wondering where it is coming from. Also the site is PR5
In the stats where it says referrer most of them come from the site or one of my other sites.
back to duplicate content, Joomla has several things that I would consider duplicate content but it doesnt seem to effect my site showing on page one of google or my site PR. I post a news article for example ant it shows as a blurb on the front page with a full article when you click on read more. This same content is delivered as RSS, PDF and blog. Isn’t this duplicate content? Is google penalizing this?
Hi Toma,
Thanks for your comment.
Regarding your website traffic, I checked your Alexa ranking and as you said it is improving very fast. Congratulations for that. It seems you don’t have a good web analyzer to detect the incoming traffic of your website and know where they come from. Of course I see that you know the percentages but there are still a lot to know. If you really want to know about the origin of the traffic and the referrers, I suggest you to sign up for the StatCounter.com service which is free and post the code on the footer template file to make the StatCounter able to detect the traffic of all the pages and not just the home page. I think Joomla should have such a feature and by posting a code on the footer template, it can work on all the pages.
Regarding the duplicated content, something you explained is known as duplicated content BUT there are two things that I have to let you know. First of all, now Google is progressed enough to distinguish spamming from the duplicated content that is caused by the platforms that we use (we have the same problem with WordPress). So you will not be penalized by Google especially if you create a good XML site map and submit it to Google Webmasters Tools.
Secondly, there are easy ways to fix this duplication. For example, when your web pages can be easily indexed, you don’t have to have the PDF files to be index too. So you can prevent search engines to see them by adding the rel=”nofollow” to the related links. I don’t think that the RSS is any problem because search engines know about it too.
So although these kinds of duplicated contents can not be any problem but you can fix them as much as you can to stay at the safe side. You can take the advantage of rel=”nofollow” and and also robots.txt files.
Vahid
Actually I have several web analyzers but they just add more confusion since I really do not know which is reliable. You mentioned Alexis and I know lots of people think the Alexis ranking is important. But that is only from people who already are tracked by Alexis. I think I am in a unique demographic which can really skew the Alexis rating. Many of my site visitors will be over the age group where most of them will be net savvy and will therefore be rather under represented in Alexis.
I already have StatCounter from statCounter.com and it provides lots of good info. I also use webalizer and Awstats on my server As well as Joomstats. Part of the problem is they all give me wildly different numbers. So Far no one has had a good explanation as to why this happens. Awstats, webalizer and Joomstats are all server side so I would assume they are a bit more reliable but these also have very different numbers reported. StatCounter generally show about half the traffic reported by Webalizer.
What is consistent is the general trends and what the user behaviors are. Still I do not know if I can trust these as thay are way off of what the experts say are good numbers. For example I am told 80% of traffic comes from search engines yet less than 18% of my site traffic is from this source. I have also read that you are doing good if over 6% of your traffic stays for more than 20 secounds yet stats are telling me over 30% are staying for over a minute and the majority of that for extended time. 4.3 % 15mn-30mn, 3.6 % 30mn-1h 2.5 %, 1h+ 4.3. These look good and are far better than what I am told I should expect, then others tell me these numbers are not reliable.
As a result I am just focusing on making the site as good as I can and hoping the rest takes care of it’s self. Even the numbers I have, If I knew I could rely on them or at least knew the deficiencies of each set, Are hard to interpret since I really do not know specifically how each is defining hits, page loads, unique visitors, etc.
My site started as a result of being diagnosed type 2 diabetic and almost dying from it. The ADA methods were not working so I started searching the internet to find better answers. I found the better answers but they were so obscure and hard to find that I decided to put everything I found for my own use on a web site to make it easier for others. Doing the web stuff has been a real uphill battle and a very steep learning curve. It is part of a self appointed mission to help other diabetics. What I need is a very talented staff of young turks like yourself who are good at this stuff. That takes money and unfortunately my sites are a volunteer effort funded from my disability income so there is no money to hire the needed talent. (sigh)
I just need to keep learning from talented people such as yourself that are willing to freely share what they know.
Thank you
Hi Vahid,
You’ve got a great blog here. I like it very much! Thanks for these helpful infor that is so great for new bloggers like me.
Have a nice day!
Hi Jackie, Thanks a lot. I am happy that you like my blog. You also have a good blog and I bookmarked it.
Hi Toma,
I believe the stats that StatCounter shows is more reliable because most of the others like webalizer are so weak in distinguishing the real visitors and spiders.
If you see that StatCounter shows half of the traffic that Webalizer, it is because that StatCounter only reports the real visitors not spiders and crawlers.
It is also the same with others like Awstats and Joomstats.
Why?
Because StatCounter crew are so active and always add the IP of different search engines spiders to their skip list. This is not easy because some of the search engines spiders like MSN have tens of IP addresses and new IP addresses come everyday.
Also the confusion you have about the time that each visitor stays in your website is because of this problem too because as I explained most web analyzers can not distinguish the search engines spiders from real visitors.
Toma, I believe you are on the right track with your website and you will see the brilliant results very soon. Just keep in your mind that fresh and unique content is the key.
Also I wanted to say that I have not worked with Joomla but I prefer WordPress as a blog plateform because it is so search engine friendly.
Vahid
Thanks for some great tips. Having just started a new blog, your advice is very useful.
Very well said. I now have an idea on how to make all of my pages get the main page result in Google..
Hi,
I am happy to have found you!
What do you mean by “keeping strange URLs as small and static” as possible?
I am “extreme newbie” - if you check my site, I would appreciate your feedback.
Thanks very much.
Lori
Hi Lori,
I have not said “keeping strange URLs as small and static” any where on this article.
I said “Avoid having strange URLs in your website. Try to have the URLs as small and static as possible.”
It means avoid having URLs that have question mark (?). Try to have static URLs like the URLs of my weblog articles and keep them as small as possible.
Vahid