Sign In  |  Register
 
 DotNetNuke Powered!
DotNetNuke Support Forums

Disabled Pages Indexed?

Rate this topic:

Please Register to post a reply.
Another benefit of registration is the ability to subscribe to and recieve notifications of new posts.

AuthorMessages
Terp
Posts:80



09/19/2007 12:49 PM  

I feel like such a slave to Google lately, but thought I would ask anyway: do disabled pages get indexed? I'm not sure if it matters, but I have no internal links linking to them, but I do have some placeholder pages on the menu that are disabled and I have been too lazy to enter titles and descriptions and the like.

...and I assume hidden modules or modules viewable only by admins are not seem by Google? How about pages viewable only by registered users? I am just trying to understand what Google sees and doesn't see. I am assuming they see only what unregistered users would see, but not quite sure.

John Mitchell
Posts:3040



09/19/2007 1:16 PM  

Google sees and knows all. :)  Well, ok, maybe not everything.

If there is a link anywhere that points to a page then Googlebot will follow it, but it will only be able to index what an unauthenticated user can access.  This may end up in Googlebot reaching a secure page and then being redirected to the login page, which in turn will make the login url get indexed. That is why I block urls with ctl in them using Robots.txt

Check out this list of pages that were indexed from the scenario I described above.

http://www.google.com/search?sourceid=navclient&ie=UTF-8&rlz=1T4GGIH_en___US207&q=allinurl%3actl+login+returnurl

Terp
Posts:80



09/19/2007 1:36 PM  

Wow, nice.  114K, so I am not alone, I guess.

So as long as I don't link to the disabled menu items, I guess it's okay to shortcut the description, etc...ergo, any container title of a hidden container won't be shown in the source and therefore not seem by Google, either, right?

Yea, I tried to change the ctl nonsense last month (when I posted the raw code and it made my post uneditable even by you ), but it didn't work...think I'll read your post again and give it another shot. Coincidently, I was just looking at the internal link numbers in the Webmaster tools and while the logo SNAFU and removing the base made my homepage the #1 internal link, the blog is killing me with dozens and dozens of duplicates, as are the terms and privacy...need to address this this weekend, I guess. I've had the static pages built; just couldn't get the sent to right...maybe 14 times is a charm.

[edit]

I added the line in the robots file,per a post of yours inthe DNN forums, and Google reindexed my site and shows only 13 restricted URLS by robots.txt...strange. I actually went overboard on the disallow:

# robots.txt file for http://www.bidlesstravel.com/
# 7/11/2007  22:17

User-agent: *
Disallow: /Admin/
Disallow: /App_Browser/
Disallow: /App_Code/
Disallow: /App_Data/
Disallow: /App_GlobalResources/
Disallow: /bin/
Disallow: /Components/
Disallow: /Config/
Disallow: /Controls/
Disallow: /DesktopModules/
Disallow: /Documentation/
Disallow: /Install/
Disallow: /js/
Disallow: /Providers/
Disallow: /Resources/
Disallow: */ctl/


# end of file

Please Register to post a reply.
Another benefit of registration is the ability to subscribe to and recieve notifications of new posts.

Forums >DotNetNuke Support >General-DNN-Help > Disabled Pages Indexed?



ActiveForums 3.7
Visit our Store for great DotNetNuke Modules and Skins
DNNMasters Sitemap/Google Sitemap 3.0

Item codeSM3-01
Price$29.00
Product Information 
DotNetNuke CSS NavMenu 3.3 (Developers)

Item codeCSSNM33DEV
Base Price$149.00
Product Information 
Snapsis PageBlaster 3.3.2 for DotNetNuke - Professional Edition

AuthorJohn Mitchell
Base Price$79.00
Product Information