SEO Advice needed - homepage devalued because out of date URLs go to Home not 404

I have a large CATALook based e-commerce site. New pages and products are regularly added and old pages and products retired. Suppose Google indexes a product page, or a link to that page is created on an external site, but then later the product and/or page is retired. The URL obviously no longer leads to the same page, but instead of resulting in a 404 error, it leads to the home page of the site (this is not a function of CATALook I think, but a general DNN feature; the only way to trigger a 404 error seems to be to request something other than default.aspx, e.g. fgbobgowu.aspx).

This means that over time there could be more and more links to the homepage appearing to exist. The risk is that the search engines will start seeing these URLs all going to the homepage, and this will devalue the homepage as it will appear that there is duplicate content- which is a bad thing for the SEO of the site.

Can anyone suggest anything to avoid this devaluation of the homepage (or tell me I am wrong)?

Yes, that is a problem with more than just Catalook.  On the scale of "how bad" I would think it is pretty small though.

What I do is create unique pages with real page names that I then point to different Catalook products when they change.  I have just recently updated my http://www.snapsis.com/PageBlaster.aspx page to point to a different internal product page.  I do this using the SiteUrls.config file.  That only helps with new versions of the same product, or if you have a page name that makes sense to point somewhere else though.  Generally you want to have Pages like this that build up links over time so that they build their page rank.

That's not really the question you asked though. If you really want a 404 response for a specific page then I think you could add a rewrite rule to siteUrls.config, but have it rewrite (sent to) a page that doesn't exist.

Technically, when you retire a product, Catalook should give you some options about the response. That would be the best place to handle it. 

Since Catalook does not have that option yet, and you may even want to do this for other modules the next best thing that I can think of is to have another module on the page that can look at the incoming Urls and using a rules based system return a specific response (404, 301 redirect, etc.) 
A 301 redirect would even have the advantage of passing the PageRank along to the page you redirect to so it might be nice for retired products to point back to the start page of the store.

Hey, I just happen to have a module that can do that. :) http://www.snapsis.com/PageBlaster.aspx

I think you may be able to do different responses with the core RequestFilter also but I honestly have not done anything with that at all since PageBlaster does that and a lot more.

As with most assumptions/theories in search engine optimization we can't really say specifically what happens, but if we use Google as a benchmark then we can get a pretty good idea of how to rank the different SEO do's and dont's. In this blog post from Google they state:

" During our crawling and when serving search results, we try hard to index and show pages with distinct information. This filtering means, for instance, that if your site has articles in "regular" and "printer" versions and neither set is blocked in robots.txt or via a noindex meta tag, we'll choose one version to list. In the rare cases in which we perceive that duplicate content may be shown with intent to manipulate our rankings and deceive our users, we'll also make appropriate adjustments in the indexing and ranking of the sites involved. However, we prefer to focus on filtering rather than ranking adjustments ... so in the vast majority of cases, the worst thing that'll befall webmasters is to see the "less desired" version of a page shown in our index. "

So I don't rank getting rid of duplicate content as high as getting quality inbound links, having keywords in the page name, having keywords in well structured html tags, etc.

If you have a lot of products that are retiring all the time, but they have inbound links, then I think it would be best to change the page they are getting redirected to so that it is a 301 redirect, and then make sure that people have a nice index of all your products when they get there.  Taking it one step further you may even be able to list related products on the product not found page.  Of course that will take some changes in Catalook, but I think Suzanne would be willig to put them in the next version.

Yes, the new PageBlaster has the ability to do redirects at the Portal level based on conditional Regular Expression rules so you could create a Regular expression that basically has a list of retired products and have them 301 redirected to a new page that is specifically designed as I described above.

I just did some research, and what is actually happening is the module that loads the item details is returning a "Product not found" message, and there isn't any redirect happening.

So if you have PageBlaster on the item page, it still gets loaded with the simple "product not found" message.  As a first step you could also change the "Product not found" message to be more like what I described above. The message is located in a local resouce file at /DesktopModules/CatalookStore/App_LocalResources/productdetails.aspx.resx

Going even farther, you could use PageBlaster to conditionally replace the "Product not found" message with dynamic content based on the Url which still has the orginal link.

Here's a newer blog post by Vanessa Fox that says basically the same thing:

http://googlewebmastercentral.blogspot.com/2007/06/duplicate-content-summit-at-smx.html

I do find it interesting that some people find it so important that they have to have a whole summit based on duplicate content.

Anyway, on to the question.

Unfortunately PageBlaster cannot currently intercept the 302 redirect back to the main portal alias that is happening from calling a page that is no longer there (or one that was never there for that matter).

What you can do for SEO is block those pages in Robots.txt. This is the best and easiest way to make sure that old pages are removed from indexes.

You could also keep the page and just hide it then put PageBlaster on it to do a proper 301 Redirect. This would have the added advantage of passing on any PageRank and also allow you to pass any inbound links to something newer or more like what the original was rather than going to the home page.