duplicate content issues.

Question

It seems that there are some problems with duplicate content that have not been addressed in q2a. For example on my site

http://mathhomeworkanswers.org

http://mathhomeworkanswers.org/questions

http://mathhomeworkanswers.org/activity

are essentially the same page. The only difference is that /questions and /activity include the additional page links to all of the questions at the bottom. This seems somewhat redundant and seems to be a duplicate content issue.

The same is true with all of the category pages.  They show up in questions and activity as well as directly off of the root

http://mathhomeworkanswers.org/algebra-1-answers

http://mathhomeworkanswers.org/activity/algebra-1-answers

http://mathhomeworkanswers.org/questions/algebra-1-answers

 

I personally would prefer that the root page and category pages automatically showed all of the additional question pages and elimiate the duplicate content

In other words show all activity on http://mathhomeworkanswers.org/

and get rid of /activity and /questions all together.

Same with the categories. I believe they should show the additional page buttons on the bottom if they exceed the length of the list and display only off of the root

http://mathhomeworkanswers.org/algebra-1-answers

and not

http://mathhomeworkanswers.org/activity/algebra-1-answers

http://mathhomeworkanswers.org/questions/algebra-1-answers

 

Any easy fix for this?

Comments

I really would like to know this solution. If anyone knows how to take care of this duplicated problems. please answer here

Answer 1

That is not really duplicate content, and you won't get penalised by search engines for it, if that was your concern. If anything, it aids their indexing of your site because you can show the top content in different forms (e.g. the first page of newest questions and the first page of active questions).

Although I didn't know that you could have all those sorting options in categories as well, that could be a little confusing for users to have so much.

You can remove menu items in the admin section, so you could turn off the "Activity" page. I don't think you can turn off "Questions" because that is the full paginated list, whereas the others are just a single page of recent questions.

Also, if you are really converned about Google indexing those pages, why not block them in robots.txt? Something like the following:

User-agent: *
Disallow: /activity/algebra-1-answers
Disallow: /questions/algebra-1-answers

And so on for all your categories.

Comments

Particularly in the area of categories I do find it somewhat confusing. I also feel that as far as inbound links that I would much prefer to not have different people linking to different versions of essentially the same page. Although I may not get penalized for duplicate content by search engines, if people like to different versions I will not get the most benefit out of the links and that will ultimately effect serp positions.  I did take the all activity link off but you are right, there is no other way of which I am aware to display "all questions" aside from the /questions

Maybe I'm over thinking it but as I've navigated through, looking from a user perspective it is a bit confusing.

---

Just discovered this issue today and I have to say I agree with mshelton. There should not be 3 different ways to essentially link to what is basically the same exact category content. I can't figure out the logic of this at all. But, I think you can deal with this via .htaccess and just redirect all categories pages to one particular type of category page. Anyone else figure out a solution for the duplicate category issues in QA?

---

It's a problem that is supposed to be solved by the rel=canonical tag.

---

yes, the rel tag would help for search engines, but it's the user perspective that I think is relevant here, as well as the linking issues. Also, when you try to customize the category pages, you end up having to customize it in three different areas. Very annoying.

There really is no need for three different types of category pages. I've never actually seen this in other software, so I'm really wondering what the logic is here, or maybe it's just something that was overlooked and can be fixed in future versions.

---

Ah - I see. This is just not optimal page construction. It makes no sense. The sidebar should be generated independently from the rest of the page. It should always be just one single link. I didn't even realize that. Heavens... my site has 53 subcategories for every category (there is a reason - there is no prefix) and that would be disastrous for SEO. No wonder it has had it's own issues... I'm going to manually change this and see what gives. I'm making a "static sidebar" by pasting the code into the html box to generate it on the sidebar.

---

The home page, questions page and activity page show different views of the content - (a) home page shows recent questions and answers, (b) questions page shows recent questions only, (c) activity page shows recent questions, answers, comments and other activity (such as editing). Each of these has their uses, and each can be navigated by category and subcategory, which seems to me to make sense from a user perspective. I'm very familiar with the duplicate content issue (I founded Copyscape) but just don't think this type of duplicate content is an issue. Tons of sites provide different ways to slice and navigate through lists of their content (think about WordPress, for example). The question pages themselves are where the real content is found, and these use rel=canonical to ensure search engines know the right URL to use.

---

Hi gidegreen,

Thanks for your response.

It's the categories that are the problem, not the various ways you group the questions (i.e. activity vs home page). I don't see much problem with those pages, per se. The real issue is that for each category, you have three different URL's for the same exact category. That really doesn't make sense. From a user perspective, I want to click on a category and then maybe sort that category by most popular questions, most answered etc. There shouldn't be three URL's for each category page. It is simply not useful and repetitive. Clearly, there needs to be a canonical URL for one of the categories. Which most can add manually. But, that doesn't help the usability issue or the maintenance issue (i.e. if you want to customize a category page, you end up having to customize it in three different places really).

As for Wordpress, there is always only one URL associated with each category, even if you can display content in various ways. You don't have 3 different URL's for each specific category.  I've actually never seen anything like this. I really think it should be fixed (though of course, I plan to fix it manually with some coding) and that each category has one category page, which displays the questions in that category. The category page should allow for pagination, and sorting of the questions in that category by popularity, recency etc. That would make the category pages very useful

Thanks.

---

I should add that what's interesting is that for Tags you actually opted to create a tag PHP page, so that each tag has one and only one URL page associated it, as opposed to three different URL's depending on the section of the website you are in. So I thought it was just an error when the same thing wasn't done for categories, which ultimately are quite similar to tags (most have difficulty figuring out what to put into categories and what into tags). I think categories should be treated like tags. One URL.

---

In terms of customization, if you're talking about basic layout, you can use the category-specific CSS classes in the <BODY> tag, e.g. qa-category-3. Similarly if you're doing custom HTML, you can look at $this->content['categoryids'] inside your theme class. But that doesn't resolve your concerns from a user perspective.

But a question: if the user is on (for example) the 'All Activity' page, and they only want to view activity within a particular category, how do you think they should do that? Or do you think that it should not be possible?

---

The difference between tags and categories is that categories are supposed to be a low number of fixed topics. Having more than about 15 categories kind of defeats their purpose IMO. But I updated my answer with a solution to block pages getting indexed in robots.txt.

There are two things that could be improved:
1. Make the sidebar links clearer. When you are on the 'All activity' page for example, the sidebar should make it clear that clicking a category will show you all activity for that category rather than the regular view.
2. Put all the sorting options under one taxonomy. In other words, you'd have
/qa/questions/core, /qa/questions/core?sort=activity, /qa/questions/core?sort=votes

---

Hi gidegreen,
I think that if they want to view the activity within a category, then they can click on the "main category page", which would be sorted by Activity. Send some URL variable that tells the page how to sort the data. Sort seems to be part of the db select anyway, so why not just send via URL?  It should work the same way with tags. Let's somebody wants to view a specific Tag, but in an All Activity page? Same sort of idea.

---

I think Disgruntledgoat's second option would be best. just saw it before my last comment.

---

Thanks for your responses. I'll think about the question for the next major release, though I'm not sure this is a burning issue for many people. Otherwise, there's always the option of implementing this via a plugin.