Noindex thread pages

[dutchbb]

For some larger threads it could be a good option to only index the first page.

 

[dutchbb]

All pages with noindex will be crawled, doesn't matter if you remove it from first page. Google will just remove those pages from the index and check from time to time to see if it is still noindexed.

So it doesn't impact the crawl budget over time and is propably better then indexing page 200 or 2000 from a thread that also will be crawled again to see if the information on it is not changed.

If you can noindex a page it is also propably a better solution than disallow with robots.txt
https://xenforo.com/community/threads/robots-txt.16735/post-1716604

With disallow those pages can still be indexed if directly linked to from other website. One or more pages doesn't matter in this case, but still better to noindex. Especially if you already have those pages indexed.

But you could with robots.txt -> Disallow: /threads/name-of-thread/page-*

In XF 2.3 you can noindex a thread fully and XF 2.2 you can do this per forum (with criteria)
https://xenforo.com/community/threads/allow-moderators-to-change-noindex-status-of-a-thread.181250/

For example we have a 'post your meals (with pics)' thread. Those 3000 pages and pics generate almost no traffic because there are many sites about meals (with better pictures) that will rank much better.

Post je maaltijden (met pics!)

Net klaar met trainen..eiwitshake genomen en mis nog iets..iets pittigs

forum.bodybuilding.nl

When we noindex this there is a higher chance google will index other pages and threads that could rank better.

Letting it index the first page only could be a better option. With some threads the title creates a good ranking or the most important information is on the first page (an article for example).

 

[frm]

If you want to not index anything other than Page 1, put this as the very last lines of the template thread_view:

<xf:comment>Do not index anything other than Page 1</xf:comment>
<xf:if is = "$page != 1">
    <xf:head option="metaNoindex"><meta name="robots" content="noindex" /></xf:head>
</xf:if>

The <xf:comment> line is not necessary, but I use it around custom XF template syntax so that I can search a template to find, modify, or remove it easily. Without it, it could get lost if you ever decide to index more than just the 1st page.

There are other variables and if statements that can be used to judge about where you'd want it to cut off, but it gets a little more complicated than that.

Let me know if this isn't suitable for your needs and I'll try to tinker with it more in the morning.

 

[dutchbb]

Google also suggest removing the canonical metatag when noindexing a page.

 

[dutchbb]

If you want to not index anything other than Page 1, put this as the very last lines of the template thread_view:

<xf:comment>Do not index anything other than Page 1</xf:comment>
<xf:if is = "$page != 1">
    <xf:head option="metaNoindex"><meta name="robots" content="noindex" /></xf:head>
</xf:if>

The <xf:comment> line is not necessary, but I use it around custom XF template syntax so that I can search a template to find, modify, or remove it easily. Without it, it could get lost if you ever decide to index more than just the 1st page.

There are other variables and if statements that can be used to judge about where you'd want it to cut off, but it gets a little more complicated than that.

Let me know if this isn't suitable for your needs and I'll try to tinker with it more in the morning.

This would noindex all thread pages on all (sub)forums. But you could only add the threads you want with a conditional.

 

[frm]

This would noindex all thread pages on all (sub)forums.

Correct, you'd just need to wrap it in another <xf:if> statement to apply to the nodes that you want.

 

[frm]

Google also suggest removing the canonical metatag when noindexing a page.

Find and re/move this to the bottom of thread_view if that's the case:

<xf:macro id="metadata_macros::metadata"
	arg-description="{$fpSnippet}"
	arg-shareUrl="{{ link('canonical:threads', $thread) }}"
	arg-canonicalUrl="{{ link('canonical:threads', $thread, {'page': $page}) }}" />

To:

<xf:comment>Do not index anything other than Page 1</xf:comment>
<xf:if is = "$page != 1">
    <xf:head option="metaNoindex"><meta name="robots" content="noindex" /></xf:head>
</xf:if>
<xf:if is = "$page == 1">
<xf:macro id="metadata_macros::metadata"
	arg-description="{$fpSnippet}"
	arg-shareUrl="{{ link('canonical:threads', $thread) }}"
	arg-canonicalUrl="{{ link('canonical:threads', $thread, {'page': $page}) }}" />
</xf:if>

 

[dutchbb]

Correct, you'd just need to wrap it in another <xf:if> statement to apply to the nodes that you want.

Per thread_id would be better option instead of full node.

<xf:if is="in_array($thread.thread_id, [1,2,3])">

 

[frm]

Per thread_id would be better option instead of full node.

<xf:comment>Do not index anything other than Page 1 of thread id 1</xf:comment>
<xf:if is = "$page != 1">
    <xf:head option="metaNoindex"><meta name="robots" content="noindex" /></xf:head>
</xf:if>
<xf:if is = "in_array($thread.thread_id, [1])">
	<xf:if is = "$page == 1">
	<xf:macro id="metadata_macros::metadata"
		arg-description="{$fpSnippet}"
		arg-shareUrl="{{ link('canonical:threads', $thread) }}"
		arg-canonicalUrl="{{ link('canonical:threads', $thread, {'page': $page}) }}" />
	</xf:if>
</xf:if>

Add as many threads as you want when needed by editing this:

<xf:if is = "!in_array($thread.thread_id, [1])">

to

<xf:if is = "in_array($thread.thread_id, [1, 2])">
[/ICODE]
[CODE=html]
<xf:if is = "in_array($thread.thread_id, [1, 2, 3])">
[/ICODE]
[CODE=html]
<xf:if is = "in_array($thread.thread_id, [1, 2, 3, 4])">

Edit: TIred and got sloppy, $thread.thread_id, not $forum.thread_id

 

[dutchbb]

<xf:if is="in_array($thread.thread_id, [1,2,3])">

Should be without ! before in_array and $thread.thread_id in the thread_view template (post.thread_id in some others)

 

[frm]

Should be without ! before in_array

RIght. It's 1 AM here and you replied so quickly, that I jumped to the quickest solutions without double-checking other threads to ensure accuracy and also making, now, 2 errors, before posting. Updated as such.

Hope this helps you out until it's implemented.

It might be better to make this an add on that you can easily type in the thread_ids as it's an easy template edit once it knows the values.

I'm sure @AndyB could work off that template edit to put it in an add on that adds that code to the end of thread_view no problem.

Otherwise, you're going to have pages upon pages of edit history as you add threads to the array.

 

[Alpha1]

As Google appreciates content freshness, why not only have the last page indexed?

 

[frm]

As Google appreciates content freshness, why not only have the last page indexed?

Freshness isn't always relevant. The first page could have the substance while the last page could be filled with GIFs. Which would have more weight then?

 

[frm]

There are other variables and if statements that can be used to judge about where you'd want it to cut off, but it gets a little more complicated than that.

After inspecting $thread and knowing how many posts per page you have, the conditional isn't as hard as I would imagine (as I was going to try to do regex on the URI).

You could just do an if statement for if $thread.reply_count is say less than or equal to 60 for the first 3 pages if you have it set to the default of 20 posts per page.

I'm going to go to bed now before I go down a rabbit hole till 3 AM.

 

[dutchbb]

RIght. It's 1 AM here and you replied so quickly, that I jumped to the quickest solutions without double-checking other threads to ensure accuracy and also making, now, 2 errors, before posting. Updated as such.

Hope this helps you out until it's implemented.

It might be better to make this an add on that you can easily type in the thread_ids as it's an easy template edit once it knows the values.

I'm sure @AndyB could work off that template edit to put it in an add on that adds that code to the end of thread_view no problem.

Otherwise, you're going to have pages upon pages of edit history as you add threads to the array.

Testing it now

Did some changes and working.

Change $page > '1' to other number for more pages indexed and $thread.thread_id, [1,2,3,4,5] for which threads.

Noindex on all pages except the first page in the selected threads

<xf:comment>Do not index anything other than Page 1 of thread id </xf:comment>

<xf:if is="$page > '1' AND in_array($thread.thread_id, [1,2,3,4,5])">
    <xf:head option="metaNoindex"><meta name="robots" content="noindex" /></xf:head>
</xf:if>

With removal of canonical tag on noindexed pages

<xf:comment>Do not index anything other than Page 1 of thread id and show canonical only on indexed pages</xf:comment>

<xf:if is="$page > '1' AND in_array($thread.thread_id, [1,2,3,4,5])">
    <xf:head option="metaNoindex"><meta name="robots" content="noindex" /></xf:head>
<xf:else/>
	<xf:macro template="metadata_macros" name="metadata"
	arg-canonicalUrl="{{ link('canonical:threads', $thread, {'page': $page}) }}" />
</xf:if>

Remove arg-canonicalUrl="{{ link('canonical:threads', $thread, {'page': $page}) }}" from thread_view template

<xf:macro template="metadata_macros" name="metadata"
	arg-description="{$fpSnippet}"
	arg-shareUrl="{{ link('canonical:threads', $thread) }}"
	arg-canonicalUrl="{{ link('canonical:threads', $thread, {'page': $page}) }}" />