Word Count Search

Word Count Search 2.11.0

No permission to download
Overview FAQ Updates (45) Reviews (1) History Discussion
@Xon Getting this Server error log.

Code: 
XFES\Elasticsearch\RequestException: Elasticsearch error: query_shard_exception, too_many_clauses: maxClauseCount is set to 1024 src/addons/XFES/Elasticsearch/Api.php:410

Generated by: Nirjonmela Dec 26, 2020 at 7:15 PM

Stack trace

#0 src/addons/XFES/Elasticsearch/Api.php(331): XFES\Elasticsearch\Api->request('get', 'nadda_forumdata...', '{
    "sort": [...')
#1 src/addons/XFES/Elasticsearch/Api.php(234): XFES\Elasticsearch\Api->requestFromIndex('get', '_search', Array)
#2 src/addons/SV/SearchImprovements/XFES/Elasticsearch/Api.php(21): XFES\Elasticsearch\Api->search(Array)
#3 src/addons/XFES/Search/Source/Elasticsearch.php(748): SV\SearchImprovements\XFES\Elasticsearch\Api->search(Array)
#4 src/addons/XFES/Search/Source/Elasticsearch.php(144): XFES\Search\Source\Elasticsearch->executeSearch(Object(SV\SearchImprovements\XF\Search\Query\KeywordQuery), Array, '200')
#5 src/XF/Search/Search.php(183): XFES\Search\Source\Elasticsearch->search(Object(SV\SearchImprovements\XF\Search\Query\KeywordQuery), '200')
#6 src/XF/Search/Search.php(212): XF\Search\Search->XF\Search\{closure}(Object(SV\SearchImprovements\XF\Search\Query\KeywordQuery), '200')
#7 src/XF/Search/Search.php(179): XF\Search\Search->executeSearch(Object(SV\SearchImprovements\XF\Search\Query\KeywordQuery), '200', Object(Closure), true)
#8 src/XF/Repository/Search.php(34): XF\Search\Search->search(Object(SV\SearchImprovements\XF\Search\Query\KeywordQuery))
#9 src/XF/Pub/Controller/Search.php(430): XF\Repository\Search->runSearch(Object(SV\SearchImprovements\XF\Search\Query\KeywordQuery), Array, true)
#10 src/XF/Pub/Controller/Search.php(117): XF\Pub\Controller\Search->runSearch(Object(SV\SearchImprovements\XF\Search\Query\KeywordQuery), Array)
#11 src/addons/SV/SearchImprovements/XF/Pub/Controller/Search.php(58): XF\Pub\Controller\Search->actionSearch()
#12 src/XF/Mvc/Dispatcher.php(350): SV\SearchImprovements\XF\Pub\Controller\Search->actionSearch(Object(XF\Mvc\ParameterBag))
#13 src/XF/Mvc/Dispatcher.php(257): XF\Mvc\Dispatcher->dispatchClass('XF:Search', 'Search', Object(XF\Mvc\RouteMatch), Object(SV\SearchImprovements\XF\Pub\Controller\Search), NULL)
#14 src/XF/Mvc/Dispatcher.php(113): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(SV\SearchImprovements\XF\Pub\Controller\Search), NULL)
#15 src/XF/Mvc/Dispatcher.php(55): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#16 src/XF/App.php(2300): XF\Mvc\Dispatcher->run()
#17 src/XF.php(488): XF\App->run()
#18 index.php(20): XF::runApp('XF\\Pub\\App')
#19 {main}

Request state

array(4) {
  ["url"] => string(14) "/search/search"
  ["referrer"] => string(37) "/threads/72320/"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(4) {
    ["keywords"] => string(15491) "সবাই বসার ঘরে ফিরে যেতেইে।"
    ["c"] => array(2) {
      ["users"] => string(0) ""
      ["word_count"] => array(1) {
        ["lower"] => string(0) ""
      }
    }
    ["_xfToken"] => string(8) "********"
  }
}
 
This add-on doesn't increase the number of clauses. It is very likely a 3rd party add-on (mine or someone else's) is doing something silly with the query terms.

If you have Search Improvements installed, enable the "log search DSL" option and please post the resulting search (you will need to disable it right after or it will keep spamming the server error logs.
xenforo.com

Search Improvements

A collection of improvements to XF's Enhanced Search and some for XenForo's default MySQL search. MySQL & Elastic Search features; Allow * (or empty search string) to return results, for MySQL and XFES range_query search DSL allows arbitrary...
xenforo.com xenforo.com
 
Xon said:
If you have Search Improvements installed, enable the "log search DSL" option and please post the resulting search (you will need to disable it right after or it will keep spamming the server error logs.
Click to expand...

Getting this Server error log with enabled Log Search DSL Option.

Code: 
ErrorException: {"sort":["_score",{"date":"desc"}],"docvalue_fields":["discussion_id","user","date"],"_source":false,"size":"200","query":{"function_score":{"query":{"simple_query_string":{"query":"This add-on doesn't increase the number of clauses. It is very likely a 3rd party add-on (mine or someone else's) is doing something silly with the query terms. If you have Search Improvements installed, enable the \"log search DSL\" option and please post the resulting search (you will need to disable it right after or it will keep spamming the server error logs.","fields":["title","message"],"default_operator":"and"}},"functions":[{"exp":{"date":{"origin":1608993772,"decay":0.5,"scale":31536000}}}]}}} src/XF/Error.php:75

Generated by: Nirjonmela Dec 26, 2020 at 8:42 PM

Stack trace

#0 src/XF.php(213): XF\Error->logError('{"sort":["_scor...', false)
#1 src/addons/SV/SearchImprovements/XFES/Elasticsearch/Api.php(19): XF::logError('{"sort":["_scor...')
#2 src/addons/XFES/Search/Source/Elasticsearch.php(748): SV\SearchImprovements\XFES\Elasticsearch\Api->search(Array)
#3 src/addons/XFES/Search/Source/Elasticsearch.php(144): XFES\Search\Source\Elasticsearch->executeSearch(Object(SV\SearchImprovements\XF\Search\Query\KeywordQuery), Array, '200')
#4 src/XF/Search/Search.php(183): XFES\Search\Source\Elasticsearch->search(Object(SV\SearchImprovements\XF\Search\Query\KeywordQuery), '200')
#5 src/XF/Search/Search.php(212): XF\Search\Search->XF\Search\{closure}(Object(SV\SearchImprovements\XF\Search\Query\KeywordQuery), '200')
#6 src/XF/Search/Search.php(179): XF\Search\Search->executeSearch(Object(SV\SearchImprovements\XF\Search\Query\KeywordQuery), '200', Object(Closure), true)
#7 src/XF/Repository/Search.php(34): XF\Search\Search->search(Object(SV\SearchImprovements\XF\Search\Query\KeywordQuery))
#8 src/XF/Pub/Controller/Search.php(430): XF\Repository\Search->runSearch(Object(SV\SearchImprovements\XF\Search\Query\KeywordQuery), Array, true)
#9 src/XF/Pub/Controller/Search.php(117): XF\Pub\Controller\Search->runSearch(Object(SV\SearchImprovements\XF\Search\Query\KeywordQuery), Array)
#10 src/addons/SV/SearchImprovements/XF/Pub/Controller/Search.php(58): XF\Pub\Controller\Search->actionSearch()
#11 src/XF/Mvc/Dispatcher.php(350): SV\SearchImprovements\XF\Pub\Controller\Search->actionSearch(Object(XF\Mvc\ParameterBag))
#12 src/XF/Mvc/Dispatcher.php(257): XF\Mvc\Dispatcher->dispatchClass('XF:Search', 'Search', Object(XF\Mvc\RouteMatch), Object(SV\SearchImprovements\XF\Pub\Controller\Search), NULL)
#13 src/XF/Mvc/Dispatcher.php(113): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(SV\SearchImprovements\XF\Pub\Controller\Search), NULL)
#14 src/XF/Mvc/Dispatcher.php(55): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#15 src/XF/App.php(2300): XF\Mvc\Dispatcher->run()
#16 src/XF.php(488): XF\App->run()
#17 index.php(20): XF::runApp('XF\\Pub\\App')
#18 {main}

Request state

array(4) {
  ["url"] => string(14) "/search/search"
  ["referrer"] => string(36) "/search/search"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(3) {
    ["keywords"] => string(365) "This add-on doesn't increase the number of clauses. It is very likely a 3rd party add-on (mine or someone else's) is doing something silly with the query terms.  If you have Search Improvements installed, enable the "log search DSL" option and please post the resulting search (you will need to disable it right after or it will keep spamming the server error logs."
    ["c"] => array(2) {
      ["users"] => string(0) ""
      ["word_count"] => array(1) {
        ["lower"] => string(0) ""
      }
    }
    ["_xfToken"] => string(8) "********"
  }
}
 
Try restarting elasticsearch and/or updating it? I can't offer much server administration advice, the query itself looks fine.
 
Somehow not all word counts are generated; same for word counts in threads. Any idea what might cause this ?
 
This add-on doesn't generate word-counts for threads, my Threadmark's add-on does that.
 
Xon said:
This add-on doesn't generate word-counts for threads, my Threadmark's add-on does that.
Click to expand...
I think, its the the "threadmarks-only" option in wordcount.

Still thanks; Xenforo should combine this plugin with their SEO criteria. Instead of replies, word count instead. Maybe @Chris D.

At least, what I gonna do to filter out spam :)
 
We had an issue with this addon recently. This was installed prior to the Enhanced Search requirements in the most recent update. However, at some point we made the decision to remove ES from our license, as we were not using any of the features. This was about the same time the most recent update was pushed. Somehow, everyone's wires got crossed and the update to 2.7.0 and removal of ES happened at basically the same time without noting the requirements change.

As a result, the addon has been the cause of a serious delay of sorts which impacted posting times, but only for the user posting. Posts would show up to others immediately, but the poster would see the "loading" animation on a quick reply or full reply for as 10-60 seconds before it finally updated on their end. I believe it was tied into the search indexing job - something was hung up that the site would wait on while posting new content. Disabling the addon did not change this behavior either.

We have spent a long time tracking this down and it only became clear this addon was the cause when we switched from enabling/disabling addons to completely uninstalling them.

Additionally, the uninstall process for this seems to be partially broken. Attempting to uninstall from the ACP resulted in a white screen, and using the cmd.php option to uninstall would hang up as well. We had shut down the site and attempt the uninstall several times before it finally worked. We have to wonder if it was trying to access some functionality of Enhanced Search as part of that uninstall process as well.

The addon is now fully uninstalled and site performance was immediately improved to the point response times on post actions are well under a second. We attempted to reinstall it just to see what happened, and it hung on the install but did not prompt us about the Enhanced Search requirement. This message is now seemingly stuck on our addons page despite deleting the installation files after the install failed as well. Any idea how to get rid of it?

1659730851594.webp
 
Last edited:
This add-on was not the cause of your issues, but how you removed Enhanced search.

You need to uninstall XFES, and then remove the files. Then disable & uninstall ElasticSearch. That the site was stalling on posting is the exact behavior that happens when XFES is trying to post to an ElasticSearch instance which is offline.

Toggling an add-on enabled/disabled is not the same as uninstalling it.

Fullmental said:
Additionally, the uninstall process for this seems to be partially broken. Attempting to uninstall from the ACP resulted in a white screen, and using the cmd.php option to uninstall would hang up as well. We had shut down the site and attempt the uninstall several times before it finally worked. We have to wonder if it was trying to access some functionality of Enhanced Search as part of that uninstall process as well.

The addon is now fully uninstalled and site performance was immediately improved to the point response times on post actions are well under a second. We attempted to reinstall it just to see what happened, and it hung on the install but did not prompt us about the Enhanced Search requirement. This message is now seemingly stuck on our addons page despite deleting the installation files after the install failed as well. Any idea how to get rid of it?

View attachment 271748
Click to expand...
You have an ancient version of MySQL which sucks at adding/removing columns, and likely have way too much data indexed into MySQL when you should be using ElasticSearch.

I seriously do not recommend more than a few hundred thousand posts indexed via stock MySQL search backend.

The add-on adds the column xf_search_index.word_count on install, and removes it on uninstall. This can take some serious amounts of time if you have an older MySQL version, and poor disk performance for the database.
 
Xon said:
This add-on was not the cause of your issues, but how you removed Enhanced search.

You need to uninstall XFES, and then remove the files. Then disable & uninstall ElasticSearch. That the site was stalling on posting is the exact behavior that happens when XFES is trying to post to an ElasticSearch instance which is offline.

Toggling an add-on enabled/disabled is not the same as uninstalling it.


You have an ancient version of MySQL which sucks at adding/removing columns, and likely have way too much data indexed into MySQL when you should be using ElasticSearch.

I seriously do not recommend more than a few hundred thousand posts indexed via stock MySQL search backend.

The add-on adds the column xf_search_index.word_count on install, and removes it on uninstall. This can take some serious amounts of time if you have an older MySQL version, and poor disk performance for the database.
Click to expand...
XFES was disabled, uninstalled, and the files were removed along with ElasticSearch in February of this year It is not just sitting in the addons directory, disabled. This addon was updated to 2.7.0 as part of the same maintenance cycle. We were not aware of a specific requirement for order of removal. Please share where that information is kept in the Xenforo documentation because I don't see it?


You have an ancient version of MySQL which sucks at adding/removing columns, and likely have way too much data indexed into MySQL when you should be using ElasticSearch.
Click to expand...

We are on a current stable version of MySQL which receives regular maintenance updates. A major version upgrade is slated for later this year.
 
Last edited:
Xon said:
Xon updated Word Count Search with a new update entry:

2.10.1 - Maintenance update



Read the rest of this update entry...
Click to expand...
I just updated and I find this error that prevents me from continuing:

Code: 
XF\Db\Exception: MySQL statement prepare error [1146]: Table '****_xenforo.xf_sv_threadmark' doesn't exist in src/XF/Db/AbstractStatement.php at line 230
XF\Db\AbstractStatement->getException() in src/XF/Db/Mysqli/Statement.php at line 198
XF\Db\Mysqli\Statement->getException() in src/XF/Db/Mysqli/Statement.php at line 40
XF\Db\Mysqli\Statement->prepare() in src/XF/Db/Mysqli/Statement.php at line 56
XF\Db\Mysqli\Statement->execute() in src/XF/Db/AbstractAdapter.php at line 96
XF\Db\AbstractAdapter->query() in src/XF/Db/AbstractAdapter.php at line 152
XF\Db\AbstractAdapter->fetchOne() in src/addons/SV/WordCountSearch/Setup.php at line 69
SV\WordCountSearch\Setup->upgrade1709054098Step2() in src/XF/AddOn/StepRunnerUpgradeTrait.php at line 124
SV\WordCountSearch\Setup->upgradeStepRunner() in src/XF/AddOn/StepRunnerUpgradeTrait.php at line 73
SV\WordCountSearch\Setup->upgrade() in src/XF/Admin/Controller/AddOn.php at line 597
XF\Admin\Controller\AddOn->actionUpgrade() in src/XF/Mvc/Dispatcher.php at line 352
XF\Mvc\Dispatcher->dispatchClass() in src/XF/Mvc/Dispatcher.php at line 258
XF\Mvc\Dispatcher->dispatchFromMatch() in src/XF/Mvc/Dispatcher.php at line 115
XF\Mvc\Dispatcher->dispatchLoop() in src/XF/Mvc/Dispatcher.php at line 57
XF\Mvc\Dispatcher->run() in src/XF/App.php at line 2487
XF\App->run() in src/XF.php at line 524
XF::runApp() in admin.php at line 13

Mysql version: 10.6.17-MariaDB-1:10.6.17

Any ideas?

If I try to uninstall or disable it, it gives me this error:

XF\Db\Exception: MySQL statement prepare error [1146]: Table '****_xenforo.xf_sv_threadmark' doesn't exist src/XF/Db/AbstractStatement.php:230

Code: 
                SELECT max(threadmark.content_id)
                FROM xf_sv_threadmark as threadmark
                join xf_post_words as postWords on threadmark.content_id = postWords.post_id
                where threadmark.content_type = 'post' and threadmark.word_count is null
            
------------

#0 src/XF/Db/Mysqli/Statement.php(198): XF\Db\AbstractStatement->getException('MySQL statement...', 1146, '42S02')
#1 src/XF/Db/Mysqli/Statement.php(40): XF\Db\Mysqli\Statement->getException('MySQL statement...', 1146, '42S02')
#2 src/XF/Db/Mysqli/Statement.php(56): XF\Db\Mysqli\Statement->prepare()
#3 src/XF/Db/AbstractAdapter.php(96): XF\Db\Mysqli\Statement->execute()
#4 src/XF/Db/AbstractAdapter.php(152): XF\Db\AbstractAdapter->query('
              ...', Array)
#5 src/addons/SV/WordCountSearch/Setup.php(69): XF\Db\AbstractAdapter->fetchOne('
              ...')
#6 src/XF/AddOn/StepRunnerUpgradeTrait.php(124): SV\WordCountSearch\Setup->upgrade1709054098Step2(Array)
#7 src/XF/AddOn/StepRunnerUpgradeTrait.php(73): SV\WordCountSearch\Setup->upgradeStepRunner(1709054098, 2, Array, NULL)
#8 src/XF/Admin/Controller/AddOn.php(597): SV\WordCountSearch\Setup->upgrade(Array)
#9 src/XF/Mvc/Dispatcher.php(352): XF\Admin\Controller\AddOn->actionUpgrade(Object(XF\Mvc\ParameterBag))
#10 src/XF/Mvc/Dispatcher.php(258): XF\Mvc\Dispatcher->dispatchClass('XF:AddOn', 'Upgrade', Object(XF\Mvc\RouteMatch), Object(Bespoke\SequentialStickies\XF\Admin\Controller\AddOn), NULL)
#11 src/XF/Mvc/Dispatcher.php(115): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(Bespoke\SequentialStickies\XF\Admin\Controller\AddOn), NULL)
#12 src/XF/Mvc/Dispatcher.php(57): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#13 src/XF/App.php(2487): XF\Mvc\Dispatcher->run()
#14 src/XF.php(524): XF\App->run()
#15 admin.php(13): XF::runApp('XF\\Admin\\App')
#16 {main}

Code: 
array(4) {
  ["url"] => string(45) "/admin.php?add-ons/SV-WordCountSearch/upgrade"
  ["referrer"] => string(73) "/admin.php?add-ons/SV-WordCountSearch/upgrade"
  ["_GET"] => array(1) {
    ["add-ons/SV-WordCountSearch/upgrade"] => string(0) ""
  }
  ["_POST"] => array(7) {
    ["_xfProcessing"] => string(1) "1"
    ["continue"] => string(1) "1"
    ["confirm"] => string(1) "1"
    ["params"] => string(2) "[]"
    ["count"] => string(1) "1"
    ["finished"] => string(1) "0"
    ["_xfToken"] => string(8) "********"
  }
}
 
Last edited:
I had to reinstall version 2.9.0 on top of it so I could force uninstall later, otherwise the option didn't appear.But equally, it is impossible to uninstall it since it ended with an error:

Code: 
XF\Db\InvalidQueryException: xf_search_index: MySQL query error [1091]: Can't DROP COLUMN `word_count`; check that it exists in src/XF/Db/AbstractStatement.php at line 230
XF\Db\AbstractStatement->getException() in src/XF/Db/Mysqli/Statement.php at line 198
XF\Db\Mysqli\Statement->getException() in src/XF/Db/Mysqli/Statement.php at line 78
XF\Db\Mysqli\Statement->execute() in src/XF/Db/AbstractAdapter.php at line 96
XF\Db\AbstractAdapter->query() in src/XF/Db/Schema/AbstractDdl.php at line 158
XF\Db\Schema\AbstractDdl->apply() in src/XF/Db/SchemaManager.php at line 154
XF\Db\SchemaManager->alterTable() in src/addons/SV/WordCountSearch/Setup.php at line 82
SV\WordCountSearch\Setup->uninstallStep2() in src/XF/AddOn/StepRunnerUninstallTrait.php at line 61
SV\WordCountSearch\Setup->uninstallStepRunner() in src/XF/AddOn/StepRunnerUninstallTrait.php at line 28
SV\WordCountSearch\Setup->uninstall() in src/XF/Admin/Controller/AddOn.php at line 657
XF\Admin\Controller\AddOn->actionUninstall() in src/XF/Mvc/Dispatcher.php at line 352
XF\Mvc\Dispatcher->dispatchClass() in src/XF/Mvc/Dispatcher.php at line 258
XF\Mvc\Dispatcher->dispatchFromMatch() in src/XF/Mvc/Dispatcher.php at line 115
XF\Mvc\Dispatcher->dispatchLoop() in src/XF/Mvc/Dispatcher.php at line 57
XF\Mvc\Dispatcher->run() in src/XF/App.php at line 2487
XF\App->run() in src/XF.php at line 524
XF::runApp() in admin.php at line 13

In the end, I rebuilt 2.9.0 by reinstalling it again, disabled it and it let me uninstall it without problems.
But it cost him...

Another day if you release an update I will try to install it again in case it continues giving errors and you need me to test something.

Greetings and thanks for the work
 
You must log in or register to reply here.

Similar threads

Ozzy47
[OzzModz] Post Message Word Count
Replies
2
Views
465
Robert9
R
Ozzy47
[OzzModz] Thread Title Word Count
Replies
3
Views
667
Ozzy47
Ozzy47
Mr Lucky
  • Question Question
XF 2.2 Format and word count for RSS imports
Replies
0
Views
285
Mr Lucky
Mr Lucky
C
  • Question Question
Any way to see a word count?
Replies
8
Views
1K
sbj
sbj
A
  • Question Question
XF 1.5 Thread Word Count
Replies
3
Views
556
Xon
X
Top Bottom