Unified search (fulltextsearch) does not find content, while search app does #617
Description
Important note
This issue is a copy from nextcloud/server nextcloud/server#26329
Originally I posted the issue there and I don't know, how to move here, because it seems it is better fit for fulltextsearch.
Steps to reproduce
- I want to search a file with specific content, a word: gazsi
- Click on the quicksearch on the top-right corner and start typing the word
- The search started spining and after a short time it shows nothing: No results for gazsi
- Open the Search applet from the menu, enter the word, press enter
- The search shows the file which contains the word
Expected behaviour
The two search should provide the same results, in other words, the unified search should show the files instead of no results.
Actual behaviour
As you can see on the screenshot above: the two results of the two searches are different.
Use case
This issue seems complex, due to that, I had two almost identical setup in two different location and after I found this in one site, I was able the exact same fault on the other one. Details below.
Server configuration
Operating system:
Debian 10.
Web server:
Apache
Database:
Mariadb 10.5
PHP version:
7.4
Nextcloud version: (see Nextcloud admin page)
20.0.8
Updated from an older Nextcloud/ownCloud or fresh install:
Regular update.
Onlyoffice
6.3
Documentserver
Standalone documentserver in a different virtual machine, working as expected for months.
Version: onlyoffice-documentserver_6.2.0-123_amd64.deb
Previous version: onlyoffice-documentserver_6.0.0-105_amd64.deb
Signing status:
Signing status
No errors have been found.
List of activated apps:
App list
Enabled:
- accessibility: 1.6.0
- activity: 2.13.4
- analytics: 3.4.1
- announcementcenter: 4.0.1
- appointments: 1.8.10
- apporder: 0.12.0
- bookmarks: 4.1.0
- bruteforcesettings: 2.0.1
- calendar: 2.2.0
- circles: 0.20.7
- cloud_federation_api: 1.3.0
- comments: 1.10.0
- contacts: 3.5.1
- cospend: 1.3.0
- dashboard: 7.0.0
- data_request: 1.7.0
- dav: 1.16.2
- deck: 1.2.5
- drawio: 1.0.0
- encryption: 2.8.1
- event_update_notification: 1.2.0
- external: 3.7.2
- extract: 1.3.1
- federatedfilesharing: 1.10.2
- federation: 1.10.1
- files: 1.15.0
- files_external: 1.11.1
- files_fulltextsearch: 20.0.1
- files_linkeditor: 1.1.5
- files_markdown: 2.3.3
- files_mindmap: 0.0.24
- files_pdfviewer: 2.0.1
- files_rightclick: 0.17.0
- files_sharing: 1.12.2
- files_trashbin: 1.10.1
- files_versions: 1.13.0
- files_videoplayer: 1.9.0
- firstrunwizard: 2.9.0
- flowupload: 1.1.2
- forms: 2.2.3
- fulltextsearch: 20.0.0
- fulltextsearch_elasticsearch: 20.0.0
- groupquota: 0.1.5
- guests: 1.6.2
- impersonate: 1.7.0
- integration_gitlab: 1.0.0
- issuetemplate: 0.7.0
- login_notes: 0.4.0
- logreader: 2.5.0
- lookup_server_connector: 1.8.0
- mail: 1.9.4
- maps: 0.1.8
- metadata: 0.13.0
- music: 1.1.0
- news: 15.3.2
- nextcloud_announcements: 1.9.0
- notes: 4.0.4
- notifications: 2.8.0
- oauth2: 1.8.0
- ocsms: 2.2.0
- onlyoffice: 6.3.0
- password_policy: 1.10.1
- passwords: 2021.3.0
- phonetrack: 0.6.7
- photos: 1.2.3
- polls: 1.8.1
- previewgenerator: 3.1.1
- privacy: 1.4.0
- provisioning_api: 1.10.0
- qownnotesapi: 20.9.0
- quicknotes: 0.6.7
- quota_warning: 1.9.1
- sensorlogger: 0.1.3
- settings: 1.2.0
- sharebymail: 1.10.0
- side_menu: 1.23.1
- socialsharing_diaspora: 2.1.0
- socialsharing_facebook: 2.1.0
- socialsharing_twitter: 2.1.0
- spreed: 10.0.6
- survey_client: 1.8.0
- suspicious_login: 3.2.1
- systemtags: 1.10.0
- talk_matterbridge: 1.22.1
- tasks: 0.13.6
- text: 3.1.0
- theming: 1.11.0
- timetracker: 0.0.55
- twofactor_backupcodes: 1.9.0
- twofactor_totp: 5.0.0
- unsplash: 1.2.2
- updatenotification: 1.10.0
- user_external: 1.0.0
- user_status: 1.0.1
- viewer: 1.4.0
- w2g2: 3.0.3
- weather: 1.7.5
- weather_status: 1.0.0
- workflowengine: 2.2.0
Disabled:
- admin_audit
- audioplayer
- checksum
- contactsinteraction
- dicomviewer
- files_accesscontrol
- files_antivirus
- files_automatedtagging
- files_downloadactivity
- files_photospheres
- files_readmemd
- groupfolders
- keeweb
- ocr
- passman
- podcast
- radio
- recommendations
- registration
- serverinfo
- social
- support
- terms_of_service
- theming_customcss
- user_ldap
- wopi
- workflow_pdf_converter
- workflow_script
Nextcloud configuration:
Config report
{
"system": {
"instanceid": "***REMOVED SENSITIVE VALUE***",
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"overwritewebroot": "\/cbox",
"dbtype": "mysql",
"version": "20.0.8.1",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbtableprefix": "oc_",
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"default_language": "hu",
"theme": "",
"maintenance": false,
"filelocking.enabled": true,
"memcache.locking": "\\OC\\Memcache\\Redis",
"memcache.local": "\\OC\\Memcache\\APCu",
"memcache.distributed": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"port": 0,
"timeout": 0
},
"trusted_domains": [
"XXXXXXXXXXXXXX"
],
"mail_smtpmode": "smtp",
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"mail_smtphost": "***REMOVED SENSITIVE VALUE***",
"mail_smtpport": "25",
"loglevel": 2,
"overwrite.cli.url": "\/cbox",
"custom_csp_policy": "default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; frame-src *; img-src *; font-src 'self' data:; media-src *",
"appstoreenabled": true,
"apps_paths": [
{
"path": "\/var\/www\/XXXXXXXXXXXXXXXX\/web\/cbox\/apps",
"url": "\/apps",
"writable": true
}
],
"secret": "***REMOVED SENSITIVE VALUE***",
"ldapIgnoreNamingRules": false,
"trashbin_retention_obligation": "auto",
"versions_retention_obligation": "30, auto",
"singleuser": false,
"preview_libreoffice_path": "\/usr\/bin\/libreoffice",
"preview_office_cl_parameters": " --headless --nologo --nofirststartwizard --invisible --norestore -convert-to pdf -outdir ",
"appstore.experimental.enabled": true,
"updater.release.channel": "stable",
"mysql.utf8mb4": true,
"app_install_overwrite": [
"spreed"
],
"mail_sendmailmode": "smtp",
"allow_local_remote_servers": true,
"onlyoffice": {
"verify_peer_off": true,
"jwt_header": "AuthorizationJwt"
},
"encryption.legacy_format_support": false,
"encryption.key_storage_migrated": false
}
}
Are you using external storage, if yes which one: local/smb/sftp/...
Only local.
Are you using encryption: yes/no
no
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
no
Client configuration
Browser:
Brave
Operating system:
Linux Mint 20
Logs
Web server error log
Web server error log
There is no error entries in the log.
Nextcloud log (data/nextcloud.log)
Nextcloud log
{"reqId":"8Piq24L5N8ao0WhOlpjY","level":3,"time":"2021-03-26T22:39:48+00:00","remoteAddr":"91.X.X.X","user":"xxxxxxx","app":"PHP","method":"GET","url":"/cbox/ocs/v2.php/search/providers/fulltextsearch/search?term=gazsi&from=%2Fcbox%2Fsettings%2Fapps%2Finstalled%2Fonlyoffice","message":{"Exception":"Error","Message":"Undefined offset: 1 at /var/www/XXXXXXXXXX/web/cbox/apps/fulltextsearch/lib/Search/UnifiedSearchProvider.php#158","Code":0,"Trace":[{"file":"/var/www/XXXXXXXXXX/web/cbox/apps/fulltextsearch/lib/Search/UnifiedSearchProvider.php","line":158,"function":"onError","class":"OC\\Log\\ErrorHandler","type":"::"},{"file":"/var/www/XXXXXXXXXX/web/cbox/apps/fulltextsearch/lib/Search/UnifiedSearchProvider.php","line":137,"function":"generateSearchRequest","class":"OCA\\FullTextSearch\\Search\\UnifiedSearchProvider","type":"->"},{"file":"/var/www/XXXXXXXXXX/web/cbox/lib/private/Search/SearchComposer.php","line":162,"function":"search","class":"OCA\\FullTextSearch\\Search\\UnifiedSearchProvider","type":"->"},{"file":"/var/www/XXXXXXXXXX/web/cbox/core/Controller/UnifiedSearchController.php","line":110,"function":"search","class":"OC\\Search\\SearchComposer","type":"->"},{"file":"/var/www/XXXXXXXXXX/web/cbox/lib/private/AppFramework/Http/Dispatcher.php","line":169,"function":"search","class":"OC\\Core\\Controller\\UnifiedSearchController","type":"->"},{"file":"/var/www/XXXXXXXXXX/web/cbox/lib/private/AppFramework/Http/Dispatcher.php","line":100,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/XXXXXXXXXX/web/cbox/lib/private/AppFramework/App.php","line":152,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/XXXXXXXXXX/web/cbox/lib/private/Route/Router.php","line":309,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/XXXXXXXXXX/web/cbox/ocs/v1.php","line":88,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/XXXXXXXXXX/web/cbox/ocs/v2.php","line":24,"args":["/var/www/XXXXXXXXXX/web/cbox/ocs/v1.php"],"function":"require_once"}],"File":"/var/www/XXXXXXXXXX/web/cbox/lib/private/Log/ErrorHandler.php","Line":91,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.105 Safari/537.36","version":"20.0.8.1"}
{"reqId":"8Piq24L5N8ao0WhOlpjY","level":3,"time":"2021-03-26T22:39:48+00:00","remoteAddr":"91.X.X.X","user":"xxxxxx","app":"PHP","method":"GET","url":"/cbox/ocs/v2.php/search/providers/fulltextsearch/search?term=gazsi&from=%2Fcbox%2Fsettings%2Fapps%2Finstalled%2Fonlyoffice","message":{"Exception":"Error","Message":"Undefined offset: 2 at /var/www/XXXXXXXXXX/web/cbox/apps/fulltextsearch/lib/Search/UnifiedSearchProvider.php#158","Code":0,"Trace":[{"file":"/var/www/XXXXXXXXXX/web/cbox/apps/fulltextsearch/lib/Search/UnifiedSearchProvider.php","line":158,"function":"onError","class":"OC\\Log\\ErrorHandler","type":"::"},{"file":"/var/www/XXXXXXXXXX/web/cbox/apps/fulltextsearch/lib/Search/UnifiedSearchProvider.php","line":137,"function":"generateSearchRequest","class":"OCA\\FullTextSearch\\Search\\UnifiedSearchProvider","type":"->"},{"file":"/var/www/XXXXXXXXXX/web/cbox/lib/private/Search/SearchComposer.php","line":162,"function":"search","class":"OCA\\FullTextSearch\\Search\\UnifiedSearchProvider","type":"->"},{"file":"/var/www/XXXXXXXXXX/web/cbox/core/Controller/UnifiedSearchController.php","line":110,"function":"search","class":"OC\\Search\\SearchComposer","type":"->"},{"file":"/var/www/XXXXXXXXXX/web/cbox/lib/private/AppFramework/Http/Dispatcher.php","line":169,"function":"search","class":"OC\\Core\\Controller\\UnifiedSearchController","type":"->"},{"file":"/var/www/XXXXXXXXXX/web/cbox/lib/private/AppFramework/Http/Dispatcher.php","line":100,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/XXXXXXXXXX/web/cbox/lib/private/AppFramework/App.php","line":152,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/XXXXXXXXXX/web/cbox/lib/private/Route/Router.php","line":309,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/XXXXXXXXXX/web/cbox/ocs/v1.php","line":88,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/XXXXXXXXXX/web/cbox/ocs/v2.php","line":24,"args":["/var/www/XXXXXXXXXX/web/cbox/ocs/v1.php"],"function":"require_once"}],"File":"/var/www/XXXXXXXXXX/web/cbox/lib/private/Log/ErrorHandler.php","Line":91,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.105 Safari/537.36","version":"20.0.8.1"}
{"reqId":"8Piq24L5N8ao0WhOlpjY","level":2,"time":"2021-03-26T22:39:48+00:00","remoteAddr":"91.X.X.X","user":"xxxxxxx","app":"fulltextsearch","method":"GET","url":"/cbox/ocs/v2.php/search/providers/fulltextsearch/search?term=gazsi&from=%2Fcbox%2Fsettings%2Fapps%2Finstalled%2Fonlyoffice","message":"Issue while loading Provider: ocsms/OCA\\OcSms\\Provider\\FullTextSearchProvider - OCP\\AppFramework\\QueryException Could not resolve OCA\\OcSms\\Provider\\FullTextSearchProvider! Class OCA\\OcSms\\Provider\\FullTextSearchProvider does not exist","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.105 Safari/537.36","version":"20.0.8.1"}
How did I reproduced the steps needed to trigger this problem
The system described above with the standalone installation of onlyoffice documentserver v6.0 was working perfectly.
Fulltextsearch initially indexed, live search was running, elasticsearch contains everything as expected.
The life was just perfect until I decided to upgrade the standalone documentserver only from 6.0 -> 6.2.
Steps:
- Documentserver upgraded (rabbitmq, pgsql etc., except onlyoffice documentserver), server rebooted.
- Dist-upgrade issued, onlyoffice-documentserver upgraded as expected, local.json file restored to meet the environment.
- Editing onlyoffice documents online perfectly working, I tested and used it for days.
- Start searching with top-right unified search box -> failed.
- Start searching with search application -> working as expected.
- The standalone elasticsearch is working well, I found the word directly on the admin console of elasticsearch (elasticvue).
At this point I checked my other system on a remote site, with the old documentserver (6.0), in other words, the documentserver was in untouched.
The result:
- Top-right search shows the expected result -> just perfect.
- Search application shows the same result -> just perfect.
- Live index is working as expected, new content indexed in the background.
- Now I upgraded the documentserver as I did earlier.
- I got exactly the same search failure as with the older system.
Note for the two systems:
They are not exactly the same, as one of them installed in the root folder of the docroot, the other one one folder deeper and some other kind of options, but the environment pretty much the same:
webserver (VM1) ------- documentserver(VM2);
| --------------- elasticsearch(VM3);
Question
Where to find help with this strange behaviour?
Here, on the onlyoffice app or where?
Thanks.
Here is the Nextcloud log in case of searching via the search application, which actually can provide the expected result:
Nextcloud log
``` {"reqId":"BYVqKMAqLOG7O4nNi3tx","level":2,"time":"2021-03-27T07:15:09+00:00","remoteAddr":"91.X.X.X","user":"xxxxxx","app":"fulltextsearch","method":"GET","url":"/cbox/index.php/apps/fulltextsearch/v1/search?request=%7B%22providers%22%3A%22all%22%2C%22options%22%3A%7B%22files_local%22%3A%220%22%2C%22files_group_folders%22%3A%220%22%2C%22files_extension%22%3A%22%22%7D%2C%22search%22%3A%22l%C3%B3tet%22%2C%22page%22%3A1%7D","message":"Issue while loading Provider: ocsms/OCA\\OcSms\\Provider\\FullTextSearchProvider - OCP\\AppFramework\\QueryException Could not resolve OCA\\OcSms\\Provider\\FullTextSearchProvider! Class OCA\\OcSms\\Provider\\FullTextSearchProvider does not exist","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.105 Safari/537.36","version":"20.0.8.1"} {"reqId":"2K5s6IGSqDDi7pNsvOyC","level":2,"time":"2021-03-27T07:15:10+00:00","remoteAddr":"91.X.X.X","user":"xxxxxx","app":"fulltextsearch","method":"GET","url":"/cbox/index.php/apps/fulltextsearch/v1/search?request=%7B%22providers%22%3A%22all%22%2C%22options%22%3A%7B%22files_local%22%3A%220%22%2C%22files_group_folders%22%3A%220%22%2C%22files_extension%22%3A%22%22%7D%2C%22search%22%3A%22l%C3%B3tet%C5%B1%22%2C%22page%22%3A1%7D","message":"Issue while loading Provider: ocsms/OCA\\OcSms\\Provider\\FullTextSearchProvider - OCP\\AppFramework\\QueryException Could not resolve OCA\\OcSms\\Provider\\FullTextSearchProvider! Class OCA\\OcSms\\Provider\\FullTextSearchProvider does not exist","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.105 Safari/537.36","version":"20.0.8.1"} ```New experience:
I just restored the old documentserver. It seems this action did not solve the problem, the situation is exactly the same as I described.
So, what I can see:
- the version of the documentserver could be not the source of the problem
- in case the version changes did not trigger any auto-setup in the nextcloud side
- restarting the documentserver could be trigger of the problem
- please note: nextcloud itself, elasticsearch did not restarted for weeks/months,
- only upgrades happened with nextcloud since 20.x, everything else was running for lot of weeks without restarting (elasticsearch: uptime 160 days)
- I experienced exactly the same problem right after I upgraded/restarted the standalone documentserver, in two independent nextcloud installation
- both instances are upgraded regularly (nextcloud), php and system files rarely, they are not the same, one of them are weeks behind the upgrade (redis, memcached etc. not restarted due to upgrade for weeks in one case, 1 week in the other case)