List search only finds exact matches: Switch to use Solr #2723

jessamynwest · 2019-12-12T02:49:23Z

~~Blocked by: #2609~~

Expanding on #364 Improve and expand list searching to search inside list titles.

A List search result is only given if the exact, full name of a list is used in the search. Searching on words that are only part of a list name returns "No results found."

Relevant url?

https://openlibrary.org/search/lists

Steps to Reproduce

Go to ... https://openlibrary.org/search/lists
Do ... enter a partial list name, for example, https://openlibrary.org/search/lists?q=banned

Actual: only exact matches returned
Expected: any list containing search term should be returned

Proposal & Constraints

We can now replace lists search with solr-backed list search since lists are now in solr! To do so, we will need to:

add a new SearchScheme for lists; see the e.g. AuthorSearchScheme as an example.

Update the /search/lists controller to now do a solr search instead

openlibrary/openlibrary/plugins/worksearch/code.py

Lines 574 to 597 in 4f0c3c7

    
           class list_search(delegate.page): 
        
               path = '/search/lists' 
        
               def GET(self): 
        
                   i = web.input(q='', offset='0', limit='10') 
        
                   lists = self.get_results(i.q, i.offset, i.limit) 
        
                   return render_template('search/lists.tmpl', q=i.q, lists=lists) 
        
               def get_results(self, q, offset=0, limit=100): 
        
                   if 'env' not in web.ctx: 
        
                       delegate.fakeload() 
        
                   keys = web.ctx.site.things( 
        
                       { 
        
                           "type": "/type/list", 
        
                           "name~": q, 
        
                           "limit": int(limit), 
        
                           "offset": int(offset), 
        
                       } 
        
                   ) 
        
                   return web.ctx.site.get_many(keys)

. Follow the pattern from authors:

openlibrary/openlibrary/plugins/worksearch/code.py

Lines 664 to 674 in 4f0c3c7

    
           def get_results(self, q, offset=0, limit=100, fields='*', sort=''): 
        
               resp = run_solr_query( 
        
                   AuthorSearchScheme(), 
        
                   {'q': q}, 
        
                   offset=offset, 
        
                   rows=limit, 
        
                   fields=fields, 
        
                   sort=sort, 
        
               ) 
        
               return resp

.

You might need to make some changes to https://github.com/internetarchive/openlibrary/blob/097f52e86c3bbbb65a31c4bc46000d259e30c504/openlibrary/templates/search/lists.html as well.

Stakeholders

@cdrini @seabelis @el4ctr0n

The text was updated successfully, but these errors were encountered:

xayhewalo · 2019-12-12T23:59:20Z

@cdrini @tfmorris I'm assuming list search is a Solr feature? Will that be apart of the reindex?

BrittanyBunk · 2019-12-27T18:41:31Z

You're asking for a search bar for a list when clicking on it? This really is part of a bigger issue, where when clicking on anything - authors, works, etc. - it's not possible to alphabetize or search. I mentioned it on #2741 too, but for authors. I believe I also created a github issue for works/editions, but maybe someone else could find that #.

tfmorris · 2019-12-27T22:07:49Z

@guyjeangilles It would be a combination of UI plus Solr. It's a new feature which would not fall naturally out of the general Solr upgrade project.

cdrini · 2024-06-11T13:43:58Z

Update: Lists are now in solr so this is no longer blocked! I updated the description with instructions. Here's a sample list record in our solr:

{
      "key":"/people/ScarTissue/lists/OL228125L",
      "name":"Cookbooks",
      "seed":["/works/OL8941312W","/works/OL189773W","/books/OL36885385M"],
      "type":"list",
      "subject":["Cookery","Cooking","Cooking, french","Cuisine en plein air","Dictionaries","English","French","French Cookery","French Cooking","French language","Outdoor cooking","Picnicking","Picnics","Pique-niques"],
      "subject_facet":["Cookery","Cooking","Cooking, french","Cuisine en plein air","Dictionaries","English","French","French Cookery","French Cooking","French language","Outdoor cooking","Picnicking","Picnics","Pique-niques"],
      "_version_":1796111292365275138,
      "subject_key":["cookery","cooking","cooking_french","cuisine_en_plein_air","dictionaries","english","french","french_cookery","french_cooking","french_language","outdoor_cooking","picnicking","picnics","pique-niques"]
}

jediD83 · 2024-07-20T08:32:23Z

@cdrini May I please take a crack on this issue?

Although I've never work with Solr before, nor am I confident with my current skills, I have done some relevant self-taught tutorials, such as Django and Flask.

cdrini · 2024-07-29T16:53:23Z

Hi @jediD83 , go for it! This might be a touch difficult; I updated the description with more details. Feel free to give it a go though! Let us know if you hit any issues

jediD83 · 2024-08-06T02:54:13Z

@cdrini Thank for assigning this task to me. I think I'm almost done, but ...I don't know how to test if my codes are working properly.

In the development version, the Search Results doesn't work like the real one. And currently, all errors encountered have been fixed, but that doesn't mean my objectives were reached.

I have sent a Slack volunteering request, and waiting for my invitation. Although I'm not sure what Slack name should I displayed: GitHub, OpenLibrary, or my Slack original name?

cdrini · 2024-08-06T05:34:09Z

Sweet! You should be able to test this by creating a new list on your local environment, and then searching for it. You can create a draft PR with your code; that'll let us help you more quickly 👍

@mekarpeles should be able to add you to slack 👍

jediD83 · 2024-08-13T05:05:29Z

@mekarpeles should be able to add you to slack 👍

@mekarpeles @cdrini I don't want to sound rude but over a week had passed since this, and applied to OL's Volunteering as a Software Engineer, and I did not receive any email or slack invitation. I need a little help with my unfinished PR work.

cdrini · 2024-08-13T18:59:21Z

Apologies @mekarpeles was away last week, he should be able to add you shortly 👍 Although I'm having trouble determining which application is yours; could you send me the name you used on your form?

jediD83 · 2024-08-13T23:33:50Z

Apologies @mekarpeles was away last week, he should be able to add you shortly 👍

No problem.

Although I'm having trouble determining which application is yours; could you send me the name you used on your form?

Dan Ken

Fix issue #2723: Improve list search functionality

jessamynwest added the Type: Feature Request label Dec 12, 2019

xayhewalo added Module: Solr Needs: Triage Priority: 3 State: Backlogged Theme: Search labels Dec 12, 2019

BrittanyBunk mentioned this issue Dec 31, 2019

Ability to sort works on Author pages alphabetically #2796

Open

xayhewalo removed the State: Backlogged label Mar 17, 2020

cdrini added the Needs: Lead label Apr 20, 2020

mekarpeles removed the Needs: Triage label Apr 20, 2020

cdrini added Lead: @cdrini State: Blocked and removed Needs: Lead labels Apr 20, 2020

mekarpeles mentioned this issue Jan 22, 2021

Improving Search (No Dead Ends) #2728

Closed

13 tasks

mekarpeles mentioned this issue Sep 26, 2022

Search: Editions in Solr #6377

Closed

63 tasks

cdrini removed the State: Blocked label Jun 11, 2024

cdrini changed the title ~~Improve list search~~ List search only finds exact matches Jun 11, 2024

cdrini mentioned this issue Jun 11, 2024

List search only finds exact matches #6974

Closed

cdrini added Priority: 2 Theme: Lists and removed Priority: 3 labels Jun 11, 2024

cdrini mentioned this issue Jun 11, 2024

List search should return partial matches #9416

Closed

mekarpeles added Needs: Response Good First Issue labels Jul 15, 2024

mekarpeles changed the title ~~List search only finds exact matches~~ List search only finds exact matches: Switch to use Solr Jul 15, 2024

cdrini assigned jediD83 Jul 29, 2024

cdrini removed the Needs: Response label Jul 29, 2024

jediD83 mentioned this issue Aug 6, 2024

2723 Issue: Solr + List Search #9695

Draft

github-actions bot added the Needs: Response label Aug 13, 2024

cdrini removed the Needs: Response label Aug 19, 2024

jediD83 removed their assignment Oct 6, 2024

isabellabonilla added a commit to isabellabonilla/openlibrary that referenced this issue Feb 17, 2025

Fix issue internetarchive#2723: Improve list search functionality

4b74a06

isabellabonilla mentioned this issue Feb 17, 2025

Fix issue #2723: Improve list search functionality #10460

Merged

cdrini pushed a commit to isabellabonilla/openlibrary that referenced this issue Mar 27, 2025

Fix issue internetarchive#2723: Improve list search functionality

7442dcd

cdrini added a commit that referenced this issue Mar 27, 2025

Merge pull request #10460 from isabellabonilla/FixListSearch

f2fb5eb

Fix issue #2723: Improve list search functionality

cdrini closed this as completed in #10460 Mar 27, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

List search only finds exact matches: Switch to use Solr #2723

List search only finds exact matches: Switch to use Solr #2723

jessamynwest commented Dec 12, 2019 •

edited by cdrini

Loading

xayhewalo commented Dec 12, 2019

BrittanyBunk commented Dec 27, 2019 •

edited

Loading

tfmorris commented Dec 27, 2019

cdrini commented Jun 11, 2024 •

edited

Loading

jediD83 commented Jul 20, 2024

cdrini commented Jul 29, 2024

jediD83 commented Aug 6, 2024 •

edited

Loading

cdrini commented Aug 6, 2024

jediD83 commented Aug 13, 2024

cdrini commented Aug 13, 2024

jediD83 commented Aug 13, 2024

List search only finds exact matches: Switch to use Solr #2723

List search only finds exact matches: Switch to use Solr #2723

Comments

jessamynwest commented Dec 12, 2019 • edited by cdrini Loading

Relevant url?

Steps to Reproduce

Proposal & Constraints

Stakeholders

xayhewalo commented Dec 12, 2019

BrittanyBunk commented Dec 27, 2019 • edited Loading

tfmorris commented Dec 27, 2019

cdrini commented Jun 11, 2024 • edited Loading

jediD83 commented Jul 20, 2024

cdrini commented Jul 29, 2024

jediD83 commented Aug 6, 2024 • edited Loading

cdrini commented Aug 6, 2024

jediD83 commented Aug 13, 2024

cdrini commented Aug 13, 2024

jediD83 commented Aug 13, 2024

jessamynwest commented Dec 12, 2019 •

edited by cdrini

Loading

BrittanyBunk commented Dec 27, 2019 •

edited

Loading

cdrini commented Jun 11, 2024 •

edited

Loading

jediD83 commented Aug 6, 2024 •

edited

Loading