Writing a search query for multiple conditions

[OleksiyAnokhin]

Hi team, thanks for a great package.

A quick question:

I want to extract information by name of the journal, time period, and first/last name of the author.
I have 10000 names, 3 years, and 5 journals. Is it possible to write a query like this in your package

Ideal example:

list_of_authors <- ("Joe Doe", "Joe Smith")
list_of_journals <- ("Radiology, "Oncology")
years <- (2019, 2020, 2021)
new_query <- 'List of authors AND list of journals AND years' 
out.A <- batch_pubmed_download(pubmed_query_string = new_query, 
                           format = "xml", 
                           batch_size = 100,
                           dest_file_prefix = "test",
                           encoding = "ASCII")

Several separate queries would be fine but not thousands!

Any suggestions are greatly appreciated!

[dami82]

Hello Alex, It can be done but not the way you suggested... Since the query is passed GET rather than POST, you cannot query 10000 names at once. Also, you should think about when to use an AND boolean operator vs an OR operator. Finally, you can extract PMIDs first, apply boolean logic and finally fetch the results. Email me in pvt, we can take this discussion offline for the fine code details. Best regards, Damiano —— Damiano Fantini Sent from my iPhone

…

On May 11, 2022, at 15:32, Alex (Oleksiy) Anokhin ***@***.***> wrote:  Hi team, thanks for a great package. A quick question: I want to extract information by name of the journal, time period, and first/last name of the author. I have 10000 names, 3 years, and 5 journals. Is it possible to write a query like this in your package Ideal example: list_of_authors <- ("Joe Doe", "Joe Smith") list_of_journals <- ("Radiology, "Oncology") years <- (2019, 2020, 2021) new_query <- 'List of authors AND list of journals AND years' out.A <- batch_pubmed_download(pubmed_query_string = new_query, format = "xml", batch_size = 100, dest_file_prefix = "test", encoding = "ASCII") Several separate queries would be fine but not thousands! Any suggestions are greatly appreciated! — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

[OleksiyAnokhin]

@dami82 , I will follow up here. Emailed you before.

 # Query all possible data?
 new_query <- 'some journal [TA]' 
 new_query <- get_pubmed_ids(new_query)

 # Fetch data
 fetched_data <- fetch_pubmed_data(new_query, encoding = "ASCII", retmax = 5000)

 # Create a dataframe
 new_PM_df <- table_articles_byAuth(pubmed_data = fetched_data, 
                               included_authors = "all", #Included all authours
                               max_chars = 0, 
                               encoding = "ASCII")

 # Print
 is_df <- new_PM_df[c("journal", "title", "year", "firstname", "lastname")]

As a result, for my journal I received 13946 rows (multiple authors for the same publication) in total from 2006 to 2022.
How to ensure that this is the extensive list, if you have 5000 as a limitation? Years 2009 and 2010 are missing in my final dataframe, but I am more concerned about 2017-2022.

Appreciate any suggestions.

[dami82]

Thanks again for testing easyPubMed and sorry for my late reply... I had a few very hectic days lately. I promise I'll look into your request 1 over the weekend.

Now, about query 2:

• The way table_articles_byAuth() works is by "exploding" results by author (one identical row for each author). This is actually pretty useful for certain applications (when you care about authors rather than abstract text). If you are not really interested in authors, just set included_authors = "last" (this will only retain the last author, typically the principal investigator behind a research paper).

• The limit of 5,000 records was set to avoid timeouts while receiving data from the Entrez server. With Entrez, you DO NOT want to download all the records at once (especially if n is large). On the contrary, you want to split the job in multiple small tasks. The Entrez API allows to do so by tuning the retstart and retmax params, to precisely fetch records returned by a query. My package allows advanced users to tweak retmax and retstart. Alternatively, you can just sit back and let batch_pubmed_download() do the job for you (and save records locally). Note that batch_pubmed_download() implements some internal QC checks so that it will keep running until all the records have been fully downloaded.

This vignette may be useful: https://www.data-pulse.com/projects/Rlibs/vignettes/easyPubMed_03_retmax_example.html

Don't hesitate to email me with any questions about easyPubMed