xml/html is unescaped

[FarrelBuch]

I used epm_fetch, then epm_parse and then getEPMData but alas I am getting a lot of unescaped text in my resultant data.frame.
Messed Up Value What It Should Be
Drąg-Zalesińska Drąg-Zalesińska
Małgorzata Małgorzata
Žentiņa žentiňa
Krūmiņa Krūmiņa
Jean François Jean François
Valérie Valérie
Service de Chirurgie Thoracique de l'H&#xf4 Service de Chirurgie Thoracique de l'Hô
Unit&#xe9 Unité
Péré Péré
Hélène Hélène
Daša Daša
Gluvajić Gluvajić

How do I fix this?
Google Gemini Advanced 2.5 Flash experimental keeps suggesting stringi::stri_unescape_html() but no such function exists. Do you have any suggestions?

[dami82]

Those are HTML or HEX codes and are part of the original response string. So, while these codes may look annoying in your R console, they make sense since the text was supposed to be consumed via a browser. For a list of HTML codes, see: https://unicodelookup.com/ Maybe you could create a dictionary and then use regular expressions (e.g., gsub() ) to replace the HTML codes with a character of choice. This may not be super elegant but it may work just fine. replace_html_codes <- function(dict, str) { stopifnot(is.character(dict), length(dict) > 0, !is.null(names(dict)), is.character(str), length(str) == 1) for (i in seq_along(dict)) { str <- gsub(names(dict)[i], dict[i], str) } return(str) } my_dict <- c('&#x16b;'='ū', '&#x146;'='ņ') my_str <-c('Hello', 'world', 'Kr&#x16b;mi&#x146;a') vapply(my_str, replace_html_codes, dict = my_dict, FUN.VALUE = 'hello', USE.NAMES = FALSE)

…

On Sat, Apr 26, 2025 at 9:18 PM Farrel Buchinsky ***@***.***> wrote: *FarrelBuch* created an issue (dami82/easyPubMed#31) <#31> I used epm_fetch, then epm_parse and then getEPMData but alas I am getting a lot of unescaped text in my resultant data.frame. Messed Up Value What It Should Be Dr&#x105;g-Zalesi&#x144;ska Drąg-Zalesińska Ma&#x142;gorzata Małgorzata &#x17d;enti&#x146;a žentiňa Kr&#x16b;mi&#x146;a Krūmiņa Jean Fran&#xe7;ois Jean François Val&#xe9;rie Valérie Service de Chirurgie Thoracique de l'H&#xf4 Service de Chirurgie Thoracique de l'Hô Unit&#xe9 Unité P&#xe9;r&#xe9; Péré H&#xe9;l&#xe8;ne Hélène Da&#x161;a Daša Gluvaji&#x107; Gluvajić How do I fix this? Google Gemini Advanced 2.5 Flash experimental keeps suggesting stringi::stri_unescape_html() but no such function exists. Do you have any suggestions? — Reply to this email directly, view it on GitHub <#31>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADQ5HIDRFTGVXQNEYEV6CID23QV45AVCNFSM6AAAAAB357TADCVHI2DSMVQWIX3LMV43ASLTON2WKOZTGAZDENJTHE3TGMA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[FarrelBuch]

OK. Thank you. I found the textutils package. HTMLdecode replaces named, hexadecimal and decimal character references as defined by HTML5 (see References) with characters. The resulting character vector is marked as UTF-8.