index.qmd

---
format:
  html:
    toc: true
execute:
  eval: true
  freeze: true
---

```{r}
#| include: false


library(dplyr)
library(dbplyr)
library(tictoc)
library(DBI)
source("site/knitr-print.R")

reticulate::use_virtualenv("python/.venv")
```

```{python}
#| include: false

import mall
```

<img src="site/images/favicon/apple-touch-icon-180x180.png" style="float:right"/>

<!-- badges: start -->

[![PyPi](https://img.shields.io/pypi/v/mlverse-mall)](https://pypi.org/project/mlverse-mall/) [![Python tests](https://github.com/mlverse/mall/actions/workflows/python-tests.yaml/badge.svg)](https://github.com/mlverse/mall/actions/workflows/python-tests.yaml) \| <a href="https://cran.r-project.org/package=mall" data-original-href="https://cran.r-project.org/package=mall"><img src="https://www.r-pkg.org/badges/version/mall" alt="CRAN status" class="img-fluid"/></a> [![R check](https://github.com/mlverse/mall/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/mlverse/mall/actions/workflows/R-CMD-check.yaml) \| [![Package coverage](https://codecov.io/gh/mlverse/mall/branch/main/graph/badge.svg)](https://app.codecov.io/gh/mlverse/mall?branch=main)

<!-- badges: end -->

Use Large Language Models (LLM) to run Natural Language Processing (NLP) 
operations against your data. It takes advantage of the LLMs general language
training in order to get the predictions, thus removing the need to train a new
NLP model. `mall` is available for R and Python.

It works by running multiple LLM predictions against your data. The predictions
are processed row-wise over a specified column. It relies on the "one-shot" 
prompt technique to instruct the LLM on a particular NLP operation to perform. 
The package includes prompts to perform the following specific NLP operations:

-   [Sentiment analysis](#sentiment)
-   [Text summarizing](#summarize)
-   [Classify text](#classify)
-   [Extract one, or several](#extract), specific pieces information from the text
-   [Translate text](#translate)
-   [Verify that something is true](#verify) about the text (binary)

For other NLP operations, `mall` offers the ability for you to [write your own prompt](#custom-prompt).

<img src="site/images/dplyr.png" style="float:left;margin-right:20px;margin-top:22px;" width="85"/>

In **R** The functions inside `mall` are designed to easily work with piped 
commands, such as `dplyr`.

``` r
reviews |>
  llm_sentiment(review)
```

<img src="site/images/polars.png" style="float:left;margin-right:10px;" width="95"/>

In **Python**, `mall` is a library extension to [Polars](https://pola.rs/).

``` python
reviews.llm.sentiment("review")
```

## Motivation

We want to new find new ways to help data scientists use LLMs in their daily work.
Unlike the familiar interfaces, such as chatting and code completion, this 
interface runs your text data directly against the LLM. This package is inspired
by the SQL AI functions now offered by vendors such as [Databricks](https://docs.databricks.com/en/large-language-models/ai-functions.html) 
and Snowflake. 

The LLM's flexibility, allows for it to adapt to the subject of your data, and
provide surprisingly accurate predictions. This saves the data scientist the 
need to write and tune an NLP model.

In recent times, the capabilities of LLMs that can run locally in your computer 
have increased dramatically.  This means that these sort of analysis can run in 
your machine with good accuracy. It also makes it possible to take 
advantage of LLMs at your institution, since the data will not leave the 
corporate network. Additionally, LLM management and integration platforms, such
as [Ollama](https://ollama.com/), are now very easy to setup and use. `mall`
uses Ollama as to interact with local LLMs.

In its latest version, `mall` lets you **use external LLMs such as
[OpenAI](https://openai.com/), [Gemini](https://gemini.google.com/) and
[Anthropic](https://www.anthropic.com/)**. In R, `mall` uses the
[`ellmer`](https://ellmer.tidyverse.org/index.html)
package to integrate with the external LLM, and the 
[`chatlas`](https://posit-dev.github.io/chatlas/) package to integrate in Python.

## Install `mall` {#get-started}

Install the package to get started:

::: {.panel-tabset group="language"}
## R

Official version from CRAN:

``` r
install.packages("mall")
```

Development version from GitHub *(required for remote LLM integration)*:

``` r
pak::pak("mlverse/mall/r")
```

## Python

Official version from PyPi:

``` python
pip install mlverse-mall
```

Development version from GitHub:

``` python
pip install "mlverse-mall @ git+https://git@github.com/mlverse/mall.git#subdirectory=python"
```
:::

## Setup the LLM

Choose one of the two following options to setup LLM connectivity:

### Local LLMs, via Ollama {#local-llms}

-   [Download Ollama from the official website](https://ollama.com/download)

-   Install and start Ollama in your computer

::: {.panel-tabset group="language"}
## R

-   Install Ollama in your machine. The `ollamar` package's website provides this 
[Installation guide](https://hauselin.github.io/ollama-r/#installation)

-   Download an LLM model. For example, I have been developing this package using 
Llama 3.2 to test. To get that model you can run:

    ``` r
    ollamar::pull("llama3.2")
    ```

## Python

-   Install the official Ollama library

    ``` python
    pip install ollama
    ```

-   Download an LLM model. For example, I have been developing this package
using Llama 3.2 to test. To get that model you can run:

    ``` python
    import ollama
    ollama.pull('llama3.2')
    ```
:::

### Remote LLMs {#remote-llms}

::: {.panel-tabset group="language"}
## R

`mall` uses the `ellmer` package as the integration point to the LLM. This package supports multiple providers such as OpenAI, Anthropic, Google Gemini, etc.

-   Install `ellmer`

    ``` r
    install.packages("ellmer")
    ```

-   Refer to `ellmer`'s documentation to find out how to setup the connections with your selected provider: <https://ellmer.tidyverse.org/reference/index.html#chatbots>

-   Let `mall` know which `ellmer` object to use during the R session. To do this, call `llm_use()`. Here is an example of using OpenAI:

    ```{r}
    library(mall)
    library(ellmer)
    chat <- chat_openai()
    llm_use(chat)
    ```

**Set a default LLM for your R session**

As a convenience, `mall` is able to automatically establish a connection with the
LLM at the beginning o R session. To do this you can use the `.mall_chat` option:

```r
options(.mall_chat =  ellmer::chat_openai(model = "gpt-4o"))
```

Add this line to your *.Rprofile* file in order for that code to run every time
you start R.  You can call `usethis::edit_r_profile()` to open your .Rprofile
file so you can add the option. 

## Python

`mall` uses the `chatlas` package as the integration point to the LLM. This 
package supports multiple providers such as OpenAI, Anthropic, Google Gemini, etc.

-   Install the `chatlas` library

    ``` python
    pip install chatlas
    ```

-   Refer to `chatlas`'s documentation to find out how to setup the connections
with your selected provider: <https://posit-dev.github.io/chatlas/reference/#chat-model-providers>

-   Let `mall` know which `chatlas` object to use during the Python session. 
To do this, call `llm_use()`. Here is an example of using OpenAI:

    ``` python
    import mall
    from chatlas import ChatOpenAI

    chat = ChatOpenAI()

    data = mall.MallData
    reviews = data.reviews

    reviews.llm.use(chat)
    ```
:::

## LLM functions

We will start with loading a very small data set contained in `mall`. It has 
3 product reviews that we will use as the source of our examples.

::: {.panel-tabset group="language"}
## R

```{r}
#| include: false

mall::llm_use("ollama", "llama3.2", seed = 100, .cache = "_readme_cache", .force = TRUE)
```

```{r}
library(mall)
data("reviews")

reviews
```

## Python

```{python}
#| include: false
import polars as pl
pl.Config(fmt_str_lengths=100)
pl.Config.set_tbl_hide_dataframe_shape(True)  
pl.Config.set_tbl_hide_column_data_types(True)
```

```{python}
import mall 
data = mall.MallData
reviews = data.reviews

reviews 
```
:::

### Sentiment {#sentiment}

Automatically returns "positive", "negative", or "neutral" based on the text.

::: {.panel-tabset group="language"}
## R

```{r}
reviews |>
  llm_sentiment(review)
```

For more information and examples visit this function's [R reference page](reference/llm_sentiment.qmd)

## Python

```{python}
reviews.llm.sentiment("review")
```

For more information and examples visit this function's [Python reference page](reference/MallFrame.qmd#mall.MallFrame.sentiment)
:::

### Summarize {#summarize}

There may be a need to reduce the number of words in a given text. Typically to 
make it easier to understand its intent. The function has an argument to control 
the maximum number of words to output (`max_words`):

::: {.panel-tabset group="language"}
## R

```{r}
reviews |>
  llm_summarize(review, max_words = 5)
```

For more information and examples visit this function's [R reference page](reference/llm_summarize.qmd)

## Python

```{python}
reviews.llm.summarize("review", 5)
```

For more information and examples visit this function's [Python reference page](reference/MallFrame.qmd#mall.MallFrame.summarize)
:::

### Classify {#classify}

Use the LLM to categorize the text into one of the options you provide:

::: {.panel-tabset group="language"}
## R

```{r}
reviews |>
  llm_classify(review, c("appliance", "computer"))
```

For more information and examples visit this function's [R reference page](reference/llm_classify.qmd)

## Python

```{python}
reviews.llm.classify("review", ["computer", "appliance"])
```

For more information and examples visit this function's [Python reference page](reference/MallFrame.qmd#mall.MallFrame.classify)
:::

### Extract {#extract}

One of the most interesting use cases Using natural language, we can tell the 
LLM to return a specific part of the text. In the following example, we request 
that the LLM return the product being referred to. We do this by simply saying 
"product". The LLM understands what we *mean* by that word, and looks for that 
in the text.

::: {.panel-tabset group="language"}
## R

```{r}
reviews |>
  llm_extract(review, "product")
```

For more information and examples visit this function's [R reference page](reference/llm_extract.qmd)

## Python

```{python}
reviews.llm.extract("review", "product")
```

For more information and examples visit this function's [Python reference page](reference/MallFrame.qmd#mall.MallFrame.extract)
:::

### Verify {#verify}

This functions allows you to check and see if a statement is true, based on the
provided text. By default, it will return a 1 for "yes", and 0 for "no". This 
can be customized.

::: {.panel-tabset group="language"}
## R

```{r}
reviews |>
  llm_verify(review, "is the customer happy with the purchase")
```

For more information and examples visit this function's [R reference page](reference/llm_verify.qmd)

## Python

```{python}
reviews.llm.verify("review", "is the customer happy with the purchase")
```

For more information and examples visit this function's [Python reference page](reference/MallFrame.qmd#mall.MallFrame.verify)
:::

### Translate {#translate}

As the title implies, this function will translate the text into a specified 
language. What is really nice, it is that you don't need to specify the language
of the source text. Only the target language needs to be defined. The 
translation accuracy will depend on the LLM

::: {.panel-tabset group="language"}
## R

```{r}
reviews |>
  llm_translate(review, "spanish")
```

For more information and examples visit this function's [R reference page](reference/llm_translate.qmd)

## Python

```{python}
reviews.llm.translate("review", "spanish")
```

For more information and examples visit this function's [Python reference page](reference/MallFrame.qmd#mall.MallFrame.translate)
:::

### Custom prompt {#custom-prompt}

It is possible to pass your own prompt to the LLM, and have `mall` run it 
against each text entry:

::: {.panel-tabset group="language"}
## R

```{r}
my_prompt <- paste(
  "Answer a question.",
  "Return only the answer, no explanation",
  "Acceptable answers are 'yes', 'no'",
  "Answer this about the following text, is this a happy customer?:"
)

reviews |>
  llm_custom(review, my_prompt)
```

For more information and examples visit this function's [R reference page](reference/llm_custom.qmd)

## Python

```{python}
my_prompt = (
    "Answer a question."
    "Return only the answer, no explanation"
    "Acceptable answers are 'yes', 'no'"
    "Answer this about the following text, is this a happy customer?:"
)

reviews.llm.custom("review", prompt = my_prompt)
```

For more information and examples visit this function's [Python reference page](reference/MallFrame.qmd#mall.MallFrame.custom)
:::

## Model selection and settings

#### Local LLMs via Ollama {#settings-local}

You can set the model and its options to use when calling the LLM. In this case,
we refer to options as model specific things that can be set, such as seed or 
temperature.

::: {.panel-tabset group="language"}
## R

Invoking an `llm` function will automatically initialize a model selection if 
you don't have one selected yet. If there is only one option, it will pre-select
it for you. If there are more than one available models, then `mall` will 
present you as menu selection so you can select which model you wish to use.

Calling `llm_use()` directly will let you specify the model and backend to use.
You can also setup additional arguments that will be passed down to the function
that actually runs the prediction. In the case of Ollama, that function is [`chat()`](https://hauselin.github.io/ollama-r/reference/chat.html).

The model to use, and other options can be set for the current R session

```{r}
#| eval: false
llm_use("ollama", "llama3.2", seed = 100, temperature = 0)
```

## Python

The model and options to be used will be defined at the Polars data frame object 
level. If not passed, the default model will be **llama3.2**.

```{python}
#| eval: false
reviews.llm.use("ollama", "llama3.2", options = dict(seed = 100))
```
:::

#### Remote LLMs

The provider and model selection will be based on the chat object you create. 
Any model related setting, such as temperature, seed and others, should be
set at the time of the object creation as well.

::: {.panel-tabset group="language"}
## R

```{r}
#| eval: false
library(mall)
library(ellmer)
chat <- chat_openai(model = "gpt-4o", seed = 100)
llm_use(chat)
```

## Python

```{python}
#| eval: false
import mall
from chatlas import ChatOpenAI
chat = ChatOpenAI(model = "gpt-4o",  seed= 100)
data = mall.MallData
reviews = data.reviews
reviews.llm.use(chat)
```
:::


## Results caching

By default `mall` caches the requests and corresponding results from a given 
LLM run. Each response is saved as individual JSON files. By default, the folder
name is `_mall_cache`. The folder name can be customized, if needed. Also, the
caching can be turned off by setting the argument to empty (`""`).

::: {.panel-tabset group="language"}
## R

```{r}
#| eval: false
llm_use(.cache = "_my_cache")
```

To turn off:

```{r}
#| eval: false
llm_use(.cache = "")
```

## Python

```{python}
#| eval: false
reviews.llm.use(_cache = "my_cache")
```

To turn off:

```{python}
#| eval: false
reviews.llm.use(_cache = "")
```
:::

For more information see the [Caching Results](articles/caching.qmd) article.

## Key considerations

The main consideration is **cost**. Either, time cost, or money cost.

If using this method with an LLM locally available, the cost will be a long 
running time. Unless using a very specialized LLM, a given LLM is a general 
model. It was fitted using a vast amount of data. So determining a response for
each row, takes longer than if using a manually created NLP model. The default
model used in Ollama is [Llama 3.2](https://ollama.com/library/llama3.2), which 
was fitted using 3B parameters.

If using an external LLM service, the consideration will need to be for the 
billing costs of using such service. Keep in mind that you will be sending a 
lot of data to be evaluated.

Another consideration is the novelty of this approach. Early tests are providing
encouraging results. But you, as an user, will still need to keep in mind that 
the predictions will not be infallible, so always check the output. At this time,
I think the best use for this method, is for a quick analysis.

## Vector functions

::: {.panel-tabset group="language"}
## R

`mall` includes functions that expect a vector, instead of a table, to run the
predictions. This should make it easier to test things, such as custom prompts
or results of specific text. Each `llm_` function has a corresponding `llm_vec_`
function:


```{r}
llm_vec_sentiment("I am happy")
```

```{r}
llm_vec_translate("Este es el mejor dia!", "english")
```

## Python 

`mall` is also able to process vectors contained in a `list` object. This allows
us to avoid having to convert a list of texts without having to first convert
them into a single column data frame. To use, initialize a new `LLMVec` class
object with either an Ollama model, or a `chatlas` `Chat` object, and then
access the same NLP functions as the Polars extension.

```{python}
# Initialize a Chat object
from chatlas import ChatOllama
chat = ChatOllama(model = "llama3.2")

# Pass it to a new LLMVec
from mall import LLMVec
llm = LLMVec(chat)    
```

Access the functions via the new LLMVec object, and pass the text to be processed.

```{python}
llm.sentiment(["I am happy", "I am sad"])
```

```{python}
llm.translate(["Este es el mejor dia!"], "english")
```

For more information visit the reference page: [LLMVec](reference/LLMVec.qmd)
:::