Skip to content

Commit

Permalink
fixups
Browse files Browse the repository at this point in the history
  • Loading branch information
MarcoGorelli committed Jan 14, 2024
1 parent 1754cf4 commit 4ebcf11
Show file tree
Hide file tree
Showing 5 changed files with 61 additions and 34 deletions.
61 changes: 33 additions & 28 deletions docs/abs.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ Next, let's define `abs_i64` on the Rust side. The general idea will
be:

- A Series is backed by a ChunkedArray. Each chunk is an Arrow Array
which is continuous in memory. So we're going to start by iterating
over chunks.
- For each chunk, we iterate over the elements in that array.
which is contiguous in memory. So we're going to start by iterating
over chunks by calling `downcast_iter`.
- For each chunk, we iterate over the elements in that array (`into_iter`)
- Each element can either be `Some(i64)`, or `None`. If it's `None`,
we return `None`, whereas if it's `Some(i64)`, then we take its
absolute value.
Expand All @@ -50,7 +50,34 @@ fn abs_i64(inputs: &[Series]) -> PolarsResult<Series> {
}
```

NOTE: there are faster ways of implementing this particular operation. If you
Let's try this out. Make a Python file `run.py` with the following:
```python
import polars as pl
import minimal_plugin # noqa: F401

df = pl.DataFrame({
'a': [1, -1, None],
'b': [4.1, 5.2, -6.3],
'c': ['hello', 'everybody!', '!']
})
print(df.with_columns(pl.col('a').mp.abs_i64().name.suffix('_abs')))
```
If this outputs
```
shape: (3, 4)
┌──────┬──────┬────────────┬───────┐
│ a ┆ b ┆ c ┆ a_abs │
│ --- ┆ --- ┆ --- ┆ --- │
│ i64 ┆ f64 ┆ str ┆ i64 │
╞══════╪══════╪════════════╪═══════╡
│ 1 ┆ 4.1 ┆ hello ┆ 1 │
│ -1 ┆ 5.2 ┆ everybody! ┆ 1 │
│ null ┆ -6.3 ┆ ! ┆ null │
└──────┴──────┴────────────┴───────┘
```
then you did everything correctly!

> NOTE: there are faster ways of implementing this particular operation. If you
look at the Polars source code, you'll see that it's a bit different there.
The purpose of this exercise is to show you an implementation which is
explicit and generalisable enough that you can customise it according to your
Expand Down Expand Up @@ -130,29 +157,7 @@ fn abs_numeric(inputs: &[Series]) -> PolarsResult<Series> {
}
```

Let's try this out:
Now, if you return to `run.py`, you should be able to run
```python
import polars as pl
import minimal_plugin # noqa: F401

df = pl.DataFrame({
'a': [1, -1, None],
'b': [4.1, 5.2, -6.3],
'c': ['hello', 'everybody!', '!']
})
print(df.with_columns(pl.col('a', 'b').mp.abs_numeric().name.suffix('_abs')))
```
If this outputs
```
shape: (3, 5)
┌──────┬──────┬────────────┬───────┬───────┐
│ a ┆ b ┆ c ┆ a_abs ┆ b_abs │
│ --- ┆ --- ┆ --- ┆ --- ┆ --- │
│ i64 ┆ f64 ┆ str ┆ i64 ┆ f64 │
╞══════╪══════╪════════════╪═══════╪═══════╡
│ 1 ┆ 4.1 ┆ hello ┆ 1 ┆ 4.1 │
│ -1 ┆ 5.2 ┆ everybody! ┆ 1 ┆ 5.2 │
│ null ┆ -6.3 ┆ ! ┆ null ┆ 6.3 │
└──────┴──────┴────────────┴───────┴───────┘
```
then you did everything correctly!
```
6 changes: 2 additions & 4 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,7 @@

![](assets/image.png){: style="width:400px"}

This tutorial is just meant to get you started. We'll make a Polars Plugin with
some basic functionality - you can then customise it according to your own needs!
This tutorial is just meant to get you started. Once you've understood the
basics, you can customise the examples according to your needs.

Code to follow along is here: https://github.com/MarcoGorelli/polars-plugins-tutorial.

Let's get this party started!
1 change: 0 additions & 1 deletion docs/noop.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ Here are the files we'll need to create:

- `Cargo.toml`: file with Rust dependencies.
- `pyproject.toml`: file with Python build info.
- `requirements.txt`: Python build dependencies.

Start by copying the `Cargo.toml` and `pyproject.toml`
files from this repository - they contain the
Expand Down
22 changes: 22 additions & 0 deletions docs/prerequisites.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,27 @@
# 0. Prerequisites

## Knowledge

> "But you know what I like more than materialistic things? Knowledge." Tai Lopez
How much Rust do you need to know to write your own Polars plugin? Less than
you think.

If you pick up [The Rust Programming Language](https://doc.rust-lang.org/book/)
and can make it through the first 9 chapters, then I postulate
that you'll have enough knowledge at least 99% of inefficient `map_elements`
calls.
If you want to make a plugin which is generic enough that you can share
it with others, then you may need chapter 10 as well.

You'll also need basic Python knowledge: classes, decorators, and functions.

Alternatively, you could just clone this repo and then hack away
at the examples trial-and-error style until you get what you're looking
for - the compiler will probably help you more than you're expecting.

## Software

First, you should probably make new directory for this project.
Either clone https://github.com/MarcoGorelli/polars-plugins-tutorial,
or make a new directory.
Expand Down
5 changes: 4 additions & 1 deletion run.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
import polars as pl
import minimal_plugin # noqa: F401

import polars as pl
import minimal_plugin # noqa: F401

df = pl.DataFrame({
'a': [1, -1, None],
'b': [4.1, 5.2, -6.3],
'c': ['hello', 'everybody!', '!']
})
print(df.with_columns(pl.col('a', 'b').mp.abs_numeric().name.suffix('_abs')))
print(df.with_columns(pl.col('a').mp.abs_numeric().name.suffix('_abs')))

0 comments on commit 4ebcf11

Please sign in to comment.