Skip to content

add bytes conversion #5111

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ inventory = { version = "0.3.5", optional = true }
# crate integrations that can be added using the eponymous features
anyhow = { version = "1.0.1", optional = true }
bigdecimal = {version = "0.4", optional = true }
bytes = { version = "1.10.1", optional = true }
chrono = { version = "0.4.25", default-features = false, optional = true }
chrono-tz = { version = ">= 0.10, < 0.11", default-features = false, optional = true }
either = { version = "1.9", optional = true }
Expand Down Expand Up @@ -125,6 +126,7 @@ full = [
# "multiple-pymethods", # Not supported by wasm
"anyhow",
"bigdecimal",
"bytes",
"chrono",
"chrono-tz",
"either",
Expand Down
3 changes: 3 additions & 0 deletions guide/src/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,9 @@ Adds a dependency on [anyhow](https://docs.rs/anyhow). Enables a conversion from
### `bigdecimal`
Adds a dependency on [bigdecimal](https://docs.rs/bigdecimal) and enables conversions into its [`BigDecimal`](https://docs.rs/bigdecimal/latest/bigdecimal/struct.BigDecimal.html) type.

### `bytes`
Adds a dependency on [bytes](https://docs.rs/bytes/latest/bytes) and enables conversions into its [`Bytes`](https://docs.rs/bytes/latest/bytes/struct.Bytes.html) and [`BytesMut`](https://docs.rs/bytes/latest/bytes/struct.BytesMut.html) types.

### `chrono`

Adds a dependency on [chrono](https://docs.rs/chrono). Enables a conversion from [chrono](https://docs.rs/chrono)'s types to python:
Expand Down
1 change: 1 addition & 0 deletions newsfragments/5111.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add bytes to/from python conversions.
161 changes: 161 additions & 0 deletions src/conversions/bytes.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
#![cfg(feature = "bytes")]

//! Conversions to and from [bytes](https://docs.rs/bytes/latest/bytes/)'s [`Bytes`] and
//! [`BytesMut`] types.
//!
//! This is useful for efficiently converting Python's `bytes` and `bytearray` types efficiently.
//!
//! # Setup
//!
//! To use this feature, add in your **`Cargo.toml`**:
//!
//! ```toml
//! [dependencies]
#![doc = concat!("pyo3 = { version = \"", env!("CARGO_PKG_VERSION"), "\", features = [\"bytes\"] }")]
//! bytes = "1.10"
//!
//! Note that you must use compatible versions of bytes and PyO3.
//!
//! # Example
//!
//! Rust code to create functions which return `Bytes` or take `Bytes` as arguements:
//!
//! ```rust,no_run
//! use pyo3::prelude::*;
//!
//! #[pyfunction]
//! fn get_message_bytes() -> Bytes {
//! Bytes::from(b"Hello Python!".to_vec())
//! }
Comment on lines +26 to +29
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be more efficient to create a PyBytes object directly here. This is an example of why I'm not a fan of the Rust to Python conversion.

//!
//! #[pyfunction]
//! fn num_bytes(bytes: Bytes) -> usize {
//! bytes.len()
//! }
Comment on lines +31 to +34
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example is not that great, the user should just use &[u8] in this case. Please change the example to something where the Bytes is actually needed.

//!
//! #[pymodule]
//! fn my_module(m: &Bound<'_, PyModule>) -> PyResult<()> {
//! m.add_function(wrap_pyfunction!(get_message_bytes, m)?)?;
//! m.add_function(wrap_pyfunction!(num_bytes, m)?)?;
//! Ok(())
//! }
//! ```
//!
//! Python code that calls these functions:
//!
//! ```python
//! from my_module import get_message_bytes, num_bytes
//!
//! message = get_message_bytes()
//! assert message == b"Hello Python!"
//!
//! size = num_bytes(message)
//! assert size == 13
//! ```
use bytes::{Bytes, BytesMut};

use crate::conversion::IntoPyObject;
use crate::exceptions::PyTypeError;
use crate::instance::Bound;
use crate::types::any::PyAnyMethods;
use crate::types::{PyByteArray, PyByteArrayMethods, PyBytes, PyBytesMethods};
use crate::{FromPyObject, PyAny, PyErr, PyResult, Python};

impl FromPyObject<'_> for Bytes {
fn extract_bound(ob: &Bound<'_, PyAny>) -> PyResult<Self> {
if let Ok(bytes) = ob.downcast::<PyBytes>() {
Ok(Bytes::from((*bytes).as_bytes().to_vec()))
} else if let Ok(bytearray) = ob.downcast::<PyByteArray>() {
Ok(Bytes::from((*bytearray).to_vec()))
} else {
Err(PyTypeError::new_err("expected bytes or bytearray"))
}
}
}

impl FromPyObject<'_> for BytesMut {
fn extract_bound(ob: &Bound<'_, PyAny>) -> PyResult<Self> {
let bytes = ob.extract::<Bytes>()?;
Ok(BytesMut::from(bytes))
}
}

impl<'py> IntoPyObject<'py> for Bytes {
type Target = PyBytes;
type Output = Bound<'py, Self::Target>;
type Error = PyErr;

fn into_pyobject(self, py: Python<'py>) -> Result<Self::Output, Self::Error> {
Ok(PyBytes::new(py, &self))
}
}

impl<'py> IntoPyObject<'py> for BytesMut {
type Target = PyBytes;
type Output = Bound<'py, Self::Target>;
type Error = PyErr;

fn into_pyobject(self, py: Python<'py>) -> Result<Self::Output, Self::Error> {
Ok(PyBytes::new(py, &self))
}
}

#[cfg(test)]
mod tests {
use bytes::{Bytes, BytesMut};

use crate::{
conversion::IntoPyObject,
ffi,
types::{any::PyAnyMethods, PyBytes},
Python,
};

#[test]
fn test_bytes() {
Python::with_gil(|py| {
let py_bytes = py.eval(ffi::c_str!("b'foobar'"), None, None).unwrap();
let bytes: Bytes = py_bytes.extract().unwrap();
assert_eq!(bytes, Bytes::from(b"foobar".to_vec()));

let bytes = Bytes::from(b"foobar".to_vec()).into_pyobject(py).unwrap();
assert!(bytes.is_instance_of::<PyBytes>());
});
}

#[test]
fn test_bytearray() {
Python::with_gil(|py| {
let py_bytearray = py
.eval(ffi::c_str!("bytearray(b'foobar')"), None, None)
.unwrap();
let bytes: Bytes = py_bytearray.extract().unwrap();
assert_eq!(bytes, Bytes::from(b"foobar".to_vec()));
});
}

#[test]
fn test_bytes_mut() {
Python::with_gil(|py| {
let py_bytearray = py
.eval(ffi::c_str!("bytearray(b'foobar')"), None, None)
.unwrap();
let bytes: BytesMut = py_bytearray.extract().unwrap();
assert_eq!(bytes, BytesMut::from(&b"foobar"[..]));

let bytesmut = BytesMut::from(&b"foobar"[..]).into_pyobject(py).unwrap();
assert!(bytesmut.is_instance_of::<PyBytes>());
});
}

#[test]
fn test_bytearray_mut() {
Python::with_gil(|py| {
let py_bytearray = py
.eval(ffi::c_str!("bytearray(b'foobar')"), None, None)
.unwrap();
let bytes: BytesMut = py_bytearray.extract().unwrap();
assert_eq!(bytes, BytesMut::from(&b"foobar"[..]));
});
}
}
1 change: 1 addition & 0 deletions src/conversions/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

pub mod anyhow;
pub mod bigdecimal;
pub mod bytes;
pub mod chrono;
pub mod chrono_tz;
pub mod either;
Expand Down
Loading