Skip to content

Remove tokio dependency#259

Open
mineichen wants to merge 8 commits into
pepperoni21:masterfrom
mineichen:remove_tokio_dependency
Open

Remove tokio dependency#259
mineichen wants to merge 8 commits into
pepperoni21:masterfrom
mineichen:remove_tokio_dependency

Conversation

@mineichen

Copy link
Copy Markdown
Contributor
  • Remove tokio/async_stream/tokio_stream dependencies and use lighter futures_util/pin-project-lite instead
  • Unify streaming apis with a Lines adapter
    • Most of them had problems, if one Bytes would contain multiple lines, which could happen everywhere
  • use try_next() instead of .next() to improve readability
  • Share common code in streaming and non-streaming methods

Breaking:

  • Change ChatMessageResponse to return OllamaError instead of ()
  • Remove GenerationResponseStreamChunks and stream all items independently
  • Chat returns Error if one occurs... The Caller can still filter errors out and continue
  • Non-Streaming consistenly returns Reqwest-Error instead of OllamaError::Other if error-text cannot be gathered from response (sometimes it was res.text().await?, sometimes res.text().await.unwrap_or_else(|e| e.to_string()),)

Functionaity should stay almost the same. Sorry for the big chunk... What i'd like to send separately later:

  • Allow WASM-Build (currently reqwest::bytes_stream() is !Send)
  • Remove all stream properties from the requests and send them with WithSendField-heper
  • Introdce pub(crate) Ollama::send_request() to remove duplication of building request with common stuff

@tetuaoro

Copy link
Copy Markdown

Absolutely needed thx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants