Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is it possible to pipe an "incoming" SELECT stream into an "outgoing" INSERT stream? #11

Open
runeksvendsen opened this issue Apr 20, 2023 · 3 comments

Comments

@runeksvendsen
Copy link

I'm in need of:

  1. Streaming the output of a SELECT to Haskell
  2. Mapping over the stream in Haskell
  3. Piping the mapped stream into an INSERT

Is this possible? If so, how do I do it?

If no, please mark this issue as a feature request.

@ocharles
Copy link
Collaborator

You probably want https://hackage.haskell.org/package/streaming-postgresql-simple-0.2.0.5/docs/Database-PostgreSQL-Simple-Streaming.html#v:copyIn. It's been a long time since I worked on this library though so that's probably the most help I can provide initially.

@runeksvendsen
Copy link
Author

@ocharles thank you for the advice.

For what it's worth, I ended up using fold from postgresql-simple: https://hackage.haskell.org/package/postgresql-simple-0.6.5/docs/Database-PostgreSQL-Simple.html#v:fold

The copyIn functions seems too low level for my needs (streaming raw ByteStrings without parsing into rows).

@ocharles
Copy link
Collaborator

If you are ok either collecting all rows and doing a single insert or doing one insert per row that will work well, yep

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants