Skip to content

feat: web streams based encoding/csv #1993

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Apr 6, 2022
Merged

feat: web streams based encoding/csv #1993

merged 7 commits into from
Apr 6, 2022

Conversation

uki00a
Copy link
Contributor

@uki00a uki00a commented Mar 3, 2022

This PR tries to implement web streams based encoding/csv module.

I'm not very familiar with web streams, so any feedback would be appreciated!

Part of #1986

Changes

  • Added encoding/csv/stream.ts
  • Added encoding/csv/_io.ts to remove duplicate codes between encoding/csv.ts and encoding/csv/stream.ts

@crowlKats crowlKats self-requested a review March 4, 2022 00:03
crowlKats
crowlKats requested changes Mar 4, 2022
View reviewed changes
encoding/csv/stream.ts Outdated
comment?: string;
}

export class CSVStream {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could rather make CSVStream extend TransformStream, then the only thing you have to do is super() with the transformstream options; and the variables you defined can just be private properties.

This isnt necessary, but imo just a lot cleaner & direct

encoding/csv/stream.ts Outdated
@@ -0,0 +1,51 @@
import { Buffer, BufReader } from "../../io/buffer.ts";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now, this is going to be the problem: any of the new streams based APIs are not allowed to depend on any Deno.Reader/Writer APIs. You'll have to wait for #1970 to land

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@uki00a #1970 landed, so this should be now doable. BufReader & BufWriter dont exist anymore now (as they dont make sense for web streams). However, i see that csv also depends on textproto, which hasnt been ported yet

Copy link
Contributor Author

@uki00a uki00a Mar 5, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rewrote CSVStream not to depend on Deno.Reader in c573f87

TextProtoReader will no longer be needed to implement encoding/csv/stream 🙂

@crowlKats crowlKats mentioned this pull request Mar 4, 2022
Closed
33 tasks
@crowlKats crowlKats self-requested a review March 5, 2022 09:18
lucacasonato
lucacasonato reviewed Mar 31, 2022
View reviewed changes
Copy link
Member

@lucacasonato lucacasonato left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the CSVStream should be a TransformStream<string, string[]> rather than TransformStream<Uint8Array, string[]>. The text decoding can be done with TextDecoderStream by the user.

crowlKats
crowlKats reviewed Mar 31, 2022
View reviewed changes
Copy link
Member

@crowlKats crowlKats left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

except a nitpick, and luca's comment, this looks good

crowlKats
crowlKats approved these changes Apr 6, 2022
View reviewed changes
Copy link
Member

@crowlKats crowlKats left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks! the PR is still a draft; do you still have things to add? if not, i'd like to get this landed soon

@uki00a uki00a marked this pull request as ready for review April 6, 2022 04:38
@uki00a uki00a requested review from bartlomieju and kt3k as code owners April 6, 2022 04:38
@crowlKats crowlKats merged commit c5202f5 into denoland:main Apr 6, 2022
@uki00a uki00a deleted the csv_stream branch April 6, 2022 04:50
@uki00a
Copy link
Contributor Author

uki00a commented Apr 6, 2022

@crowlKats Thanks for the review! I'd like to add more tests later in a separate PR, since the tests are a little low

crowlKats pushed a commit to crowlKats/deno_std that referenced this pull request Apr 27, 2022
@uki00a @crowlKats
feat: web streams based encoding/csv (denoland#1993)
2b6fc3c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lucacasonato lucacasonato lucacasonato left review comments

@crowlKats crowlKats crowlKats approved these changes

@bartlomieju bartlomieju Awaiting requested review from bartlomieju

@kt3k kt3k Awaiting requested review from kt3k kt3k is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@uki00a @lucacasonato @crowlKats