-
Notifications
You must be signed in to change notification settings - Fork 178
RUST-1392 Add GridFS support: Implement public API with placeholder code #688
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
23 commits
Select commit
Hold shift + click to select a range
756869f
push public api skeleton
sanav33 b245a67
add public api documentation
sanav33 796c1c0
convert generic tfileid to bson
sanav33 d915614
run fmt
sanav33 8cbfffe
split GridFsStream type
sanav33 e5f6e99
add finish
sanav33 49cdb7d
make gridfs stream methods return result
sanav33 6a8fddb
add abort method for upload stream
sanav33 2849c7c
eliminate options in gridfsbucket struct
sanav33 72b39b3
change method names in gridfs api
sanav33 6fa8267
make type changes
sanav33 e7ae7a7
run fmt
sanav33 1487008
remove noCursorTimeout option
sanav33 8f8f0fc
fixed docs for options
sanav33 b34e2b5
change stream id field name to remvoe ambiguity
sanav33 92b2a53
update download stream id field name
sanav33 86ac29d
store options directly in bucket
sanav33 81672d4
change helpers for bucket
sanav33 9c94237
add futures version to methods
sanav33 4aa6cd9
remove futures-util import from gridfs
sanav33 fac86a0
remove futures dependency
sanav33 01238f5
fix destination trait bound
sanav33 8325459
remove link from doc for bucket constructor
sanav33 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,331 @@ | ||
pub mod options; | ||
|
||
use core::task::{Context, Poll}; | ||
use std::pin::Pin; | ||
|
||
use crate::{ | ||
concern::{ReadConcern, WriteConcern}, | ||
cursor::Cursor, | ||
error::Result, | ||
selection_criteria::SelectionCriteria, | ||
Database, | ||
}; | ||
use bson::{oid::ObjectId, Bson, DateTime, Document}; | ||
use options::*; | ||
use serde::{Deserialize, Serialize}; | ||
use tokio::io::ReadBuf; | ||
|
||
pub const DEFAULT_BUCKET_NAME: &str = "fs"; | ||
pub const DEFAULT_CHUNK_SIZE_BYTES: u32 = 255 * 1024; | ||
|
||
// Contained in a "chunks" collection for each user file | ||
struct Chunk { | ||
id: ObjectId, | ||
files_id: Bson, | ||
n: u32, | ||
// default size is 255 KiB | ||
data: Vec<u8>, | ||
} | ||
|
||
/// A collection in which information about stored files is stored. There will be one files | ||
/// collection document per stored file. | ||
#[derive(Serialize, Deserialize)] | ||
pub struct FilesCollectionDocument { | ||
id: Bson, | ||
length: i64, | ||
chunk_size: u32, | ||
upload_date: DateTime, | ||
filename: String, | ||
metadata: Document, | ||
} | ||
|
||
/// Struct for storing GridFS managed files within a [`Database`]. | ||
pub struct GridFsBucket { | ||
// Contains a "chunks" collection | ||
pub(crate) db: Database, | ||
pub(crate) options: GridFsBucketOptions, | ||
} | ||
|
||
// TODO: RUST-1395 Add documentation and example code for this struct. | ||
pub struct GridFsUploadStream { | ||
files_id: Bson, | ||
} | ||
|
||
impl GridFsUploadStream { | ||
/// Gets the file `id` for the stream. | ||
pub fn files_id(&self) -> &Bson { | ||
&self.files_id | ||
} | ||
|
||
/// Consumes the stream and uploads data in the stream to the server. | ||
pub async fn finish(self) { | ||
todo!() | ||
} | ||
|
||
/// Aborts the upload and discards the upload stream. | ||
pub async fn abort(self) { | ||
todo!() | ||
} | ||
} | ||
|
||
impl tokio::io::AsyncWrite for GridFsUploadStream { | ||
fn poll_write( | ||
self: Pin<&mut Self>, | ||
cx: &mut Context<'_>, | ||
buf: &[u8], | ||
) -> Poll<tokio::io::Result<usize>> { | ||
todo!() | ||
} | ||
|
||
fn poll_flush(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<tokio::io::Result<()>> { | ||
todo!() | ||
} | ||
|
||
fn poll_shutdown(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<tokio::io::Result<()>> { | ||
todo!() | ||
} | ||
} | ||
|
||
impl futures_util::AsyncWrite for GridFsUploadStream { | ||
fn poll_write( | ||
self: Pin<&mut Self>, | ||
cx: &mut Context<'_>, | ||
buf: &[u8], | ||
) -> Poll<core::result::Result<usize, futures_util::io::Error>> { | ||
todo!() | ||
} | ||
|
||
fn poll_flush( | ||
self: Pin<&mut Self>, | ||
cx: &mut Context<'_>, | ||
) -> Poll<core::result::Result<(), futures_util::io::Error>> { | ||
todo!() | ||
} | ||
|
||
fn poll_close( | ||
self: Pin<&mut Self>, | ||
cx: &mut Context<'_>, | ||
) -> Poll<core::result::Result<(), futures_util::io::Error>> { | ||
todo!() | ||
} | ||
} | ||
|
||
pub struct GridFsDownloadStream { | ||
files_id: Bson, | ||
} | ||
|
||
impl GridFsDownloadStream { | ||
/// Gets the file `id` for the stream. | ||
pub fn files_id(&self) -> &Bson { | ||
&self.files_id | ||
} | ||
} | ||
|
||
impl tokio::io::AsyncRead for GridFsDownloadStream { | ||
sanav33 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
fn poll_read( | ||
self: Pin<&mut Self>, | ||
cx: &mut Context<'_>, | ||
buf: &mut ReadBuf<'_>, | ||
) -> Poll<tokio::io::Result<()>> { | ||
todo!() | ||
} | ||
} | ||
|
||
impl futures_util::io::AsyncRead for GridFsDownloadStream { | ||
fn poll_read( | ||
self: Pin<&mut Self>, | ||
cx: &mut Context<'_>, | ||
buf: &mut [u8], | ||
) -> Poll<core::result::Result<usize, futures_util::io::Error>> { | ||
todo!() | ||
} | ||
} | ||
|
||
impl GridFsBucket { | ||
/// Gets the read concern of the [`GridFsBucket`]. | ||
pub fn read_concern(&self) -> Option<&ReadConcern> { | ||
self.options.read_concern.as_ref() | ||
} | ||
|
||
/// Gets the write concern of the [`GridFsBucket`]. | ||
pub fn write_concern(&self) -> Option<&WriteConcern> { | ||
self.options.write_concern.as_ref() | ||
} | ||
|
||
/// Gets the selection criteria of the [`GridFsBucket`]. | ||
pub fn selection_criteria(&self) -> Option<&SelectionCriteria> { | ||
self.options.selection_criteria.as_ref() | ||
} | ||
|
||
/// Opens a [`GridFsUploadStream`] that the application can write the contents of the file to. | ||
/// The application provides a custom file id. | ||
/// | ||
/// Returns a [`GridFsUploadStream`] to which the application will write the contents. | ||
pub async fn open_upload_stream_with_id( | ||
&self, | ||
id: Bson, | ||
filename: String, | ||
options: impl Into<Option<GridFsUploadOptions>>, | ||
) -> Result<GridFsUploadStream> { | ||
todo!() | ||
} | ||
|
||
/// Opens a [`GridFsUploadStream`] that the application can write the contents of the file to. | ||
/// The driver generates a unique [`Bson::ObjectId`] for the file id. | ||
/// | ||
/// Returns a [`GridFsUploadStream`] to which the application will write the contents. | ||
pub async fn open_upload_stream( | ||
&self, | ||
filename: String, | ||
options: impl Into<Option<GridFsUploadOptions>>, | ||
) -> Result<GridFsUploadStream> { | ||
self.open_upload_stream_with_id(Bson::ObjectId(ObjectId::new()), filename, options) | ||
.await | ||
} | ||
|
||
/// Uploads a user file to a GridFS bucket. The application supplies a custom file id. Uses the | ||
/// `tokio` crate's `AsyncRead` trait for the `source`. | ||
pub async fn upload_from_tokio_reader_with_id( | ||
&self, | ||
id: Bson, | ||
filename: String, | ||
source: impl tokio::io::AsyncRead, | ||
options: impl Into<Option<GridFsUploadOptions>>, | ||
) { | ||
todo!() | ||
} | ||
|
||
/// Uploads a user file to a GridFS bucket. The application supplies a custom file id. Uses the | ||
/// `futures-0.3` crate's `AsyncRead` trait for the `source`. | ||
pub async fn upload_from_futures_0_3_reader_with_id( | ||
&self, | ||
id: Bson, | ||
filename: String, | ||
source: impl futures_util::AsyncRead, | ||
options: impl Into<Option<GridFsUploadOptions>>, | ||
) { | ||
todo!() | ||
} | ||
|
||
/// Uploads a user file to a GridFS bucket. The driver generates a unique [`Bson::ObjectId`] for | ||
/// the file id. Uses the `tokio` crate's `AsyncRead` trait for the `source`. | ||
pub async fn upload_from_tokio_reader( | ||
&self, | ||
filename: String, | ||
source: impl tokio::io::AsyncRead, | ||
options: impl Into<Option<GridFsUploadOptions>>, | ||
) { | ||
self.upload_from_tokio_reader_with_id( | ||
Bson::ObjectId(ObjectId::new()), | ||
filename, | ||
source, | ||
options, | ||
) | ||
.await | ||
} | ||
|
||
/// Uploads a user file to a GridFS bucket. The driver generates a unique [`Bson::ObjectId`] for | ||
/// the file id. Uses the `futures-0.3` crate's `AsyncRead` trait for the `source`. | ||
pub async fn upload_from_futures_0_3_reader( | ||
&self, | ||
filename: String, | ||
source: impl futures_util::AsyncRead, | ||
options: impl Into<Option<GridFsUploadOptions>>, | ||
) { | ||
self.upload_from_futures_0_3_reader_with_id( | ||
Bson::ObjectId(ObjectId::new()), | ||
filename, | ||
source, | ||
options, | ||
) | ||
.await | ||
} | ||
|
||
/// Opens and returns a [`GridFsDownloadStream`] from which the application can read | ||
/// the contents of the stored file specified by `id`. | ||
pub async fn open_download_stream(&self, id: Bson) -> Result<GridFsDownloadStream> { | ||
todo!() | ||
} | ||
|
||
/// Opens and returns a [`GridFsDownloadStream`] from which the application can read | ||
/// the contents of the stored file specified by `filename` and the revision | ||
/// in `options`. | ||
pub async fn open_download_stream_by_name( | ||
&self, | ||
filename: String, | ||
options: impl Into<Option<GridFsDownloadByNameOptions>>, | ||
) -> Result<GridFsDownloadStream> { | ||
todo!() | ||
} | ||
|
||
/// Downloads the contents of the stored file specified by `id` and writes | ||
/// the contents to the `destination`. Uses the `tokio` crate's `AsyncWrite` | ||
/// trait for the `destination`. | ||
pub async fn download_to_tokio_writer( | ||
&self, | ||
id: Bson, | ||
destination: impl tokio::io::AsyncWrite, | ||
) { | ||
todo!() | ||
} | ||
|
||
/// Downloads the contents of the stored file specified by `id` and writes | ||
/// the contents to the `destination`. Uses the `futures-0.3` crate's `AsyncWrite` | ||
/// trait for the `destination`. | ||
pub async fn download_to_futures_0_3_writer( | ||
&self, | ||
id: Bson, | ||
destination: impl futures_util::AsyncWrite, | ||
) { | ||
todo!() | ||
} | ||
|
||
/// Downloads the contents of the stored file specified by `filename` and by | ||
/// the revision in `options` and writes the contents to the `destination`. Uses the | ||
/// `tokio` crate's `AsyncWrite` trait for the `destination`. | ||
pub async fn download_to_tokio_writer_by_name( | ||
&self, | ||
filename: String, | ||
destination: impl tokio::io::AsyncWrite, | ||
options: impl Into<Option<GridFsDownloadByNameOptions>>, | ||
) { | ||
todo!() | ||
} | ||
|
||
/// Downloads the contents of the stored file specified by `filename` and by | ||
/// the revision in `options` and writes the contents to the `destination`. Uses the | ||
/// `futures-0.3` crate's `AsyncWrite` trait for the `destination`. | ||
pub async fn download_to_futures_0_3_writer_by_name( | ||
&self, | ||
filename: String, | ||
destination: impl futures_util::AsyncWrite, | ||
options: impl Into<Option<GridFsDownloadByNameOptions>>, | ||
) { | ||
todo!() | ||
} | ||
|
||
/// Given an `id`, deletes the stored file's files collection document and | ||
/// associated chunks from a [`GridFsBucket`]. | ||
pub async fn delete(&self, id: Bson) { | ||
todo!() | ||
} | ||
|
||
/// Finds and returns the files collection documents that match the filter. | ||
pub async fn find( | ||
&self, | ||
filter: Document, | ||
options: impl Into<Option<GridFsBucketOptions>>, | ||
) -> Result<Cursor<FilesCollectionDocument>> { | ||
todo!() | ||
} | ||
|
||
/// Renames the stored file with the specified `id`. | ||
pub async fn rename(&self, id: Bson, new_filename: String) { | ||
todo!() | ||
} | ||
|
||
/// Drops the files associated with this bucket. | ||
pub async fn drop(&self) { | ||
todo!() | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.