Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add restrictions on blobstore consumption #3392

Open
yamarten opened this issue Jan 18, 2025 · 0 comments
Open

Add restrictions on blobstore consumption #3392

yamarten opened this issue Jan 18, 2025 · 0 comments

Comments

@yamarten
Copy link

Is your feature request related to a problem? Please describe.

Currently, there is an individual blob size limit (PDS_BLOB_UPLOAD_LIMIT) in the PDS implementation here, but the total is unlimited. Therefore, there is a concern that a small number of accounts will over-consume storage.

Describe the solution you'd like

Set an upper limit on the total blob size that can be uploaded within a certain period of time. This will allow PDS administrators to control their costs.

Describe alternatives you've considered

The rate limit of uploadBlob already exists. However, this does not directly reflect the PDS cost because the individual blob sizes vary.

You can also consider a way to simply limit the total amount of data without window. In this case, please note that if the PDS does not count at least GC or deletion, it is very disadvantageous for long-term users.

Additional context

There are several movements that accelerate the blobstore consumption.

  • Increase the upper limit of individual blob size to support Bluesky videos
  • A new tool called ATFile that uses PDS as file storage
  • Video streaming experiment using blob by GrayHaze
  • Reasing interest in image posting services
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant