Skip to content

feat: add GCS support as file provider (#2013) #2435

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 49 commits into
base: development
Choose a base branch
from
Open

feat: add GCS support as file provider (#2013) #2435

wants to merge 49 commits into from

Conversation

@Umang01-hash
Copy link
Member

Pull Request Template

Description:

  • Added GCS file provider in pkg/datasource/file/gcs .
  • Implemented Create , Remove , ReadDir , Open ,Stat and MakeDir using cloud.google.com/go/storage .
  • Simulates directories using object key prefixes .

Checklist:

  • I have formatted my code using goimport and golangci-lint.
  • All new code is covered by unit tests.
  • This PR does not decrease the overall code coverage.
  • I have reviewed the code comments and documentation for clarity.

Fixes #2013

NOTE: This PR is a continuation of PR #2117. Most of the work here belongs to Suryakantdsa. I only extended it to include common logger and metrics for File Systems.

Suryakantdsa and others added 30 commits October 3, 2025 23:53
@Suryakantdsa
feat(file/gcs): add GCS support with Go 1.25 and fix requested changes
a47f699
@Suryakantdsa
added gcs filestore and some testcase
d3dd786
@Suryakantdsa
Added GCS FIlestore
652914d
@Suryakantdsa
added testcase for Stat
8fafe1a
@Suryakantdsa
format code using goimports and fix lint issues via golangci-lint
e481b69
@Suryakantdsa
fix lint issue
0a2f863
@Suryakantdsa
refactor: update docs, logging, and contribution guidelines based on …
4528c4b 
…feedback
@Suryakantdsa
test: add missing coverage for GCS
0e468c5
@Suryakantdsa
docs(gcs): add EndPoint field in config example
a367972
@Umang01-hash
Merge branch 'development' into feat/gcs-file-provider
d1d73b4
@Umang01-hash
Merge branch 'development' into feat/gcs-file-provider
802e568
@Umang01-hash
Merge branch 'development' into feat/gcs-file-provider
7b1006c
@Suryakantdsa
refactor(tests): remove if-else by splitting into separate test funct…
441b089 
…ions
@Suryakantdsa
Merge branch 'feat/gcs-file-provider' of https://github.com/Suryakant…
5dd78e1 
…dsa/gofr into feat/gcs-file-provider
@Suryakantdsa
feat(file/gcs): add GCS support with Go 1.25 and fix requested changes
ec5ce82
@Suryakantdsa
Added GCS FIlestore
f30f9a1
@Suryakantdsa
added testcase for Stat
d38cb41
@Suryakantdsa
format code using goimports and fix lint issues via golangci-lint
c2c6a31
@Suryakantdsa
fix lint issue
e43697e
@Suryakantdsa
refactor: update docs, logging, and contribution guidelines based on …
d81576f 
…feedback
@Suryakantdsa
docs(gcs): add EndPoint field in config example
2d8782a
@Suryakantdsa
refactor(tests): remove if-else by splitting into separate test funct…
1d243d8 
…ions and fix linting issue
@Umang01-hash
Merge branch 'development' into feat/gcs-file-provider
60aa36f
@Suryakantdsa
feat(file/gcs): add GCS support with Go 1.25 and fix requested changes
23fe152
@Suryakantdsa
added gcs filestore and some testcase
b820a8d
@Suryakantdsa
Added GCS FIlestore
2d49098
@Suryakantdsa
added testcase for Stat
64eaf78
@Suryakantdsa
format code using goimports and fix lint issues via golangci-lint
ad4e0b8
@Suryakantdsa
fix lint issue
f119426
@Suryakantdsa
test: add missing coverage for GCS
391facb
Suryakantdsa and others added 17 commits October 13, 2025 17:44
@Suryakantdsa
docs(gcs): add EndPoint field in config example
b9d4d87
@Suryakantdsa
refactor(tests): remove if-else by splitting into separate test funct…
05d41fb 
…ions
@Suryakantdsa
feat(file/gcs): add GCS support with Go 1.25 and fix requested changes
ea87e09
@Suryakantdsa
Added GCS FIlestore
d026aeb
@Suryakantdsa
added testcase for Stat
cff2eb7
@Suryakantdsa
format code using goimports and fix lint issues via golangci-lint
4692870
@Suryakantdsa
fix lint issue
e084087
@Suryakantdsa
refactor: update docs, logging, and contribution guidelines based on …
006af26 
…feedback
@Suryakantdsa
docs(gcs): add EndPoint field in config example
02b872f
@Suryakantdsa
refactor(tests): remove if-else by splitting into separate test funct…
e2fb9b3 
…ions and fix linting issue
@Suryakantdsa
fix: handle nil metrics in Connect
dfc13b4
@Suryakantdsa
fix: handle nil metrics in Connect
071e2f9
@Umang01-hash
Merge branch 'development' into feat/gcs-file-provider
f8643ce
@Umang01-hash
Merge branch 'development' into feat/gcs-file-provider
f8e6fac
@Umang01-hash
extract out logger and metrics and use them for diff file providers
fa26ed1
@Umang01-hash
use LogFileoperation method to log and record metrics for gcs impleme…
762c927 
…ntation and removing separate logger for gcs
@Umang01-hash
rename common Log method and resolve linters
dda5b26
This was referenced Oct 16, 2025
Open
Open
@Umang01-hash
Copy link
Member Author

@Suryakantdsa I have updated this PR with the Seek, ReadAt and WriteAt method implemented by you.

I think we have refactored almost all of the review comments given on PR #2013. And moreover GCS implementation is now using these common components. (logger, metrics).

@akshat-kumar-singhal Can you take a final review on this.

@Suryakantdsa
Copy link

Suryakantdsa commented Oct 28, 2025 •
edited
Loading

Thank you so much @Umang01-hash for taking the time to review and implement the required GCS changes. Really appreciate your help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@aryanmehrotra aryanmehrotra Awaiting requested review from aryanmehrotra

@coolwednesday coolwednesday Awaiting requested review from coolwednesday

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

Datasource gofr-soc

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Google Cloud Storage (GCS) integration as File System

3 participants

@Umang01-hash @Suryakantdsa