Skip to content

Comments

Refactor initialization of storages & memory resource clients#143

Merged
vdusek merged 19 commits intomasterfrom
rm-base-resource-clients
May 10, 2024
Merged

Refactor initialization of storages & memory resource clients#143
vdusek merged 19 commits intomasterfrom
rm-base-resource-clients

Conversation

@vdusek
Copy link
Collaborator

@vdusek vdusek commented May 6, 2024

Description

  • Refactor initialization of storages & memory resource clients.
    • BaseStorage and BaseResourceClients are only ABC with abstract methods; all the creation-related code is moved to a separate module.
    • I created _creation_management.py helper modules in both storages/ and memory_storage_client/.
  • I had to move crawlee/storages/models.py to crawlee/models.py because of the import loops, I merged it with request.py. I am open to other ideas in this regard.

Related issues

Testing

  • Tests pass

@github-actions github-actions bot added this to the 89th sprint - Tooling team milestone May 6, 2024
@github-actions github-actions bot added the t-tooling Issues with this label are in the ownership of the tooling team. label May 6, 2024
@vdusek vdusek marked this pull request as ready for review May 6, 2024 13:06
@vdusek vdusek requested a review from janbuchar May 6, 2024 13:06
@vdusek vdusek force-pushed the rm-base-resource-clients branch from f091771 to 554ab4d Compare May 9, 2024 12:14
@vdusek vdusek requested a review from janbuchar May 10, 2024 14:48
@vdusek vdusek requested a review from janbuchar May 10, 2024 15:07
Copy link
Collaborator

@janbuchar janbuchar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolve my comments in any way you like, then feel free to merge 🙂

@vdusek vdusek merged commit c5cf64e into master May 10, 2024
@vdusek vdusek deleted the rm-base-resource-clients branch May 10, 2024 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants