Skip to content

Merge index over rsync or extra index #133

@eshattow

Description

@eshattow

Hello,

I am reviewing Home Assistant home-assistant/wheels workflow which provides a wheels builder for that software ecosystem. There is used index-503 presumably as some externally administrated task for hosting a set of wheels at the official Home Assistant wheels repository https://wheels.home-assistant.io

When building a forked repository of these Home Assistant component docker images (the various parts of that build ecosystem), initially the wheels build outputs directory as-built are simply rsync-over-ssh to a custom target host, and then in a later section of the same workflow it refers to the custom URL input for subsequent pip install use but there is not any PEP503 index built as part of the workflow. The required wheels are then built again because they are not found as there is no index built yet until some unrelated administration action occurs outside the scope of the build.

What I think the needs are:

Initially there needs to be a PEP503 index generated locally and that will be transferred (rsync ssh) along with the compiled wheels. The exception occurs if there is a pre-existing remote index, it is not acceptable to disrupt an existing index overwriting it with less than a complete view of the wheels after rsync has completed; There may be unknown quantity of wheels files data on the remote and so first downloading the entire wheels files content of a remote PEP503 to generate a complete view of the wheels with index-503 is to be avoided. Also we may not wish to allow arbitrary code execution on the remote server beyond rsync, making it difficult to assume in the build system that all remote ssh rsync targets would also allow code execution and that index-503 would be available.

If the remote target is web-accessible then can index-503 support a merge-with-remote feature?

Also, would it be feasible to implement an rsync compatible mode?

What do you think?

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions