|
| 1 | +--- |
| 2 | +title: Rucio - Exascale data management |
| 3 | +layout: gsdocs_proposal |
| 4 | +project: Rucio |
| 5 | +year: 2020 |
| 6 | +organization: |
| 7 | + - CERN |
| 8 | +--- |
| 9 | + |
| 10 | +## Description of project idea |
| 11 | + |
| 12 | +Rucio is a software framework that provides functionality to organize, manage, and access large volumes of scientific data using customisable policies. The data can be spread across globally distributed locations and across heterogeneous data centers, uniting different storage and network technologies as a single federated entity. Rucio offers advanced features such as distributed data recovery or adaptive replication, and is highly scalable, modular, and extensible. Rucio has been originally developed to meet the requirements of the high-energy physics experiment ATLAS, and is continuously extended to support LHC experiments and other diverse scientific communities. For example, the ATLAS experiment orchestrated an Exabyte of data transfer and processing and is growing rapidly. |
| 13 | + |
| 14 | +The current documentation is available from multiple places and in different formats, including [scientific articles](https://arxiv.org/abs/1902.09857), [readthedocs.io](https://rucio.readthedocs.io/en/latest/) with source in the [code](https://github.com/rucio/rucio/tree/master/doc/source), [Google Drive](https://drive.google.com/drive/folders/1EEN8l1dFjDSgavPrAMMooDjEodHP7aU7?usp=sharing), [Github](https://github.com/rucio/rucio), [DockerHub](https://hub.docker.com/u/rucio), or [Wikis](https://twiki.cern.ch/twiki/bin/view/AtlasComputing/AtlasDistributedComputing). This dispersion and diversity makes it difficult to pinpoint information and to recognise which information is outdated or superseded, wrong, or simply lacking in detail. Especially in Wikis the information is usually directed towards a single experiment instance of Rucio, however the underlying concepts can be applicable to multiple different instances. |
| 15 | + |
| 16 | +With this proposal we aim to achieve the following tasks: |
| 17 | + |
| 18 | + * Due to this wide dispersion of different documentation pieces, we want to move our documentation to one central place. Possibly based on mkdocs, but we are open to other documentation technology. |
| 19 | + * Restructure the documentation into clear and distinct parts. (User Howtos, Setup and tutorials, Operators documentation, Developer documentation, etc.) |
| 20 | + * Consolidate the different sources and integrate them into this new structure. |
| 21 | + * Write new documentation, to connect the different pieces where applicable. |
| 22 | + |
| 23 | +## Project duration |
| 24 | + |
| 25 | +We are open to both 3 month and 6 month projects, depending on what you think is required to achieve these tasks. |
| 26 | + |
| 27 | +## Related material |
| 28 | + |
| 29 | +- [Rucio Website](https://rucio.cern.ch) |
| 30 | +- [Rucio Documentation](https://rucio.readthedocs.io/en/latest/) |
| 31 | +- [Rucio Documentation Source](https://github.com/rucio/rucio/tree/master/doc/source) |
| 32 | +- [Rucio Journal Article](https://arxiv.org/abs/1902.09857) |
| 33 | +- [Rucio Github](https://github.com/rucio/rucio) |
| 34 | +- [Rucio Docker](https://hub.docker.com/u/rucio/) |
| 35 | +- [Rucio Workshop 2018](https://indico.cern.ch/event/676472/) |
| 36 | +- [Rucio Workshop 2019](https://indico.cern.ch/event/773489/) |
| 37 | +- [Rucio Workshop 2020](https://indico.cern.ch/event/867913/) |
| 38 | + |
| 39 | +## Expected results |
| 40 | +One central documentation page servicing the different types of users. The different existing documentation sources should all be consolidated in this documentation page and restructured. New documentation, especially to support the information flow, should be written. |
| 41 | + |
| 42 | +## Experience required |
| 43 | +General knowledge of Python, Docker, GIT is required. |
| 44 | + |
| 45 | +## Mentors |
| 46 | + * [Martin Barisits ](mailto:[email protected]), CERN |
| 47 | + * [Mario Lassnig ](mailto:[email protected]), CERN |
0 commit comments