Skip to content

Commit

Permalink
add base documentation in .md files
Browse files Browse the repository at this point in the history
  • Loading branch information
LeonieFreisinger committed Jan 31, 2023
1 parent 7197fe1 commit 6d33ed9
Show file tree
Hide file tree
Showing 2 changed files with 278 additions and 2 deletions.
209 changes: 209 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,209 @@
# Contributing to Test-of-Time
:+1::tada: First off, thanks for taking the time to contribute! :tada::+1:

Welcome to the NeuralProphet/ Test-of-time community and thank you for your contribution to its continued legacy.
We compiled this page with practical instructions and further resources to help you get started.

Please come join us on the NeuralProphet [Slack](https://join.slack.com/t/neuralprophet/shared_invite/zt-sgme2rw3-3dCH3YJ_wgg01IXHoYaeCg), you can message any core dev there.

## Get Started On This
We have a GitHub section tracking all [issues](https://github.com/ourownstory/test-of-time/issues) to be solved.
There are some issues marked with the label 'good first issue'. These issues are ideal to get started on for you as a beginner.
Feel free to assign your self to one on GitHub and have a try solving it. In case you need help, don't hestitate to reach out
via the commenting function in GutHub or Slack.

## Process
Here's a great [beginner's guide to contributing to a GitHub project](https://akrabat.com/the-beginners-guide-to-contributing-to-a-github-project/#to-sum-up).

In Summary:
* Fork the project & clone locally.
* Create an upstream remote and sync your local copy before you branch.
* Branch for each separate piece of work.
* Do the work, write good commit messages, and read the CONTRIBUTING file if there is one.
* Push to your origin repository.
* Create a new PR in GitHub. It is helpful to open a PR right when you start on a new issue to show that work is in progress.
* Respond to any code review feedback.

Please make sure to include tests and documentation with your code.

## Dev Install
Before starting it's a good idea to first create and activate a new virtual environment:
```
python3 -m venv <path-to-new-env>
source <path-to-new-env>/bin/activate
```
Now you can install test-of-time:

```
git clone <copied link from github>
cd test-of-time
pip install -e ".[dev]"
```

Please don't forget to run the dev setup script to install the hooks for black and pytest, and set git to fast forward only:
```
tot_dev_setup.py
git config pull.ff only
```

Notes:
* Including the optional `-e` flag will install test-of-time in "editable" mode, meaning that instead of copying the files into your virtual environment, a symlink will be created to the files where they are.
* The `tot_dev_setup` command runs the dev-setup script which installs appropriate git hooks for Black (pre-commit) and PyTest (pre-push).
* setting git to fast-forward only prevents accidental merges when using `git pull`.
* To run tests without pushing (or when the hook installation fails), run from test-of-time folder: `pytest -v`
* To run black without commiting (or when the hook installation fails): `python3 -m black {source_file_or_directory}`
* If running `tot_dev_setup.py` gives you a `no such file` error, try running `python ./scripts/tot_dev_setup.py`

In case you spot an error in this description, please reach out.

## Docstring

Docstrings need to be formatted according to [NumPy Style](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html#example-numpy) in order to display their API reference correctly using Spinx.
Please refer to [Pandas Docstring Guide](https://pandas.pydata.org/pandas-docs/stable/development/contributing_docstring.html#) for best practices.

The length of line inside docstrings block must be limited to 80 characters to fit into Jupyter documentation popups.

You can check for adherence to the style guide by running:
```sh
pydocstyle --convention=numpy path/my_file.py
```
(You may need to install the tool first. On Linux: `sudo apt install pydocstyle`.)


#### Example
See how Pandas does this for `melt` in their [melt documentation page](https://pandas.pydata.org/docs/reference/api/pandas.melt.html) and how it looks in the [melt docstring](https://github.com/pandas-dev/pandas/blob/v1.4.1/pandas/core/shared_docs.py#L153).

Docstring architecture sample:

```
def return_first_elements(n=5):
"""
Return the first elements of a given Series.
This function is mainly useful to preview the values of the
Series without displaying all of it.
Parameters
----------
n : int
Number of values to return.
Return
------
pandas.Series
Subset of the original series with the n first values.
See Also
--------
tail : Return the last n elements of the Series.
Examples
--------
If you have multi-index columns:
>>> df.columns = [list('ABC'), list('DEF')]
>>> df
A B C
D E F
0 a 1 2
1 b 3 4
2 c 5 6
"""
return self.iloc[:n]
```

## Typing

We try to use type annotations across the project to improve code readability and maintainability.

Please follow the official python recommendations for [type hints](https://docs.python.org/3/library/typing.html) and [PEP-484](https://peps.python.org/pep-0484/).

### Postponing the evaluation type annotations and python version

The Postponed Evaluation of Annotations [PEP 563](https://docs.python.org/3/whatsnew/3.7.html#pep-563-postponed-evaluation-of-annotations) provides major benefits for type annotations. To use them with our currently support python versions we must use the following syntax:

```python
from __future__ import annotations
```

### Circular imports with type annotations

When using type annotations, you may encounter circular imports. To avoid this, you can use the following pattern based on the [typing.TYPE_CHECKING](https://docs.python.org/3/library/typing.html#typing.TYPE_CHECKING) constant:

```python
from __future__ import annotations
from typing import TYPE_CHECKING

# Imports only needed for type checking
if TYPE_CHECKING:
from my_module import MyType
```

## Testing and Code Coverage

We are using `PyTest` to run tests within our projects. All tests can be found in `tests/` directory.

All tests can be triggered via the command:

```bash
pytest tests -v
```

Running specific tests can be done by running the command:

```bash
pytest tests -k "name_of_test"
```

We are using [pytest-cov](https://pypi.org/project/pytest-cov/) and [codecov](https://app.codecov.io/gh/ourownstory/neural_prophet) to create transparent code coverage reports.
To locally trigger and output a code coverage report via the commandline, run the following command:

```bash
pytest tests -v --cov=./
```


## Continous Integration

We are using Github Actions to setup a CI pipeline. The creation as well as single commits to a pull request trigger the CI pipeline.

Currently there is one workflow called `.github/worklfows/ci.yml` to trigger testing, create code coverage reports via [pytest-cov](https://pypi.org/project/pytest-cov/) and subsequently uploading reports via [codecov](https://app.codecov.io/gh/ourownstory/neural_prophet) for the major OS systems (Linux, Mac, Windows).


## Style
We deploy Black, the uncompromising code formatter, so there is no need to worry about style. Beyond that, where reasonable, for example for docstrings, we follow the [Google Python Style Guide](http://google.github.io/styleguide/pyguide.html)

As for Git practices, please follow the steps described at [Swiss Cheese](https://github.com/ourownstory/swiss-cheese/blob/master/git_best_practices.md) for how to git-rebase-squash when working on a forked repo. (Update: all PR are now squashed, so you can skip this step, but it's still good to know.)

### String formatting
Please use the more readable [f-string formatting style](https://docs.python.org/3/tutorial/inputoutput.html).

## Pull requests
Add one or multiple **labels** to your Pull request to be able to grasp its purpose at one glance.
List of current labels:

* documentation
* enhancement
* model
* plotting
* tutorial
* workflow
* help wanted
* need fix
* need review


## Issues
In case you spot a bug in the code or would like to suggest an improvement point,
it is best practice to open a new issue to address it. Add one or multiple **labels** to your newly created issue.
* good first issue
* bug
* duplicate
* wontfix
* plotting
* tutorial
* workflow
* model
* documentation




71 changes: 69 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,69 @@
# test-of-time
A small framework to benchmark forecasting models via backtesting
[![Pypi_Version](https://img.shields.io/pypi/v/test-of-time.svg)](https://pypi.org/project/test-of-time/)
[![Python Version](https://img.shields.io/badge/python-3.7+-blue?logo=python)](https://www.python.org/)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![Tests](https://github.com/ourownstory/test-of-time/actions/workflows/ci.yml/badge.svg)](https://github.com/ourownstory/test-of-time/actions/workflows/ci.yml)
[![Slack](https://img.shields.io/badge/[email protected]?logo=slack&logoColor=white&labelColor=3F0E40)](https://neuralprophet.slack.com/join/shared_invite/zt-sgme2rw3-3dCH3YJ_wgg01IXHoYaeCg#/shared-invite/email)
[![Downloads](https://static.pepy.tech/personalized-badge/test-of-time?period=total&units=international_system&left_color=black&right_color=blue&left_text=Downloads)](https://pepy.tech/project/test-of-time)


# Test-of-time
**Test-of-time** is a small framework to **benchmark time-series forecasting models** via backtesting. It offers a simple
template for benchmarking as well as a variety of ready-to-use datasets and models.

## Vision
Our vision is to become the central hub for benchmarking time series forecasts.
As a user, you need to either (1) find a model that is best suited for your use case, or (2) evaluate the performance of your model
for different use cases and derive its improvement potential.

We enable you to do this by integrating commonly used datasets and available models from open source or existing
frameworks.
We bundle these resources in one place and make them easily usable through a lean benchmarking template.
This enables fast benchmarking as a standard. In addition, we provide you with flexibility though customizable features,
such as customizing the individual experiments in a benchmark, adding your own models or dataset.

## Install
You can now install test-of-time directly with pip:
```shell
pip install test-of-time
```

### Install options
If you would like the most up to date version, you can instead install direclty from github:
```shell
git clone <copied link from github>
cd test-of-time
pip install .
```

##Features

### Core framework features
* offers a simple benchmarking template to compare multiple models on multiple datasets via backtesting
* provides various error metrics
* provides a selection of ready-to-use models and datasets

### Add-on features
* offers to add your own model

For a list of past changes, please refer to the [releases page](https://github.com/ourownstory/test-of-time/releases).


## Tutorials & Examples
You can find a first tutorial on how to use the framework in the [tutorial section](tutorials/BenchmarkingTemplates.ipynb).

## Contribution
We compiled a [Contributing to Test-of-Time](CONTRIBUTING.md) page with practical instructions and further resources
to help you become part of the family.

## Community
#### Discussion and Help
If you have any question or suggestion, you can participate with [our community right here on Github](https://github.com/ourownstory/test-of-time/discussions) (coming soon)

#### Slack Chat
We also have an active [Slack community](https://join.slack.com/t/neuralprophet/shared_invite/zt-sgme2rw3-3dCH3YJ_wgg01IXHoYaeCg) for our main framework NeuralProphet.
Come and join the conversation!

## About
Test-of-time is a sub-framework of the open-source community project NeuralProphet. It is supported by awesome
people like you. If you are interested in joining the project, please feel free to reach out to Oskar -
email on the [NeuralProphet Paper](https://arxiv.org/abs/2111.15397).

0 comments on commit 6d33ed9

Please sign in to comment.