|
| 1 | +================== |
| 2 | +Welcome to RETURNN |
| 3 | +================== |
| 4 | + |
| 5 | +`GitHub repository <https://github.com/rwth-i6/returnn>`__. |
| 6 | +`RETURNN paper 2016 <https://arxiv.org/abs/1608.00895>`_, |
| 7 | +`RETURNN paper 2018 <https://arxiv.org/abs/1805.05225>`_. |
| 8 | + |
| 9 | +RETURNN - RWTH extensible training framework for universal recurrent neural networks, |
| 10 | +is a Theano/TensorFlow-based implementation of modern recurrent neural network architectures. |
| 11 | +It is optimized for fast and reliable training of recurrent neural networks in a multi-GPU environment. |
| 12 | + |
| 13 | +The high-level features and goals of RETURNN are: |
| 14 | + |
| 15 | +* **Simplicity** |
| 16 | + |
| 17 | + * Writing config / code is simple & straight-forward (setting up experiment, defining model) |
| 18 | + * Debugging in case of problems is simple |
| 19 | + * Reading config / code is simple (defined model, training, decoding all becomes clear) |
| 20 | + |
| 21 | +* **Flexibility** |
| 22 | + |
| 23 | + * Allow for many different kinds of experiments / models |
| 24 | + |
| 25 | +* **Efficiency** |
| 26 | + |
| 27 | + * Training speed |
| 28 | + * Decoding speed |
| 29 | + |
| 30 | +All items are important for research, decoding speed is esp. important for production. |
| 31 | + |
| 32 | +See our `Interspeech 2020 tutorial "Efficient and Flexible Implementation of Machine Learning for ASR and MT" video <https://www.youtube.com/watch?v=wPKdYqSOlAY>`__ |
| 33 | +(`slides <https://www-i6.informatik.rwth-aachen.de/publications/download/1154/Zeyer--2020.pdf>`__) |
| 34 | +with an introduction of the core concepts. |
| 35 | + |
| 36 | +More specific features include: |
| 37 | + |
| 38 | +- Mini-batch training of feed-forward neural networks |
| 39 | +- Sequence-chunking based batch training for recurrent neural networks |
| 40 | +- Long short-term memory recurrent neural networks |
| 41 | + including our own fast CUDA kernel |
| 42 | +- Multidimensional LSTM (GPU only, there is no CPU version) |
| 43 | +- Memory management for large data sets |
| 44 | +- Work distribution across multiple devices |
| 45 | +- Flexible and fast architecture which allows all kinds of encoder-attention-decoder models |
| 46 | + |
| 47 | +See `documentation <https://returnn.readthedocs.io/>`__. |
| 48 | +See `basic usage <https://returnn.readthedocs.io/en/latest/basic_usage.html>`__ |
| 49 | +and `technological overview <https://returnn.readthedocs.io/en/latest/tech_overview.html>`__. |
| 50 | + |
| 51 | +`Here is the video recording of a RETURNN overview talk <https://www-i6.informatik.rwth-aachen.de/web/Software/returnn/downloads/workshop-2019-01-29/01.recording.cut.mp4>`_ |
| 52 | +(`slides <https://www-i6.informatik.rwth-aachen.de/web/Software/returnn/downloads/workshop-2019-01-29/01.returnn-overview.session1.handout.v1.pdf>`__, |
| 53 | +`exercise sheet <https://www-i6.informatik.rwth-aachen.de/web/Software/returnn/downloads/workshop-2019-01-29/01.exercise_sheet.pdf>`__; |
| 54 | +hosted by eBay). |
| 55 | + |
| 56 | +There are `many example demos <https://github.com/rwth-i6/returnn/blob/master/demos/>`_ |
| 57 | +which work on artificially generated data, |
| 58 | +i.e. they should work as-is. |
| 59 | + |
| 60 | +There are `some real-world examples <https://github.com/rwth-i6/returnn-experiments>`_ |
| 61 | +such as setups for speech recognition on the Switchboard or LibriSpeech corpus. |
| 62 | + |
| 63 | +Some benchmark setups against other frameworks |
| 64 | +can be found `here <https://github.com/rwth-i6/returnn-benchmarks>`_. |
| 65 | +The results are in the `RETURNN paper 2016 <https://arxiv.org/abs/1608.00895>`_. |
| 66 | +Performance benchmarks of our LSTM kernel vs CuDNN and other TensorFlow kernels |
| 67 | +are in `TensorFlow LSTM benchmark <https://returnn.readthedocs.io/en/latest/tf_lstm_benchmark.html>`__. |
| 68 | + |
| 69 | +There is also `a wiki <https://github.com/rwth-i6/returnn/wiki>`_. |
| 70 | +Questions can also be asked on |
| 71 | +`StackOverflow using the RETURNN tag <https://stackoverflow.com/questions/tagged/returnn>`_. |
| 72 | + |
| 73 | +.. image:: https://github.com/rwth-i6/returnn/workflows/CI/badge.svg |
| 74 | + :target: https://github.com/rwth-i6/returnn/actions |
0 commit comments