Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add features #12

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 19 additions & 26 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,34 +1,27 @@
FROM ubuntu:16.04
MAINTAINER [email protected]
FROM python:3.5

RUN DEBIAN_FRONTEND=noninteractive \
apt-get update && apt-get install -y \
language-pack-en

ENV LANG en_US.UTF-8
ENV WORKON_HOME /usr/local/python-venvs
ENV GOMAXPROCS 1

RUN mkdir -p /usr/local/python-venvs
RUN mkdir -p /usr/go/
ENV GOPATH /usr/go/

RUN DEBIAN_FRONTEND=noninteractive \
apt-get update && apt-get install -y \
autoconf automake libtool build-essential \
python3 python3-pip git nodejs golang gosu

RUN pip3 install vex
RUN vex --python=python3.5 -m bench pip install -U pip
RUN mkdir -p /var/lib/cache/pip
ENV GOMAXPROCS 1

ADD servers /usr/src/servers
RUN cd /usr/src/servers && go build goecho.go && \
go get github.com/golang/groupcache/lru && go build gohttp.go
RUN vex bench pip --cache-dir=/var/lib/cache/pip \
install -r /usr/src/servers/requirements.txt

RUN vex bench pip freeze -r /usr/src/servers/requirements.txt
RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get install -y \
autoconf automake libtool build-essential nodejs golang

RUN mkdir -p /usr/local/python-venvs \
&& mkdir -p /usr/go \
&& pip3 install vex \
&& vex --python=python3.5 -m bench pip install -U pip \
&& mkdir -p /var/lib/cache/pip \
&& vex bench pip --cache-dir=/var/lib/cache/pip install -r /usr/src/servers/requirements.txt \
&& vex bench pip freeze -r /usr/src/servers/requirements.txt \
&& curl -L -o /usr/local/bin/gosu https://github.com/tianon/gosu/releases/download/1.10/gosu-$(dpkg --print-architecture | awk -F- '{ print $NF }') \
&& chmod +x /usr/local/bin/gosu \
&& cd /usr/src/servers \
&& go build goecho.go \
&& go get github.com/golang/groupcache/lru \
&& go build gohttp.go

EXPOSE 25000

Expand All @@ -37,4 +30,4 @@ VOLUME /tmp/sockets

ENTRYPOINT ["/entrypoint"]

ADD entrypoint /entrypoint
ADD entrypoint /entrypoint
158 changes: 150 additions & 8 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,160 @@ network performance of a variety of server frameworks.
The servers are run inside a Docker container for environment stability,
so to use this toolbench you need a reasonably recent Docker.

HTTP Servers Tested
-------------------

- HTML Report : http://espace-groupware.com/docs/vmbench/report-http.html
- Json Report : http://espace-groupware.com/docs/vmbench/report-http.json

=================== ==================== =============
Product Benchmark Key Comments
=================== ==================== =============
`AioHTTP/asyncio`_ http-asyncio-aiohttp
`AioHTTP/uvloop`_ http-uvloop-aiohttp
`Sanic`_ http-sanic uvloop
`Sanic`_ http-sanic-workers uvloop / multi-workers
`Gevent/WSGI`_ http-gevent-wsgi
`Gevent/Flask`_ http-gevent-flask
`Node.js`_ http-nodejs http module - Node.js 0.10.29
`Golang`_ http-golang net/http - Golang 2:1.3.3
=================== ==================== =============

.. _`AioHTTP/asyncio`: http://aiohttp.readthedocs.io
.. _`AioHTTP/uvloop`: https://github.com/MagicStack/uvloop
.. _`Sanic`: https://github.com/channelcat/sanic
.. _`Gevent/WSGI`: http://gevent.org/
.. _`Gevent/Flask`: http://flask.pocoo.org
.. _`Node.js`: https://nodejs.org/
.. _`Golang`: https://golang.org/

Screenshots
-----------

Requests/Seconds
::::::::::::::::

.. image:: http://espace-groupware.com/docs/vmbench/http-report-requests.png
:align: center

Latency
:::::::

.. image:: http://espace-groupware.com/docs/vmbench/http-report-latency.png
:align: center


Test Server Description
-----------------------

Information complémentaire sur les différences obtenues par rapport au test original sur https://magic.io/blog/uvloop-blazing-fast-python-networking/http-bench.html

- magic.io: Linux 4.4.5 (Ubuntu 16.04, x86_64) on Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz

- Me: Linux 4.4.0-57-generic (Docker/Debian 8.7, x86_64) on Intel(R) Xeon(R) CPU W3520 @ 2.67GHz

My server: (https://www.soyoustart.com/fr/offres/142sys4.xml)

- OVH - So you Start - 16G ECC - Intel(R) Xeon(R) CPU W3520 @ 2.67GHz 4c/8t SoftRaid 2x2 To

Installation
------------

Install the following:
Requirements:

- Docker
- Python 3
- Docker (tested on Docker 17.04.0-ce / Ubuntu 16.04)
- Python 3 (tested on Python 3.5.3 64bits)
- Wrk (tested on Wrk 4.0.1)
- Numpy

Build the docker image containing the servers being tested by running
``./build.sh``.
::

$ apt-get install -y python3-numpy wrk

$ git clone -b add_features https://github.com/srault95/vmbench

$ cd vmbench

# Build the docker image containing the servers
# being tested by running
$ ./build.sh

Running Tests
-------------

::

# The benchmarks can then be ran with
$ ./run_benchmarks --help

# Example for run all http tests:
$ ./run_benchmarks --save-html report-http.html --benchmarks http-*

# Run one test for Sanic Http Server with duration (30sec)
# and concurency (300)
$ ./run_benchmarks --save-html report-http.html \
--benchmarks http-uvloop-aiohttp \
--duration 30 --concurrency-levels 300

Debug
-----

Pour la mise au point des services à tester, vous pouvez lancement manuellement
le container Docker comme le ferait le script ``run_benchmark``

Avec un volume vers $PWD/servers:/usr/src/servers, vous pouvez éditer les services
sans relancer le build de l'image Docker.

::

# Update docker image: magic/benchmark
$ ./build.sh

$ docker run --rm -it -p 25000:25000 -e UID=0 -e GID=0 \
-v $PWD/servers:/usr/src/servers \
-v $PWD/.cache:/var/lib/cache \
-v $PWD/sockets:/tmp/sockets magic/benchmark \
vex bench python /usr/src/servers/sanic_http_server.py --addr=0.0.0.0:25000 --worker=1

# Go to navigator and open http://YOUR_IP:25000/[MSIZE]
# http://YOUR_IP:25000/1024

# Or run the http_client/echo_client
$ ./http_client --output-format=json --addr=127.0.0.1:25000 \
--msize=1024 --concurrency=300 --duration=30

Python Libraries Version
------------------------

::

$ docker run --rm -it magic/benchmark vex bench pip freeze

The benchmarks can then be ran with ``./run_benchmarks``. Use
``./run_benchmarks --help`` for various options, including selective
benchmark running.
aiofiles==0.3.1
aiohttp==2.0.7
appdirs==1.4.3
async-timeout==1.2.1
chardet==3.0.3
click==6.7
curio==0.4
Flask==0.12.2
gevent==1.2.1
greenlet==0.4.12
httptools==0.0.9
itsdangerous==0.24
Jinja2==2.9.6
MarkupSafe==1.0
multidict==2.1.5
packaging==16.8
pyparsing==2.2.0
sanic==0.5.4
six==1.10.0
tornado==4.5.1
Twisted==16.1.1
ujson==1.35
uvloop==0.8.0
websockets==3.3
Werkzeug==0.12.2
yarl==0.10.2
zope.interface==4.4.1

Binary file added report/http-report-latency.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added report/http-report-requests.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading