Skip to content

Latest commit

 

History

History
236 lines (174 loc) · 6.27 KB

ManualSetup.md

File metadata and controls

236 lines (174 loc) · 6.27 KB

Infrastructure Setup

This document provide a quick and straightforward way to prepare dev enviroment for the workshop via docker.

Basic infrastructure services needed for demonstration purpose may include:

  • Docker
  • Docker Registry
  • Jenkins
  • Pack Broker
  • MySQL
  • MongoDB
  • Kafka

Docker

Service containerize runtime tool. Installation is easy:

$ curl -sSL https://get.docker.com/ | sh

This service should be setup first, before any other ones below.

Docker Registry

Before comamnd would setup a minimum docker container image registry, without Web UI.

$ docker run -d --name registry \
         -v /opt/registry_data:/var/lib/registry \
         --restart always -p 5000:5000 \
         registry:2

For Mac docker user:
You may have to change -v /opt/registry_data:/var/lib/registry parameter to something like -v ${HOME}/registry_data:/var/lib/registry to avoid the permission issue, or you could follow the docs to allow docker to mount in the /opt folder.

Registry will listen on 5000 port of the host.

The service run on http protocol by default, which is not trusted by docker. So an explicit --insecure-registry paramter is needed on dockerd service.

$ REGISTRY_HOST=10.71.84.160         #<= Change this IP to your own registry host IP
$ sudo sed -i "s#/usr/bin/dockerd#/usr/bin/dockerd --insecure-registry $REGISTRY_HOST:5000#" \
       /lib/systemd/system/docker.service

Also enable remote API port for all nodes that will be used to deploy petstore services.

$ sudo sed -i "s#-H fd://#-H tcp://0.0.0.0:2376 -H unix:///var/run/docker.sock#g" \
        /lib/systemd/system/docker.service

Finally, reload systemd config and restart docker daemon.

$ sudo systemctl daemon-reload
$ sudo systemctl restart docker

Jenkins

Open source delivery pipeline dashboard. You could start its container via docker command.

The default user in this image is jenkins, so you may need to grant full access to /opt/jenkins_data in order to let jenkins service initialize successfully.

$ sudo mkdir /opt/jenkins_data
$ sudo chmod 777 /opt/jenkins_data
$ docker run -dt --name jenkins \
         -v /opt/jenkins_data:/var/jenkins_home \
         -p 8000:8000 \
         -p 10022:22 \
         --privileged \
         --restart always \
         flin/jenkins-v2-basic:2.75

This command only works on Linux server. Other system user may have to use jenkins image with docker binaries installed inside, e.g. here.

Jenkins will listen on 8000 port of the host.

Pact Broker

This service would hold all pactfiles generated by consumer driven contract test.

Below installation step is original recorded here.

    1. create postgres databases service
$ docker run -d --name pactbroker-db \
         -e POSTGRES_PASSWORD=ThePostgresPassword \
         -e POSTGRES_USER=admin \
         -e PGDATA=/data/pgdata \
         -v /opt/postgresql:/data \
         postgres
    1. create pactbroker database
$ docker run -it --rm --link pactbroker-db:postgres \
         postgres sh -c 'exec psql -U admin \
         -h "$POSTGRES_PORT_5432_TCP_ADDR" \
         -p "$POSTGRES_PORT_5432_TCP_PORT"'
> CREATE USER pactbrokeruser WITH PASSWORD 'TheUserPassword';
> CREATE DATABASE pactbroker WITH OWNER pactbrokeruser;
> GRANT ALL PRIVILEGES ON DATABASE pactbroker TO pactbrokeruser;
    1. create pact broker service
$ docker run -d --name pactbroker \
         --link pactbroker-db:postgres \
         -e PACT_BROKER_DATABASE_USERNAME=pactbrokeruser \
         -e PACT_BROKER_DATABASE_PASSWORD=TheUserPassword \
         -e PACT_BROKER_DATABASE_HOST=postgres \
         -e PACT_BROKER_DATABASE_NAME=pactbroker \
         -p 2000:80 \
         dius/pact-broker

Service run on 2000 port on the host.

Mongo

NoSQL service.

    1. create mongo
$ docker run -d --name mongo \
       -v /opt/mongo:/data/db \
       -p 27017:27017 \
       mongo --auth
    1. setup user
$ docker exec -it mongo mongo admin
> db.createUser({ user: 'root', pwd: 'root', roles: [ { role: "root", db: "admin" } ] });
    1. test connection
$ docker run -it --rm --network=host mongo \
         mongo -u root -p root \
         --authenticationDatabase admin \
         localhost/demo-db
> db.getName();

Service run on 27017 port on the host, with user root password root.

MySQL

Relational database service.

$ docker run -d --name mysql \
         -v /opt/mysql:/var/lib/mysql \
         -p 3306:3306 \
         -e MYSQL_ROOT_PASSWORD=root \
         mysql

Service run on 3306 port on the host, with user root password root. Test connection:

$ docker run -it --rm --network=host mysql \
         sh -c 'exec mysql -h127.0.0.1 -P3306 -uroot -proot'

Kafka

Firstly, run a zookeeper service instance:

$ docker network create kafka-net
$ docker run -d --name zookeeper \
         --network kafka-net zookeeper:3.4

Then start up kafka service:

$ docker run -d --name kafka \
         --network kafka-net \
         -p 9092:9092 \
         --env ZOOKEEPER_IP=zookeeper \
         ches/kafka

Zookeeper run on 2181 port and kafka run on 9092 port on the host.

Create new topic:

$ docker run --rm --network kafka-net ches/kafka \
             kafka-topics.sh --create --topic test \
             --replication-factor 1 --partitions 1 \
             --zookeeper zookeeper:2181

Make some message:

$ docker run --rm -i --network kafka-net ches/kafka \
             kafka-console-producer.sh --topic test \
             --broker-list kafka:9092

Consum the message in queue:

$ docker run --rm -i --network kafka-net ches/kafka \
             kafka-console-producer.sh --topic test \
             --broker-list kafka:9092

Yahoo's kafka-manager could be useful for tracing MQ issues.

$ docker run -dt ---network kafka-net \
         -p 9000:9000 \
         -e ZK_HOSTS="zookeeper:2181" \
         -e APPLICATION_SECRET=letmein \
         sheepkiller/kafka-manager