You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add extension to kafka-cron project - port to minikube (#206)
* Add extension to kafka-cron project - port to minikube
* small fixes
* Update projects/kafka-cron/README.md
Co-authored-by: Daniel Wagner-Hall <[email protected]>
* Update projects/kafka-cron/README.md
Co-authored-by: Daniel Wagner-Hall <[email protected]>
---------
Co-authored-by: Daniel Wagner-Hall <[email protected]>
Copy file name to clipboardExpand all lines: projects/kafka-cron/README.md
+30Lines changed: 30 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -242,6 +242,36 @@ How many containers being down can our system tolerate?
242
242
243
243
What happens to the Kafka system logs and the metrics that our binaries export? Did our alerts fire? If not, consider how they could be improved - remember, the point of them is to tell us when something's wrong!
244
244
245
+
### Porting your system from docker-compose to minikube
246
+
247
+
In this project, and in several previous projects, you have used `docker-compose` to deploy our code, alongside dependencies (such as Kafka and
248
+
Zookeeper here). `docker-compose` is an extremely convenient tool for running a multi-part software stack locally (it also works well for running
249
+
integration tests in a Continuous Integration workflow as part of your development process). However, in most deployments, we want to be able
250
+
to run our code and its dependencies across more than one host, in order to scale horizontally and to be robust to single-node failures. For this,
251
+
`kubernetes` is a better tool. `Kubernetes`, like `docker-compose`, is a platform for running containerised applications, but where `docker-compose`
252
+
is focused on running a set of related containers on a single host, `kubernetes` is optimized for running services across many hosts.
253
+
254
+
Here are some introductions to Kubernetes:
255
+
*[What is Kubernetes](https://www.digitalocean.com/community/tutorials/an-introduction-to-kubernetes)
256
+
*[Kubernetes Basics](https://kubernetes.io/docs/tutorials/kubernetes-basics/) - explore the linked pages here
257
+
258
+
You may have already used `minikube` in one of the previous projects. `Minikube` is a local version of Kubernetes, which we can use to learn
259
+
(rather than needing to incur the expense of cloud services such as [EKS](https://aws.amazon.com/eks/) for learning on.)
260
+
261
+
Get your local `minikube setup` working by following [minikube start](https://minikube.sigs.k8s.io/docs/start/).
262
+
Follow the steps to install the program and run the `hello-minikube` application.
263
+
264
+
Once you have done this, you will need to convert your `docker-compose.yml` files to `kubernetes` deployment files.
265
+
There is a tool, [kompose](https://kubernetes.io/docs/tasks/configure-pod-container/translate-compose-kubernetes/) which may assist you.
266
+
267
+
Test that your system works as it did on `docker-compose`.
268
+
269
+
Learn your way around the `kubernetes` command-line tool, `kubectl` (see the [kubectl documentation](https://kubernetes.io/docs/reference/kubectl/)).
270
+
* How can you resize your service, i.e. change the number of running instances (pods)?
271
+
* How can you remove one instance of your service (a single pod)?
272
+
* How can you see the logs for your service?
273
+
* How can you see the log of Kubernetes' management operations?
274
+
245
275
### Dealing with long-running jobs and load (challenging)
246
276
247
277
What does our system do if someone submits a very long-running job? Try testing this with the `sleep` command.
0 commit comments