diff --git a/bin/webserver.sh b/bin/webserver.sh new file mode 100755 index 0000000..631e771 --- /dev/null +++ b/bin/webserver.sh @@ -0,0 +1,4 @@ +#!/bin/sh + +cd doc/build/html +python3 -m http.server 5000 diff --git a/doc/source/account creation b/doc/source/account creation deleted file mode 100644 index 2a217d8..0000000 --- a/doc/source/account creation +++ /dev/null @@ -1,16 +0,0 @@ -================================ -GitHub Documentation Inboarding -================================ -How to join GitHub ---------------------- -1. Got to https://github.com/ -2. On the top right corner is a button with “sign up” -3. Enter your corporate Email and press continue -4. Create strong and memorable password -5. Enter a username e.g. joedoe -6. Would you like to receive updates? I suggest opting out with entering “n” -7. Verify your account by solving the task to prove you aren´t a bot -8. Hit a “create an account” -9. You are almost done! GitHub just sent you a launch you that has to be entered to create the account ultimately. Note: The Code has to be entered manually and can´t be entered via “copy & paste” -10. You are asked with how many people you are working with and if you are a student or a teacher. This personalization can be skipped at the bottom of the Website. I would suggest skipping this step. -11. Congratulations you just created you GitHub account diff --git a/doc/source/account_creation.rst b/doc/source/account_creation.rst new file mode 100644 index 0000000..204bd91 --- /dev/null +++ b/doc/source/account_creation.rst @@ -0,0 +1,37 @@ +================= +GitHub Onboarding +================= + +Most of our documentation is hosted on GitHub, a service based on, but +not the same as Git. To work with GitHub, an account is needed. + + +How to join GitHub +------------------ + +1. Go to https://github.com/ . + +2. On the top right corner is a button with "sign up". + +3. Enter your corporate email address and press "continue". + +4. Create a strong password. + +5. Enter a username e.g. johndoe + +6. Would you like to receive updates? I suggest opting out with + entering "n". + +7. Verify your account by solving the task to prove you aren't a bot. + +8. Click on the "create an account" button. + +9. You are almost done! GitHub just sent you a launch you that has to + be entered to create the account ultimately. Note: The code has to + be entered manually and you can't enter it via copy & paste. + +10. You are asked with how many people you are working with and if you + are a student or a teacher. This personalization can be skipped at + the bottom of the Website. I suggest skipping this step. + +11. Congratulations! You just created your GitHub account! diff --git a/doc/source/index.rst b/doc/source/index.rst index b253871..d420439 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -2,31 +2,33 @@ Documentation about documentation ================================= -This repository will contain information that anybody -maintaining Open Telekom Cloud documentation need to know. +This repository contains information that maintainers of the Open +Telekom Cloud documentation need to know. + Working in the Open =================== The Open Telekom Cloud values transparency as a main principle. -Therefore, all public facing documentation is maintained in the -open, by open-source components. All steps of the process are -publicly visible and transparent. This eases the collaboration -of our suppliers, OTC staff, and even customers or users. All -can propose changes, all can participate in the review process, -and each team can appoint arbitrary persons to approve changes, -ensuring the integrity of the overall documentation. All changes -are recorded and archived, so changes can be rolled back, and -audit trails are available even for the tiniest change. +Therefore, all public facing documentation is maintained in the open, +by open-source components. All steps of the process are publicly +visible and transparent. This eases the collaboration of our +suppliers, OTC staff, and even customers or users. All can propose +changes, all can participate in the review process, and each team can +appoint arbitrary persons to approve changes, ensuring the integrity +of the overall documentation. All changes are recorded and archived, +so changes can be rolled back, and audit trails are available even for +the tiniest change. + Responsibilities and Ownership ============================== The content of the projects is owned by the Open Telekom Cloud Squads responsible for the services respectively. Each team can appoint one -or several Squad members (or external contributors, if -suitable). Typically, one documentation lead and a deputy are -recommended, but other setups are also possible. +or several Squad members (or external contributors, if suitable). +Typically, one documentation lead and a deputy are recommended, but +other setups are also possible. All assets in the repository are public visible, and so are the PRs or the review messages themselves. That’s why sensitive data like @@ -48,6 +50,7 @@ was initiated by the Ecosystem Squad. It was presented to the Product Owner Community and during the XXXXXXXXXXX meeting/call/board. It was approved on November XX, 2021 as authoritative by the board. + Target Audience and Prerequisites ================================= @@ -75,6 +78,7 @@ Content change_proposal_process reviewer approver + account_creation git_account code_editors rst diff --git a/doc/source/presentations/ansible/ansible-play.png b/doc/source/presentations/ansible/ansible-play.png new file mode 100644 index 0000000..da6f064 Binary files /dev/null and b/doc/source/presentations/ansible/ansible-play.png differ diff --git a/doc/source/presentations/ansible/ansible-play.yaml b/doc/source/presentations/ansible/ansible-play.yaml new file mode 100644 index 0000000..b1a6ac3 --- /dev/null +++ b/doc/source/presentations/ansible/ansible-play.yaml @@ -0,0 +1,21 @@ +--- +- name: Update web servers + hosts: webservers + remote_user: root + + tasks: + - name: Ensure apache is at the latest version + ansible.builtin.yum: + name: httpd + state: latest + - name: Write the apache config file + ansible.builtin.template: + src: /srv/httpd.j2 + dest: /etc/httpd.conf + + + + + + + diff --git a/doc/source/presentations/ansible/ansible_collections.rst b/doc/source/presentations/ansible/ansible_collections.rst new file mode 100644 index 0000000..22b9b22 --- /dev/null +++ b/doc/source/presentations/ansible/ansible_collections.rst @@ -0,0 +1,628 @@ +:orphan: + +=============================== +Cloud Provisioning with Ansible +=============================== + +.. revealjs-slide:: + +Is that even possible? + + +**Nils Magnus** + +Senior Cloud Architect Open Telekom Cloud + +stackconf 2022, Berlin, Germany + +July 19-20, 2022 + +.. figure:: ./stackconf.png + :align: right + + +Nils Magnus +=========== + +.. list-table:: + :widths: 30 70 + :header-rows: 0 + + * - .. figure:: ./nils-magnus.png + + - * Ecosystem Squad of Open Telekom Cloud + * Tools for our users and Community work + * 20 years experience in security, operations, cloud + * tech writer, editor, and journalist + * Director for German Unix User Group + +*Automation should make work easier, not take it away. --- Fred Ammon* + +Disclaimer: I'm an Ansible user, not an expert. Plase correct me, if necessary. + + +Agenda +------ + +* What is Provisioning? +* 20 quick, but important Cloud and Automation Terms +* Success Story of Ansible +* Architecture Overview +* Installation and technical Prerequisites +* Examples and Demo +* Ansible Collections for Developers + + +Provisioning +============ + +modern software setups come in (at least two) layers: + +.. figure:: ./conf-vs-provis.png + :align: right + +* **provisioning** means to build and configure **infrastructure** +* **configuring** means to build and adapt **applications** + + +Why automate building infrastructure? +------------------------------------- + +disaster recovery: + fast and automated rebuild of setups, short downtime +updating and code hygiene: + fighting configuration drift: instead of patching, better rebuild +infrastructure testing, compliance of APIs and interfaces: + setup consists of many components whose interaction is complex +knowledge sharing: + passing on codified best practices + + +Forms of Provisioning +--------------------- + +check these methods of provisioning: + +* manually +* run-books stored in wikis or on paper +* bash scripts +* installer (how do they actually work internally?) +* **domain specific languages ​​that support provisioning** + +how applicable are *manual error prevention*, *desired state specification*, or *reproducibility* to those methods? + + +Infrastructure as Code +---------------------- + +should be + +* **declarative** (*what* instead of *how*) +* **convergent** (to approach the target) +* **idempotent** (repetition doesn't hurt), and +* assess and manage **status** + + +Typical Representatives +----------------------- + +IaC comes in many disguises: + +* VSphere (haha) +* **Terraform** +* Pulumi +* Heat (or its proprietary cousins like Cloud Formation, ...) + +But does Ansible also belong to this list? + + +Important Terms and Prerequisites +================================= + +Ansible: + configuration management framework, that leverages SSH and + Python to configure a target from a controller. + +controller a. k. a. controller node: + The system you sit in front of. Has typically checked out a Git + repo. + +targets a. k. a managed nodes: + The systems you manage as an administrator, devops, or system + engineer. + + +Ansible Terms and Concepts +-------------------------- + +module: + takes care that a resource has a specific state, for example make + sure that user "magnus" exists or RPM package "emacs" is installed. + +task: + description of the circumstances a module operates on. + +role: + a template on how to combine several tasks into a more holistic + activity like installing a webserver (installing the package, + configuring some files, starting the service) play collection. + + +Cloud Terms and Concepts +------------------------ + +cloud: + service that enables you to manage various resources such as + servers, storage, and networks via an API. + +OpenStack: + Open Source cloud framework, developed under the four Opens: Open + Source, Open Design, Open Development, Open Community. + +Open Telekom Cloud: + Public cloud operated for Deutsche Telekom by + T-Systems International GmbH in Europe by Europeans, based on + OpenStack. Complies to several certifications and is GDPR compliant, + which is debatable for other Hyperscaler clouds. + + +Cloud Access Terms and Conepts +------------------------------ + +SDK: + a Python library that accesses the OpenStack API and performs some + housekeeping tasks. + +Client: + a CLI tool to query and manipulate cloud resources on the command + line via the SDK and API. Used for manual tasks. + +bastion: + a multipurpose server in a cloud domain, exposed to the + Internet. Can (after proper authorization) access servers and other + ressoures hidden from public users views. This is our today's + project goal to install. + + +Python Terms and Concepts +------------------------- + +Python: + Programming language Ansible and OpenStack SDK are mostly written + in. We only cover Python 3 here, sometimes at least version 3.8 is + required. + +pip: + Python Package manager. We install Ansible, the SDK and Client with + it. + +virtual environment: + sandbox that makes sure that installed Python packages don't mess + with your Linux distribution. + + +OpenStack Terms and Concepts +---------------------------- + +ECS: + Elastic cloud server or just a VM. + +image: + virtual installation medium containing a linux distribution like + Ubuntu 22.04 or CentOS Stream. + + +OpenStack Terms and Concepts (II) +--------------------------------- + +flavor: + abstraction for the combination of CPUs and memory applied to + a VM. The flavor "s3.medium.1" describes a server with one core and + 1 GByte RAM, for example. + +volumes: + hard drives or block devices in cloud speech. + +network: + IP addresses, networks, subnets, routers, security groups, and some + more resources work similar like their physical conterparts, but can + be configured via API and SDK. + + +Ansible Architecture for Classic and Cloud Setups +================================================= + +.. list-table:: + :widths: 70 30 + :align: left + + * - well known as a configuration management tool to: + + * create users, groups, ... + * install packages + * edit configuration files + * start services + + - is (or should be): + + * declarative + * convergent + * idempotent + * . + +But what changes if it is used as a provisioning tool? + + +Classic Ansible Operation Architecture +-------------------------------------- + +.. list-table:: + :widths: 50 50 + :header-rows: 0 + + * - * controller has playbook (from a Git repo) and runs it (optimally through a CI/CD server). + * Ansible connects (almost always with SSH) to the target. + * there it builds an ad-hoc runtime in Python, executes everything and deletes the tools again. + * reports the success of its work back to the controller + - .. figure:: ./arch-cfgmgmt.png + + +Key Terms of Ansible in Two Minutes +----------------------------------- + +.. list-table:: + :widths: 50 50 + :header-rows: 0 + + * - * **playbooks** (in YAML) + * **tasks** describe the desired state + * **modules** for everything: users, packages, config entries in files, etc. + * modules are grouped, structured and reused: **plays** and **roles** + * the top-level entities are in a static or dynamic **inventory** + * multiple (and sometimes confusing) options for parameterization + - .. code-block:: yaml + + --- + - name: Update web servers + hosts: webservers + remote_user: root + + tasks: + - name: Ensure Apache is at the latest version + ansible.builtin.yum: + name: httpd + state: latest + - name: Write the Apache config file + ansible.builtin.template: + src: /srv/httpd.j2 + dest: /etc/httpd.conf + + +One Major Difference for Provisioning +------------------------------------- + +.. list-table:: + :widths: 30 70 + :header-rows: 0 + + * - * Initially there are no targets to address + * Cloud-API is accessed via SDK installed on controller + - .. figure:: ./arch-cloud.png + :width: 80% + + +Marry OpenStack with Ansible +---------------------------- + +Ansible modules manage OpenStack and Open Telekom Cloud ressources + +modules do not access the APIs directly + +OpenStack SDK acts as a proxy between Ansible and cloud + +modules are packaged as Ansible collections + +collections are installed with ``ansible-galaxy`` + +My team develops and maintains the modules ``openstack.cloud`` and ``opentelekomcloud.cloud``. + + +Installation +============ + +Only the controller needs software: + +#. prepare virtual environment + + .. code-block:: console + + $ python3 -mvenev ansidemo + $ . ansidemo/bin/activate + $ pip install --update pip + +#. install Ansible (core version 2.13.1 as of July 2022) + + recommended to install via ``pip`` in a virtual environment, + alternatively via package manager of your Linux distro + + Ubuntu 22.04: 2.12.0, CentOS Stream 9: 2.9.27 + + .. code-block:: console + + $ pip install ansible + $ ansible --version | head -1 + ansible [core 2.13.1] + + +Installation (II) +----------------- + +3. install OpenStack SDK (currently, version 0.61.0 is required) + + .. code-block:: console + + $ pip install openstacksdk==0.61.0 + +#. install OpenStack collection for Ansible: + + .. code-block:: console + + $ ansible-galaxy install openstack.cloud + +#. optionally install OTC Extensions to the SDK and collection: + + .. code-block:: console + + $ pip install openstacksdk==0.61.0 otcextensions + $ ansible-galaxy install openstack.cloud opentelekomcloud.cloud + + +Cloud Authentication +-------------------- + +several options where to place cloud credentials: command line, envvars, files + +files in ``$HOME/.config/openstack``, ``/etc/openstack`` or ``.`` are recommended: + + .. code-block:: console + + $ git clone https://github.com/Nils-Magnus/ac-demo.git + $ cd ac-demo + $ cat clouds.yaml + clouds: + stackconf-demo: + profile: otc + auth: + username: nilsmagnus + password: ******** + project_name: eu-de_stackconf + user_domain_name: OTC00000000001000000447 + +you may externalize the password in the file ``secure.yaml`` + +in the same directory and with the same file structure + + +Optional: Installing CLI and Verify Setup +----------------------------------------- + +the commandline tool ``openstack`` is also available in PyPI + +also consumes the ``openstack.yaml`` files + +.. code-block:: console + + $ pip install -U openstackclient + $ openstack flavor list | head + +the CLI is just a frontend for the SDK + +automatically discovers libraries and extensions + +``openstack --help`` is extensive + +``openstack list *resource*`` or ``openstack show *resource*`` are helpful + + +Provisioning a Bastion Host +=========================== + +Repo at https://github.com/Nils-Magnus/ac-demo.git comes with an example: + +.. code-block:: yaml + + $ cat allinone.yml + --- + - name: My complete Bastion Playbook + hosts: "localhost" + vars: + router_name: "my-router" + network_name: "my-network" + subnet_name: "my-subnet" + subnet_cidr: "192.168.1.0/24" + subnet_dns_servers: "['100.125.4.25', '8.8.4.4']" + enable_snat: false + server_name: "my-server" + region: "eu-de" + az: "eu-de-02" + server_sg: "sg-vc-web" + image: "Standard_Ubuntu_22.04_latest" + pubkey_name: "nils-magnus-otc" + pubkey: "ssh-rsa AAAAB3Nz[...]1u8Gw== Nils Magnus (nils.magnus@t-systems.com)" + flavor: "s3.medium.1" + cloud: "stackconf" # for credentials + + +Bastion Host (II): VPC and actually creating Server +--------------------------------------------------- + +.. code-block:: yaml + + roles: + - { role: opentelekomcloud.vpc, state: 'present' } + tasks: + - name: 1 | Upload SSH public key + openstack.cloud.keypair: + name: "{{ pubkey_name }}" + public_key: "{{ pubkey }}" + - name: 2 | Create bastion server + openstack.cloud.server: + name: "{{ server_name }}" + region_name: "{{ region }}" + network: "{{ network_name }}" + wait: "yes" + security_groups: "{{ server_sg }}" + availability_zone: "{{ az }}" + image: "{{ image }}" + key_name: "{{ pubkey_name }}" + cloud: "{{ cloud }}" + volume_size: "{{ systemdisk_size }}" + flavor: "{{ flavor }}" + register: returned_server + + +Bastion Host (III): Display and Make Use of new IP +-------------------------------------------------- + +.. code-block:: yaml + + - name: 3 | Display my new IP address + debug: + msg: "The IP is {{ returned_server.server.accessIPv4 }}" + - name: 4 | Delete the now outdated hostkey on local controller + remote_user: magnus + delegate_to: localhost + command: "ssh-keygen -f /home/magnus/.ssh/known_hosts -R {{ returned_server.server.accessIPv4 }}" + - name: 5 | Wait for SSH port become ready + delegate_to: "{{ returned_server.server.accessIPv4 }}" + remote_user: ubuntu + wait_for: + port: 22 + + +Bastion Host (IV): Update default Packages +------------------------------------------ + +.. code-block:: yaml + + - name: 6 | Update all Packages + remote_user: ubuntu + become: yes + delegate_to: "{{ returned_server.server.accessIPv4 }}" + apt: + upgrade: dist + update_cache: yes + - name: 7 | Install additional packages + remote_user: ubuntu + become: yes + delegate_to: "{{ returned_server.server.accessIPv4 }}" + apt: + name: + - python3-venv + - emacs + + +Bastion Host (V): Housekeeping and Clean-Up +------------------------------------------- + +.. code-block:: yaml + + - name: 8 | Delete nasty files + remote_user: ubuntu + become: yes + delegate_to: "{{ returned_server.server.accessIPv4 }}" + file: + path: "{{ item }}" + state: absent + with_items: + - /etc/update-motd.d/10-help-text + - /etc/update-motd.d/50-motd-news + - /etc/update-motd.d/99-otc-motd + - /home/magnus/.bash_logout + - /home/magnus/.ostackrc + - /home/magnus/.s3rc + + +Bastion Host (VI): Manual User Management +----------------------------------------- + +.. code-block:: yaml + + - name: 9 | Create groups + remote_user: ubuntu + become: yes + delegate_to: "{{ returned_server.server.accessIPv4 }}" + group: + name: docker + - name: 10 | Create user + remote_user: ubuntu + become: yes + delegate_to: "{{ returned_server.server.accessIPv4 }}" + user: + name: magnus + comment: Nils Magnus + password: $6$Ge3PyABP5Isk9NaK$2CpmDCb2dxxxfgVfs5ZwTHxUibL55AQsPRgzEcuvSu2tGzJfy2wmA9GG05d7VLCa7z4w1.Bh/LgA9rw3KPiQr1 + shell: /bin/bash + groups: admin, docker + append: yes + - name: 11 | Install my GitHub SSH key in authorized_keys + remote_user: ubuntu + become: yes + delegate_to: "{{ returned_server.server.accessIPv4 }}" + authorized_key: + user: magnus + key: https://github.com/Nils-Magnus.keys + +Run Everything +-------------- + +.. code-block:: console + + $ ansible-playbook allinone.yml + + +Future Project: Dynamic Inventory +--------------------------------- + +This example just creates and manages a single Bastion host + +For more extensive setups you should externalize your managed hosts + +Mapping tags (or other properties) to Ansible plays + +Inspiration: ``openstack_inventory.py`` + + +Development: Extending, Testing, Documentation +============================================== + +Reporting issues, contributions and PRs are always welcome: + +- generic OpenStack collection: + + https://github.com/openstack/ansible-collections-openstack + +- extra modules for Open Telekom Cloud extensions: + + https://github.com/opentelekomcloud/ansible-collection-cloud + +- Ansible upstream maintainers currently do some overhaul + + https://hackmd.io/szgyWa5qSUOWw3JJBXLmOQ + + requires temporarily pinning of version 0.61.0 of SDK + + +Thank you: Q&A?! +================ + + **Get in touch with us!** + + Nils Magnus, Senior Cloud Architect Open Telekom Cloud + + nils.magnus@t-systems.com + + - https://open-telekom-cloud.com/ + - https://community.open-telekom-cloud.com/ + - https://doc.otc-service.com/ + diff --git a/doc/source/presentations/ansible/arch-cfgmgmt.png b/doc/source/presentations/ansible/arch-cfgmgmt.png new file mode 100644 index 0000000..d0a55cc Binary files /dev/null and b/doc/source/presentations/ansible/arch-cfgmgmt.png differ diff --git a/doc/source/presentations/ansible/arch-cfgmgmt.svg b/doc/source/presentations/ansible/arch-cfgmgmt.svg new file mode 100644 index 0000000..71130dd --- /dev/null +++ b/doc/source/presentations/ansible/arch-cfgmgmt.svg @@ -0,0 +1,381 @@ + + + +PythonplaybookSSHreportController diff --git a/doc/source/presentations/ansible/arch-cloud.png b/doc/source/presentations/ansible/arch-cloud.png new file mode 100644 index 0000000..d47a819 Binary files /dev/null and b/doc/source/presentations/ansible/arch-cloud.png differ diff --git a/doc/source/presentations/ansible/arch-cloud.svg b/doc/source/presentations/ansible/arch-cloud.svg new file mode 100644 index 0000000..9638980 --- /dev/null +++ b/doc/source/presentations/ansible/arch-cloud.svg @@ -0,0 +1,1218 @@ + + + +ResourcesSDKplaybookRESTmanage (CRUD)ControllerCloud-API diff --git a/doc/source/presentations/ansible/conf-vs-provis.png b/doc/source/presentations/ansible/conf-vs-provis.png new file mode 100644 index 0000000..4d79a16 Binary files /dev/null and b/doc/source/presentations/ansible/conf-vs-provis.png differ diff --git a/doc/source/presentations/ansible/nils-magnus.png b/doc/source/presentations/ansible/nils-magnus.png new file mode 100644 index 0000000..c1af742 Binary files /dev/null and b/doc/source/presentations/ansible/nils-magnus.png differ diff --git a/doc/source/presentations/ansible/stackconf.png b/doc/source/presentations/ansible/stackconf.png new file mode 100644 index 0000000..4bf3a35 Binary files /dev/null and b/doc/source/presentations/ansible/stackconf.png differ diff --git a/doc/source/presentations/index.rst b/doc/source/presentations/index.rst index 3b891f2..d4f469d 100644 --- a/doc/source/presentations/index.rst +++ b/doc/source/presentations/index.rst @@ -1,9 +1,11 @@ Presentations ============= - .. toctree:: :maxdepth: 1 - introduction - vault_openstack + intro/introduction + vault/vault_openstack + ansible/ansible_collections + sandbox/sandbox + usercreate/createusers diff --git a/doc/source/presentations/introduction.rst b/doc/source/presentations/intro/introduction.rst similarity index 99% rename from doc/source/presentations/introduction.rst rename to doc/source/presentations/intro/introduction.rst index 70abe3d..70d3a50 100644 --- a/doc/source/presentations/introduction.rst +++ b/doc/source/presentations/intro/introduction.rst @@ -78,7 +78,7 @@ Solution Options How === -.. image:: ../_static/images/workflow.png +.. image:: ../../_static/images/workflow.png :width: 70% Key Features diff --git a/doc/source/presentations/sandbox/sandbox.rst b/doc/source/presentations/sandbox/sandbox.rst new file mode 100644 index 0000000..614270f --- /dev/null +++ b/doc/source/presentations/sandbox/sandbox.rst @@ -0,0 +1,44 @@ +:orphan: + +======= +Sandbox +======= + +RST + Sphinx + HTML + CSS + Javascript + RevealJS are not for the +faint hearted. This presentation tries to figure out a few secrets. + + +Use Ansible in two minutes +-------------------------- + +.. list-table:: + :widths: 60 40 + :header-rows: 0 + + * - * **playbooks** (in YAML) + * **tasks** describe the desired state + * **modules** for everything: users, packages, config entries in files, etc. + * modules are grouped, structured and reused: **plays** and **roles** + * the top-level entities are in a static or dynamic **inventory** + * multiple (and sometimes confusing) options for parameterization + - .. code-block:: yaml + + --- + - name: Update web servers + hosts: webservers + remote_user: root + + tasks: + - name: Ensure apache is at the latest version + ansible.builtin.yum: + name: httpd + state: latest + - name: Write the apache config file + ansible.builtin.template: + src: /srv/httpd.j2 + dest: /etc/httpd.conf + +More +---- + +haha diff --git a/doc/source/presentations/usercreate/createusers.rst b/doc/source/presentations/usercreate/createusers.rst new file mode 100644 index 0000000..155bca1 --- /dev/null +++ b/doc/source/presentations/usercreate/createusers.rst @@ -0,0 +1,317 @@ +:orphan: + +================================================ +Creating Users in a Batch with the OpenStack SDK +================================================ + +.. revealjs-slide:: + +.. list-table:: + :widths: 70 30 + :header-rows: 0 + + * - * **Nils Magnus** + * Senior Cloud Architect Open Telekom Cloud + * OpenInfra Meetup, Budapest, Hungary + * live from Berlin, Germany + * August 4, 2022 + + - .. figure:: ./openinframeetup.png + + +Nils Magnus +=========== + +.. list-table:: + :widths: 30 70 + :header-rows: 0 + + * - .. figure:: ./nils-magnus.png + + - * Ecosystem Squad of Open Telekom Cloud + * Tools for our users and Community work + * 20 years experience in security, operations, cloud + * tech writer, editor, and journalist + * Director for German Unix User Group + + +*Automation should make work easier, not take it away. --- Fred Ammon* + + +Prerequisites: Python +--------------------- + +Python 3 installed. + +Any version up from Python 3.6 should do. + +Verify: + +.. code-block:: console + + $ python --version + +Install: + +.. code-block:: console + + $ apt install python3 + $ dnf install python3 + + +Install OpenStack SDK +--------------------- + +recent version of the OpenStack SDK: 0.100.1 + +Verify: + +.. code-block:: console + + $ pip show openstacksdk + +Install: + +.. code-block:: console + + $ pip install openstacksdk + +Virtual environment or a user-based installation is recommended: + +.. code-block:: console + + $ python3 - m venv demo + $ ./demo/bin/activate + (demo) + + +OTC Extensions +-------------- + +Optionally: Open Telekom Cloud specific functions + +.. code-block:: console + + $ pip install otcextensions + +pulls the SDK dependency automatically + +not necessary for this very project, though. + + +Credentials +----------- + +* commandline ``--os-password``: insecure + +* environment variables ``export OS_PASSWORD``: somewhat insecure + +* credentials stored in a file: recommended + +Search path by descending precedence: + +* ./cloud.yaml +* $HOME/.conf/openstack/clouds.yaml +* /etc/oenstack/clouds.yaml + +Applies also to ``secure.yaml`` in the same directories. + + +Tokens +------ + +normal cloud resources require a regular token + +user management requires a so-called **domain scoped token** + +.. code-block:: yaml + + clouds: + otc: + profile: otc + auth: + username: 'johndoe' + password: 'supersecret' + project_name: 'eu-de' + user_domain_name: 'OTC00000000001000000***' + + otcadmin: + auth: + username: 'johndoe' + password: 'supersecret' + auth_url: 'https://iam.eu-de.otc.t-systems.com:443/v3' + user_domain_name: 'OTC00000000001000000***' + domain_name: 'OTC00000000001000000***' + interface: 'public' + identity_api_version: 3 + + +Quotas +------ + +Quota limit the number of users you can create + +default for maximum users: 10 + +changed by a call or email to the Cloud Handling Support: + +* domain ID (OTC00000000001000000*** or similar), +* the service (“Identity and Access Management”), +* the resource (user), and +* the new value of the quota. + +The team is really quick, heads up to you guys! + +Role Based Access Control +------------------------- + +* tenant != domain ~ company, organization +* project ~ division, department + +* policies: structure of capabilities +* groups: list of users a policy applies to + +predefined groups in Open Telekom Cloud + +* admin +* poweruser + + +Data to Create Users From +------------------------- + +need four column heads in the CSV file: + +.. figure:: ./table.png + :scale: 80% + +columns can be in any order + + +Code Walkthrough +================ + +.. code-block:: python + + import openstack + import csv + + co = openstack.connect("otcadmin") + idm = co.identity + + with open("workshop-users.csv", mode="r") as infile: + for user in csv.DictReader(infile, delimiter=";"): + print("User %s " % user["realname"], end="") + userobj = idm.find_user(user["username"]) + if userobj: + print("already exits.") + else: + userobj = idm.create_user(name=user["username"], + email=user["email"], + description=user["realname"]) + result = idm.post( + "https://iam.eu-de.otc.t-systems.com/v3.0/OS-USER/users/{user_id}/welcome" + .format(user_id=userobj.id)) + print(result.reason) + print("Assigning user to group ", user["group"]) + groupobj = idm.find_group(user["group"]) + + co.add_user_to_group(userobj, groupobj) + +Connecting to the Cloud +----------------------- + +We first import openstack to be able to connect to the Open Telekom +Cloud and csv to read the spreadsheet data. + +In the actual code, we connect to the cloud with the otcadmin instance +of the clouds.yaml file I mentioned above. The result is the +connection object co. It contains many services, but today, we just +need the IAM service, which I store in idm. Please keep in mind that +my config file provided me with a connection object based on a domain +scoped token, not a normal one. + +Reading the Spreadsheets +------------------------ + +Next lines open the CSV file for reading and process its content line +by line, storing each row in the dictionary user. The idm service now +fetches the actual user object from the API and stores it in +userobj. If the cloud did not return an object, it has to be created, +passing the three attributes name, email, and description. + +How to Set the Password +----------------------- + +While the user resource now exists, the person behind it has to find +out about their credentials. Since dealing with clear-text passwords +is considered an antipattern, I send a welcome message to the +associated email address containing a brief welcome notification and a +link to log in. After that, the user can pick a password of her or his +own choice. + +Dealing with Special API Calls +------------------------------ + +The API call for the welcome message is currently not directly covered +by the SDK, but the helper method `post()` of the IAM object can be +used to call the API directly. This should be done only as a last +resort since it circumvents several other useful mechanisms like +service catalog discovery and input validation. However, in this case, +we can get over it, I think. We just have to fill in the numeric user +ID in the REST call. We can verify if that worked by the value of +`result.reason`. + +Assigning a Group +----------------- + +Just creating users is not enough, though. It results in users with no +capabilities at all. That’s why I look up a group object from the IAM +and then associate it with the user object. + +Wrap-Up +======= + +* Installing Python, SDK (and OTC Extensions). +* Have credentials ready in `clouds.yaml`, Check quotas. +* Identify (or create) suitable groups. +* Arrange data in a spreadsheet. + +* Import SDK and CSV handling. Connect to a cloud stanza that provides a domain scoped token. +* For each entry in the table: + + * create user if necessary, + * invite user by email + * assign proper group to user + +Ressources +---------- + +IAM Documentation of the SDK: + +https://docs.openstack.org/openstacksdk/latest/user/proxies/identity_v3.html + +Blog artice in the Open telekom Cloud Community Portal: + +https://community.open-telekom-cloud.com/communitysys_id=67d644cbb7985958d15aa7b16b8c02f0 + +Helpcenter: + +https://docs.otc-service.com/ + +Open Telekom Cloud: + +https://open-telekom-cloud.com/ + +Thanks for attending! +--------------------- + +Questions? Answers! + +**Nils Magnus** + +Senior Cloud Architect Open Telekom Cloud + +nils.magnus@t-systems.com + + diff --git a/doc/source/presentations/usercreate/nils-magnus.png b/doc/source/presentations/usercreate/nils-magnus.png new file mode 100644 index 0000000..c1af742 Binary files /dev/null and b/doc/source/presentations/usercreate/nils-magnus.png differ diff --git a/doc/source/presentations/usercreate/openinframeetup.png b/doc/source/presentations/usercreate/openinframeetup.png new file mode 100644 index 0000000..87c061f Binary files /dev/null and b/doc/source/presentations/usercreate/openinframeetup.png differ diff --git a/doc/source/presentations/usercreate/table.png b/doc/source/presentations/usercreate/table.png new file mode 100644 index 0000000..af23fb9 Binary files /dev/null and b/doc/source/presentations/usercreate/table.png differ diff --git a/doc/source/presentations/vault_openstack.rst b/doc/source/presentations/vault/vault_openstack.rst similarity index 100% rename from doc/source/presentations/vault_openstack.rst rename to doc/source/presentations/vault/vault_openstack.rst diff --git a/doc/source/presentations/vault_openstack.svg b/doc/source/presentations/vault/vault_openstack.svg similarity index 100% rename from doc/source/presentations/vault_openstack.svg rename to doc/source/presentations/vault/vault_openstack.svg