Skip to content

Setup Chef Server

Jump to bottom
Ashish Jain edited this page Mar 30, 2022 · 4 revisions

Following are the steps required to setup chef infra server on Ubuntu and configure it to work with MobiledgeX services. Following steps are tested with Ubuntu 18.04 and Chef server version 13.2.0.

Download & install Chef Infra Server package

wget https://packages.chef.io/files/stable/chef-server/13.2.0/ubuntu/18.04/chef-server-core_13.2.0-1_amd64.deb
dpkg -i chef-server-core_13.2.0-1_amd64.deb
chef-server-ctl reconfigure

Verify if installation is successful

curl -D - http://localhost:8000/_status

Create Chef admin user

Execute chef-server-ctl commands on chef server. chefadmin.pem is the generated private key. Store this in a secure place as it'll be required by the chef client & knife CLI to communicate with chef server

chef-server-ctl user-create chefadmin <full-name> <email-address> <password> --filename ./chefadmin.pem

Create new organization

mobiledgex-validator.pem is the generated file, which is of no use to us so it can be ignored

chef-server-ctl org-create mobiledgex "MobiledgeX Inc." --association_user chefadmin --filename mobiledgex-validator.pem

Store chefadmin key to vault

  • Store chefadmin key to vault path /secret/data/accounts/chef with following JSON object as input:
{
    "apikey": "<contents-of-chefadmin.pem-file>",
}
  • Ensure the controller service has read permission for above vault path

Setup new group/user for different setup (optional)

  • It is better to create a new group/user for every setup with minimum permissions, this way even if the private key is compromised, the attack wil be very limited
  • For example, let's create a new group called dev for development setup with dev user
knife group create dev
  • Create permissions for group to read,update its policy group:
knife acl add group dev policy_groups dev read,update
  • Add permissions for group specific containers
knife acl bulk add group dev clients 'dev.*' create,read,update,delete
knife acl bulk add group dev nodes 'dev.*' create,read,update,delete
  • Setup user (say devadmin) for dev group (this should be performed on chef server)
chef-server-ctl user-create -p devadmin Dev Admin <email-address>
chef-server-ctl org-user-add mobiledgex devadmin
  • Add user to group
knife group add user devadmin dev

Setup Knife CLI

  • Because we use Knife CLI to interact with Chef-server, following are steps on how to install it and set it up to talk to Chef-server. This can be on a separate host
  • Follow this guide to install Chef-Workstation on your system. Chef-Workstation includes all the binaries we require: https://docs.chef.io/workstation/install_workstation/
  • Once above is installed, verify knife is installed and is of appropriate version (better to have of version 15 or above):
❯ knife -v
Chef Infra Client: 15.10.12
  • Also verify chef binary is installed:
❯ chef -v
Chef Workstation version: 0.18.3
Chef Infra Client version: 15.10.12
Chef InSpec version: 4.18.111
Chef CLI version: 2.0.10
Test Kitchen version: 2.5.0
Cookstyle version: 6.3.4
  • For Knife to talk to Chef-server it needs config file to be setup up. Create a file in this path ~/.chef/knife.rb with following contents. chefadmin.pem file comes from steps mentnioned above. This also requires the clone of edge-cloud-infra repository, please change the cookbook_path according to the repository location. And also use hostname assigned to the chef server for chef_server_url 
current_dir = File.dirname(__FILE__)
home_dir = Dir.home

log_level                 :info
log_location              STDOUT
node_name                 "chefadmin"
client_key                "#{current_dir}/chefadmin.pem"
chef_server_url           "https://chef.mobiledgex.net/organizations/mobiledgex"
cookbook_path             [ "#{home_dir}/go/src/github.com/mobiledgex/edge-cloud-infra/chef/cookbooks/" ]
  • Make sure chefadmin.pem is present in the current working direction (.i.e. at ~/.chef/). Also, ensure that cookbook path is to chef/cookbooks directory under edge-cloud-infra repo
  • Once above is setup you should be able to use Knife commands to get info from Chef-server. For example:
❯ knife node list
dev-EU-chef-berlin-1-TDG-pf
dev-EU-chef-berlin-2-TDG-pf

Setup ACLs

  • Setup ACLs so the non-admin user should have very limited permissions
  • Remove group users (create,update,delete,grant) permissions from all containers
knife acl remove group users containers nodes create,update,delete,grant
knife acl remove group users containers clients create,update,delete,grant
knife acl remove group users containers policies create,update,delete,grant
knife acl remove group users containers policy_groups create,update,delete,grant
knife acl remove group users containers cookbooks create,update,delete,grant
knife acl remove group users containers data create,update,delete,grant
knife acl remove group users containers environments create,update,delete,grant
knife acl remove group users containers groups create,update,delete,grant
knife acl remove group users containers roles create,update,delete,grant
knife acl remove group users containers sandboxes create,update,delete,grant
knife acl remove group users containers cookbook_artifacts create,update,delete,grant
  • Remove group users permission from existing objects
knife acl bulk remove group users clients '.*' create,update,delete,grant
knife acl bulk remove group users nodes '.*' create,update,delete,grant
knife acl bulk remove group users policy_groups '.*' create,update,delete,grant
knife acl bulk remove group users cookbooks '.*' create,update,delete,grant
  • For databags, remove read permissions as well, so that non-admin user cannot access it
knife acl bulk remove group users data '.*' create,update,delete,grant,read
  • Allow non-admin users to read,update policies. This is required for cloudlet upgrade
knife acl add group users containers policies read,update
knife acl bulk add group users policies '.*' read,update

knife acl add group users containers sandboxes read,update,create
knife acl add group users containers cookbook_artifacts read,update,create

Setup data bags required for MobiledgeX services

  • We use data bags to store docker registry credentials and maintain mapping between container versions and their respective release tags. Add release tags for all the supported versions
❯ knife data bag create mexsecrets

❯ cat reg_pass.json
{
  "id": "docker_registry",
  "mex_docker_username": "*******",
  "mex_docker_password": "*******"
}

❯ knife data bag from file mexsecrets reg_pass.json

❯ knife data bag create mex_releases
Created data_bag[mex_releases]

❯ cat release_map.json
{
  "id": "2021-08-06",
  "release": "R3.0 RC2"
}

❯ knife data bag from file mex_releases release_map.json
Updated data_bag_item[mex_releases::2021-08-06]

Upload dependent 3rd party cookbooks

  • There are few 3rd party cookbooks which are required for MobiledgeX cookbooks and hence they should be downloaded and uploaded it to chef server before we upload MobiledgeX cookbooks
  • Get the following cookbook versions from chef policyfiles in edge-cloud-infra repository (chef/policyfies)
knife supermarket download docker 7.7.0 -f /tmp/chef_docker_7_7_0.tar.gz
tar -xzf /tmp/chef_docker_7_7_0.tar.gz -C /tmp/
knife cookbook upload docker --cookbook-path /tmp/
rm -r /tmp/chef_docker_7_7_0.tar.gz

knife supermarket download chef_client_updater 3.11.0 -f /tmp/chef_chef_client_updater_3_11_0.tar.gz
tar -xzf /tmp/chef_chef_client_updater_3_11_0.tar.gz -C /tmp/
knife cookbook upload chef_client_updater --cookbook-path /tmp/
rm -r /tmp/chef_chef_client_updater_3_11_0.tar.gz

knife supermarket download iptables 8.0.0 -f /tmp/iptables_8_0_0.tar.gz
tar -xzf /tmp/iptables_8_0_0.tar.gz -C /tmp/
knife cookbook upload iptables --cookbook-path /tmp/
rm -r /tmp/iptables_8_0_0.tar.gz

Upload all the cookbooks

  • Following will upload all the cookbooks from edge-cloud-infra repo:
knife cookbook upload -a
  • Verify that the cookbook of required version is uploaded to chef-server:
knife cookbook show <cookbook-name>

Push all the policygroups from MobiledgeX edge-cloud-infra repository to appropriate setup

  • Setup can be main/qa/stage/upg etc
cd ~/go/src/github.com/mobiledgex/edge-cloud-infra/chef/policyfiles
chef install docker_crm.rb
chef install base.rb
chef install k8s_crm.rb
chef install k8s_worker_crm.rb
chef push <setup> docker_crm.rb
chef push <setup> base.rb
chef push <setup> k8s_crm.rb
chef push <setup> k8s_worker_crm.rb

Setup appropriate args for controller service

  • Pass chef server path to controller as arg chefServerPath. It defaults to https://chef.mobiledgex.net/organizations/mobiledgex

Clone this wiki locally