Cephalocon 2025 Demo Materials

[phlogistonjohn]

Demo 1

Demo 1 is a brief demonstration of a minimal set of commands needed to bring up an SMB cluster on Ceph. Said cluster uses CTDB and is joined to AD.

Recording

YouTube

Prereqs

• 3 Node Ceph cluster (VMs)
• 1 AD Domain Controller (Samba based)
• 1 Windows Server 2025 (acting as a client)
• CephFS is enabled and subvolumes have already been created

Commands

• ceph orch ls - Shows no existing SMB services
• ceph smb cluster create demo1 active-directory DOMAIN1.SINK.TEST --domain-join-user-pass=Administrator%Passw0rd --placement=3 - create smb cluster resource
• ceph smb share create demo1 photos cephfs --subvolume=g1/sv1 --path=/ --share-name='Photo Archive - create smb share resource
• ceph orch ls - View services status

Switch to windows client and connect to share at \\192.168.76.200\Photo Archive demonstrating windows client access. Connect to same share at \\192.168.76.201\Photo Archive demonstrating the share through a different host.

Demo 2

This demo is more advanced. It shows setting up three clusters via the declarative interface to the smb mgr module. One cluster uses custom ports while two others use custom bind addresses showing cluster colocation on just three ceph nodes. One cluster is configured to use the remote control sidecar service.

Recording

YouTube

Resource Definitions

The YAML based resource definitions used in this demo:

resources:
  - resource_type: ceph.smb.cluster
    cluster_id: adminsonly
    auth_mode: user
    user_group_settings:
      - ref: admins
    placement:
      count: 1
      label: ilovesmb
    custom_ports:
      smb: 5445
      smbmetrics: 9009
  - resource_type: ceph.smb.usersgroups
    users_groups_id: admins
    linked_to_cluster: adminsonly
    values:
      users:
        - name: bob
          password: Passw0rd
        - name: alice
          password: l3tm31n
      groups: []
  - resource_type: ceph.smb.share
    cluster_id: adminsonly
    share_id: admin
    name: 'Admin Documents'
    cephfs:
      volume: cephfs
      path: /
      subvolumegroup: g1
      subvolume: admins
  ############################### 
  # Shared by c76 and c78
  - resource_type: ceph.smb.join.auth
    auth_id: join1-admin
    auth:
      username: Administrator
      password: Passw0rd
  ############################### 
  - resource_type: ceph.smb.cluster
    cluster_id: c76
    auth_mode: active-directory
    domain_settings:
      realm: DOMAIN1.SINK.TEST
      join_sources:
        - source_type: resource
          ref: join1-admin
    custom_dns:
      - "192.168.76.210"
    placement:
      count: 3
    bind_addrs:
      - network: "192.168.76.0/24"
    remote_control:
      cert: {ref: cert1}
      key: {ref: key1}
      ca_cert: {ref: cacert1}
  - resource_type: ceph.smb.share
    cluster_id: c76
    share_id: supervol
    name: 'Super Volume'
    cephfs:
      volume: cephfs
      path: /
      subvolumegroup: g1
      subvolume: sv1
  - resource_type: ceph.smb.share
    cluster_id: c76
    share_id: sv2
    name: 'Soup Vegetables'
    cephfs:
      volume: cephfs
      path: /
      subvolumegroup: g1
      subvolume: sv2
  - resource_type: ceph.smb.tls.credential
    tls_credential_id: cert1
    credential_type: cert
    value: |
      -----BEGIN CERTIFICATE-----
      MIIGKjCCBBKgAwIBAgIURu7XJYRnwdp+CE7EJgW6umWOkJgwDQYJKoZIhvcNAQEN
      BQAwgYgxCzAJBgNVBAYTAlVTMQswCQYDVQQIDAJNQTEPMA0GA1UEBwwGTG93ZWxs
      MR8wHQYDVQQKDBZCaXJjaCBTdHJlZXQgQ29tcHV0aW5nMRYwFAYDVQQDDA1Kb2hu
      IE11bGxpZ2FuMSIwIAYJKoZIhvcNAQkBFhNqb2hubUBhc3luY2hyb25vLnVzMB4X
      DTI1MDcyMTE3NTMwNFoXDTI2MDgxNDE3NTMwNFowaDELMAkGA1UEBhMCVVMxCzAJ
      BgNVBAgMAk1BMQ8wDQYDVQQHDAZMb3dlbGwxHzAdBgNVBAoMFkJpcmNoIFN0cmVl
      dCBDb21wdXRpbmcxGjAYBgNVBAMMEUROUzpjeC5mZG9wZW4ubmV0MIICIjANBgkq
      hkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAsRZPtoE66Llr22yWZD1hUIh1Nbxj6Ct3
      tLfvq8VkKWNrUEVhE8eDxR0iE8PnXCZVrtUuH1BnWbk20RxIJNtaiCHp48NMMkbN
      3jkIBvqJFCsH61+rwysSSQ1G9T8D3GWBzUaf4ttSogDYZ0SKbBJgQk4LP4bAMtyV
      CESD9Prfvs9PgadsClZfBN1VVoG2aGdVeGt+Cwr6Ede2SYlcLSEDMQgzC2q0r0am
      eoVJq4gN5T54d19Du8hDnxPthUX4C5uJdQ4CLEOKKo25p29wpPqjVAP5KDVWJii3
      3CdATTsIznuIyLJIEjBpdv2BU9YIrFv46gCr9j4DDNrtCW0BQX256dsZfyKzXHe9
      v6YEcK1ctwf8cS+LiMeHGONC6LuCQmmsJrTJS8vP72tSQZLT9tFNNjVKXIwDbcGn
      u59kjNrzNNBH0VXSEourcE1w3puB06yEXSG8lZvaAcWdfnYmDrnJboYYFAQSkoAH
      AegMb6iZN063rHbka6sqbhAxjTukuO9m7NQ/XVQ5vvffyAgjfanOcmQrikmbOWPT
      QWJ14f6YT40bLt5BG14eEiuRPtvLQkKLbZwcFTcpEvNLfPFIP7AzqpqMXn4z1Md7
      I4uHmz5B9Xh0+B0I7tTNbOHtkmlCI26/ZAgHU2GTq+prBNzG5+k0bo2gL2Jp08dZ
      NONmFi0mEfUCAwEAAaOBqjCBpzAJBgNVHRMEAjAAMDsGA1UdEQQ0MDKCDWN4LmZk
      b3Blbi5uZXSCDyouY3guZmRvcGVuLm5ldIcEwKhMyIcEwKhMyYcEwKhMyjAdBgNV
      HSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwHQYDVR0OBBYEFKmUM2u3jkipFH+I
      x2TLLcGWGgg5MB8GA1UdIwQYMBaAFCjmzojTtaat5YwPL/52kWh0zJ1VMA0GCSqG
      SIb3DQEBDQUAA4ICAQBcL+plftpcayUFWjtOB062exRsv1nlXZwaa8K0tzWFrVeb
      IN+PGROZjjjHXT9+YTgqH9CDj66jbeKcmcUggqetMxBeQLfa+OQ/pe14nm9/sqNB
      EXCHyOo8YAKZK9cLCfXT1Qiqw2+reB12uZfQyy9Uno+UIotMoUj8OgP1vygo2unn
      BxyMjMsY5cmmMbE5ot76NVEF6HW5ATxQ9qhfAWiVAyfumePJGscKVYMHNVK+xplK
      4DIUD50zZBCAj6A42MRoXd8dwEg8ctOkG2KWemqN7s169yTOyb43LM07ECNs59S5
      bVD/+5vwFoMBQWzf5rMebUGWyuBQ+t5zA8cQQqh3y+JMyxS5f84r0kN8WaMdjdIa
      qVAf7/4YjdNN3qKZZ4N+nvXtF4IEmj6zOU1yGXB6V7RJMMy166rXUflqC0p+cmRm
      N9Vg7Zf259902TrKMDNDm132fgdfPi/xY7HJZ3b/Hhv0I4THC5pqeScrrteIgUYz
      L6LwkHv6lk8u73ZV/57mNPz+hu0x/Ge05S9gfB4Wx29sp9YsCN62kYKzzqdW4DbD
      5StGfUdQaSzSvXyLRNwExOYMd9yDn2D0ak6ZL9AOvK7VRvFVBuVzFNm6KKoITgi3
      YKLLkkVAF53hOvc0sresNU0o8XEJkKChY3MWSXKFgInXUuM+YMDDDAMkgMRncQ==
      -----END CERTIFICATE-----
  - resource_type: ceph.smb.tls.credential
    tls_credential_id: key1
    credential_type: key
    value: |
      -----BEGIN PRIVATE KEY-----
      MIIJQQIBADANBgkqhkiG9w0BAQEFAASCCSswggknAgEAAoICAQCxFk+2gTrouWvb
      bJZkPWFQiHU1vGPoK3e0t++rxWQpY2tQRWETx4PFHSITw+dcJlWu1S4fUGdZuTbR
      HEgk21qIIenjw0wyRs3eOQgG+okUKwfrX6vDKxJJDUb1PwPcZYHNRp/i21KiANhn
      RIpsEmBCTgs/hsAy3JUIRIP0+t++z0+Bp2wKVl8E3VVWgbZoZ1V4a34LCvoR17ZJ
      iVwtIQMxCDMLarSvRqZ6hUmriA3lPnh3X0O7yEOfE+2FRfgLm4l1DgIsQ4oqjbmn
      b3Ck+qNUA/koNVYmKLfcJ0BNOwjOe4jIskgSMGl2/YFT1gisW/jqAKv2PgMM2u0J
      bQFBfbnp2xl/IrNcd72/pgRwrVy3B/xxL4uIx4cY40Lou4JCaawmtMlLy8/va1JB
      ktP20U02NUpcjANtwae7n2SM2vM00EfRVdISi6twTXDem4HTrIRdIbyVm9oBxZ1+
      diYOucluhhgUBBKSgAcB6AxvqJk3TresduRrqypuEDGNO6S472bs1D9dVDm+99/I
      CCN9qc5yZCuKSZs5Y9NBYnXh/phPjRsu3kEbXh4SK5E+28tCQottnBwVNykS80t8
      8Ug/sDOqmoxefjPUx3sji4ebPkH1eHT4HQju1M1s4e2SaUIjbr9kCAdTYZOr6msE
      3Mbn6TRujaAvYmnTx1k042YWLSYR9QIDAQABAoICAFgMONwMSwb1UmxKBEiYwC7z
      ehuarK3+Fsmy/qaQQWnWtb+2jzrvY1P5VT0wlXMa2FVSR1lod8qDrX6xKimxKsUu
      34TQUK1ayPcpshUOMmPltU1RRyyF2NSblmFP0JKfobc32z1HQUGtW+uUh6KOTHAL
      L8qJLDzdsffnYRjIN8E+gFB9ttk48ouPpOObIN+uKS/zejrxXT7L2yWa90q09EpH
      wmF1z5qqbRHT/bocrpQByJHJuvq96ulIVjuCe38B+YlKysfJWQQpbVqTrRVKTTTz
      uMlGI6xbaHc8yNrddFZ905e+FNl+WqHAD7JoY+2W2R838EhBsvCMVXjvSsWORoXr
      6rvQKHEgpHXjGfv8GNg46XsQRkocLAfwipDjCliGsu7khDkazE9l4ag6Q0n6BZ8c
      tmgpxELj5vhiZaDdso2scHPQ01v/uBFMGRsb82WM+HKG+foDB2LIjOCF0oKaxpBs
      /qLTpOrc5OELekIzdWFR1s98FT94Tf0bKUQvkYBhIRQaabDM4tVFqhWRn3YWd6FF
      BHgmRJJ9tuzPr4mke8zRFWRPoxwBvwMD32niJiQtvb68SW7xqFbc9pZMQf1LRaTt
      uO4cm4mLmESb8ULlZOl/X9f4BxPUX5jwFojS6TttoBLDTkmtPJGUjWtT9dmXPyw9
      6WFqL1KGPwWHzI1onTitAoIBAQDs7fnIRYwrAFjDactMApKsnKkuwZDt2T3sY7zb
      CLqtjtthzlufTz+acScK/mIgzCXBGxS0Yz5HJj53Jv85Gv807aJjzNXS4K+BpaPo
      QI0Zb7275ZBSx95n2QGp+HEtnwH4dhqjtYgLPIuhG54Kn82Lc8QybDRkHvs86ZqB
      INC4EZQUhsqbzV5DN2QwbNoqHzX3sWxU5YYdZ0FsQ4/HFZAYuFPcSS0enHqcPpjd
      OH3fcDBec/DPwi30cHm93oMcwWfMygJ29C9k5MbvCqd4HvivwKAzwpi0RsfT2UpO
      i6fdSQmBIhHySiQBfk1M03MGXIgZdSrgw6VptP9pOdhNwUyDAoIBAQC/V0Js9cwk
      oOEMIkcebMxkGNijD0Ik+Th9cIzd7WxKyluRTc9pnz0afjz3Oc1/GTu9ZuujdMAQ
      5MaIaQVANcillpDDUBweBViIZhGodtYlnB4eKYlHxXwm1J+4tbdGYE8wiL+rTLAP
      7shTiDa1EURPR9M4l8VI90cqj/0HkrN4qdUSe060BtMzWtB3mSnwCGx13A2j6Ly5
      W31OLWiJYOh5Q2lDSnr6/5w0MYEYshB6IE2G4Hx/Kk1TaP+sronWYhVF7nNllTdr
      Gum1y2Q9njEk/Zn19mIav4fC08AFQ7TLkGsDtZlV6MHrnpITmVDpN3wh/knnvKkV
      9KDmHGTMts4nAoIBACvPHP6RDYft+nu3liWp3CEanpXMqNWx86dAEe2WQZ5R6fK9
      y2c7qhEOlx+LRe22kcyRC1UHfL5/LdGuXkba3RGWVw6JE4h4jzszu4j4Vp67cKPL
      oNINruwYzhv1mkfLPPwKKobWzB73xQG3L5PbVJBSiZahN1bD+8SZlT4HKVC/v1fE
      TLZeVO2s0lO57OUY/EuefN99yTyqmQhBvdYcAJbxjmAPD863NFyrnmxZQ70K64ar
      fX2M9B5cpcYb3LZ2dJEDUU1ZfWH0g2wz1h956pXk0jp/4uiCpfRgG0NGw3VBSgac
      nCDm4J1+EyD/gCdO+MNsTM9enblcBFYaogpOka0CggEAHhJcwjsvRmJyfDG287Ut
      ul2dTqNLRfxVQIG74fU6m9aJ5aBRBMyeEdfdRQ20mlEg7neKeToUJZOPEElRJfJO
      AphTUB77DEzSyT0hcVSb+3U51ou4o6sKBCBOqf0FPO5OA9a6KPX4hciFe3tg/bwk
      /EDRlYdk3j7e1HWk0tfflQs8DZeLmwenkd6n/OP5j1wDj9Jx4lzlrEwQuYSxdUXh
      w40wsf62rVQW/Kz7GSKDLG5/QL9vUZ0YYg+Sf1U/HI51wXPkdwaomxCzwnqg2n5O
      Qv4IgHukl4eakqgs+abWd1fPOdoEq1fVgm59js79xGmP/Ne2TdF80ZKM5x13SiVA
      6QKCAQBRYNMJB8MAsPHytsQeKGNu7Xs+3nTSsnsZqGbiSbTHZTj02CA/zw2ULhux
      MuNmjyzgkkem/LV8XtkNTi0Ii7oBJO4BzrEdhVKCLoxD6FXGzMcyRRjdT6ma6YuW
      97EeIXlwyozzMVGCOsSukJfHxeg6V4+E09AWTBf8UNfRxpXl6jFyV+IUKOxhROEf
      oPskV+07AFVYFtowQ2MPMQlPt6OjVyb0SAeUEnJJ4eHM8QdnSkcICvZ6udffg8b4
      MtbLgx+McoE0nX++sSr+y2+J+XcrbK6vGuwDzFO462xvjQIzEkK4q4ILhQVZdJLE
      5zbKv+GvBv8Sm/y0WmJh1kBPRuhk
      -----END PRIVATE KEY-----
  - resource_type: ceph.smb.tls.credential
    tls_credential_id: cacert1
    credential_type: ca-cert
    value: |
      -----BEGIN CERTIFICATE-----
      MIIF8zCCA9ugAwIBAgIUHlGH70I309k8k3wR7hVKz4emYZIwDQYJKoZIhvcNAQEN
      BQAwgYgxCzAJBgNVBAYTAlVTMQswCQYDVQQIDAJNQTEPMA0GA1UEBwwGTG93ZWxs
      MR8wHQYDVQQKDBZCaXJjaCBTdHJlZXQgQ29tcHV0aW5nMRYwFAYDVQQDDA1Kb2hu
      IE11bGxpZ2FuMSIwIAYJKoZIhvcNAQkBFhNqb2hubUBhc3luY2hyb25vLnVzMB4X
      DTI1MDcyMTE3NDg1MFoXDTI5MDcyMDE3NDg1MFowgYgxCzAJBgNVBAYTAlVTMQsw
      CQYDVQQIDAJNQTEPMA0GA1UEBwwGTG93ZWxsMR8wHQYDVQQKDBZCaXJjaCBTdHJl
      ZXQgQ29tcHV0aW5nMRYwFAYDVQQDDA1Kb2huIE11bGxpZ2FuMSIwIAYJKoZIhvcN
      AQkBFhNqb2hubUBhc3luY2hyb25vLnVzMIICIjANBgkqhkiG9w0BAQEFAAOCAg8A
      MIICCgKCAgEAu7SxhNfuVFePUHzG6OogriGWRcB+LPBebZzD6f1ZoNcXHRTSobD4
      c86fraQLbHtlt2w55kpvj+Gh8bJ/UXZ5klCFh/p4w5ZC2djUVc9FVr5TZHVrbltn
      vVrpwmqVkE1ZKzK7qYAVht+wEsVz8+RC7WP2wETqbtlmn1Ew98UKzhwL245L4w4s
      da1Zay8J+hQk9ODch/3+NQ1+vsdt3Sdx4s2HFPmD07VmbN3WWfcTbYs1FuVQ9/Ug
      gmMiRkwk04P5OwD0OdduLxNQQu3rxcmD3uohFTAF0PmDKpUR8i0dKBjMUSSRXLBU
      t5EPSibxdbjGVYRco3S4/BWIyPHYCs9ttdgvY3eamE87pWJEbjwkiH+d6c99XD02
      mWbYsUhNRecrKGoKf8NxOfhxgKX3ZyFawFZPnYTcs+lPrekuHfEMYht0sHGGuVme
      Dh17wp11lxyZjyANZhlxpJXLqaFSlAZXkvKjTg6AZoJe5V39lmfzWCy43BWVC75+
      GytnyDuccs1KTlz21KcfAFegWehC/6SUywBolqd5xAgB6XjyQO8AkGsBMIuC2vbt
      jOE6+hlXaGzBfB4nc4bLPsWKPt2UwhifQaRt6iyjauxIVaoelA5a0mpA8MRPajHL
      5nCkvfJjXpiLAaZtg2nknXi3WLp15tqgxrml/qmKNhD4tBhGyyev34MCAwEAAaNT
      MFEwHQYDVR0OBBYEFCjmzojTtaat5YwPL/52kWh0zJ1VMB8GA1UdIwQYMBaAFCjm
      zojTtaat5YwPL/52kWh0zJ1VMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQEN
      BQADggIBAJOEp7lwdGfvWZGzNGz+0lCOpSkLx9W98PNUtJeeZDGWPtcbTor5+ux9
      T60iCh8vJs3qrdCtUjiyyQ44SWroJ1dcr0pBiq99GzRycOBOS/DQysrz5MeYNkAM
      R9p9wn3gwxDJn2POuk3Gkwz70qFoEiJJFnWKKLdjWy5aUgA41egWcfxGX0NjnMTJ
      Nsvg6Ard3iJpHh5DDqNs2ieZT9h1xYuFdQI2vJdJJ9/2KQsT6Nb5uL7q54RO34A3
      5z2i7zsbwrRBiIR6mYAs0dyO6WktKEhgf6W3l+VnTXxNZap8AVVE/QIKLc4gMi7I
      w2C1M/FpSAsNMO1ZNjQZ0esrTJEaBvz7NCayDSmLkSlKDOie/LsLDdGWfTvc9/xH
      EtvtordPgpMfAjoQo/ZnMadY65DZofM4/b22/ta4Ki1j5L4ZEEEGCS8uOmAhtwy2
      NDO2C7vnABnHq14N0DFpXZqqIMl3dBzxNAtcZnI1UBCVAPnEpxSwgEbCkgPoMbyb
      PxPtgQupgR1AWx3gWOYEiRZYqbZUD0IbBsB1vpKxxKMBIHsdWvmzBnDGqAd4zR3U
      qh4+08lblN1MdS6nDourhrvCcc9vO/KFMQuzB/5ZkWtiKlwBzf0RsUbLtKfs9Uje
      q34kVWrNQahLKW9RKpe7VvUCiNk9UNufY44PGEKU9y3Vj97380DU
      -----END CERTIFICATE-----
  ############################### 
  - resource_type: ceph.smb.cluster
    cluster_id: c78
    auth_mode: active-directory
    domain_settings:
      realm: DOMAIN1.SINK.TEST
      join_sources:
        - source_type: resource
          ref: join1-admin
    custom_dns:
      - "192.168.76.210"
    placement:
      count: 3
    bind_addrs:
      - network: "192.168.78.0/24"
  - resource_type: ceph.smb.share
    cluster_id: c78
    share_id: supervol
    name: 'Extreme Volume'
    cephfs:
      volume: cephfs
      path: /
      subvolumegroup: g1
      subvolume: e1

Commands

Commands used in the demo follow

SMB Configuration

• ceph fs subvolume create cephfs --group-name=g1 --sub-name=admins --mode=0777 --casesensitive=false - create a cephfs subvolume to demonstrate the --casesensitive being set to false, making the subvolume case insenstive
• ceph orch ls - Show cephadm services before creating smb clusters
• ceph smb apply -i - < demo2.yaml - submit the resource definitions to the smb mgr module
• ceph orch ls - Show cephadm services after creating smb clusters
• ceph smb show --format=yaml --password-filter=hidden - view resources as configured, avoid printing sensitive data on the console
• ceph smb show --format=yaml --password-filter=hidden ceph.smb.cluster - show only cluster resources
• ceph smb show --format=yaml --password-filter=hidden ceph.smb.tls.credential - show only tls credential resources

Admins cluster client connection

• smbclient -U 'bob%Passw0rd' --port 5445 //192.168.76.200/'Admin Documents' - connect to the share on the cluster with custom port values

Remote Control

The grpcurl command makes use of TESTTLSDIR, to locate a directory holding tls certs, and SAMBACCDIR to locate a directory containing the sambacc git checkout where a copy of the grpc proto file is available.
Note that the remote control gRPC service must have matching CA, server, and client certs for the
grpcurl commands to work successfully.

• smbclient -U 'domain1\bwayne' //192.168.76.200/'Soup Vegetables' - connect linux client to share
• grpcurl -cacert $TESTTLSDIR/ca/ca.crt -cert $TESTTLSDIR/edfu.crt -key $TESTTLSDIR/edfu.key -import-path $SAMBACCDIR/sambacc/grpc/protobufs/ -proto control.proto 192.168.76.200:54445 SambaControl/Status - get client connection information
• grpcurl -cacert $TESTTLSDIR/ca/ca.crt -cert $TESTTLSDIR/edfu.crt -key $TESTTLSDIR/edfu.key -import-path $SAMBACCDIR/sambacc/grpc/protobufs/ -proto control.proto -d '{"ip_address": "192.168.76.1"}' 192.168.76.200:54445 SambaControl/KillClientConnection - disconnect the linux client