Custom CA

Created by: chess-knight

This PR addresses epic #372 (closed) - issues #373 (closed) #374 (closed) #375 (closed) #376 (closed) in that epic.

Edit: Issue #488 (closed) created - document CA rotation - will be addressed in separate PR

Terraform automatically detects cacert in clouds.yaml and transfers it to the mgmt host so openstack commands work. Then cacert is included via OPENSTACK_CLOUD_CACERT_B64 to the CAPO controller so CAPO can create an infrastructure for the workload cluster. Finally, cacert is also transferred to the master and worker nodes to the "/etc/ssl/certs" destination and then OCCM and CCSI can pick it up via hostPath mounts. There is no action for OCCM as the mounts are already present in the manifest, but for the CCSI we need to create these mounts (it is done by yq).

How I tested custom CA I used CAPO for spawning devstack on gx-scs OpenStack. See docs for CAPO e2e testing. Follow these steps to create devstack with a custom CA:

1. create ssh key
2. my environment looks like this:

   export RESOURCE_TYPE="openstack"
   export OS_CLOUD=gx-scs
   export OPENSTACK_FLAVOR_controller=SCS-16V:64
   export OPENSTACK_FLAVOR_worker=SCS-8V:32
   export OPENSTACK_PUBLIC_NETWORK=ext01
   export SSH_PUBLIC_KEY_FILE=</path/to/the/public_key>
   export SSH_PRIVATE_KEY_FILE=</path/to/the/private_key>
   
   export OPENSTACK_ADDITIONAL_SERVICES=,tls-proxy

3. Service tls-proxy will spawn devstack with tls support, and it also generates cacert
4. ./hack/ci/create_devstack.sh it will create openstack controller and worker. When tls-proxy is used worker will not connect to controller because of ssl problems and I was not able to configure scripts in that way that worker will successfully connect. But the controller is enough for these purposes.
5. cleanup can be done via ./hack/ci/create_devstack.sh cleanup

Follow these steps to test k8s-cluster-api-provider with devstack custom CA:

1. controller has public IP, so it is possible to connect via e.g. ssh -i ${SSH_PRIVATE_KEY_FILE} cloud@213.131.230.213
2. git clone https://github.com/SovereignCloudStack/k8s-cluster-api-provider.git -b custom_ca
3. cd k8s-cluster-api-provider/terraform/
4. sudo snap install yq
5. sudo snap install terraform --classic
6. create environments/environment-devstack.tfvars:

   cloud_provider    = "devstack"
   availability_zone = "testaz1"
   external          = "public"
   kind_flavor       = "m1.small"
   controller_flavor = "m1.small"
   worker_flavor     = "m1.small"
   dns_nameservers   = ["62.138.222.111", "62.138.222.222"]
   worker_count = 2
   prefix = "devstack"
   testcluster_name = "devstackcluster"
   image = "focal-server-cloudimg-amd64"
   kube_image_raw = false

7. Add interface: public to the /etc/openstack/clouds.yaml cloud devstack, cacert can be also seen there
8. comment availability_zone_hints in neutron.tf
9. export OS_CLOUD=devstack
10. make create
11. Creation of management and workload cluster should be successful, and later it can be tested e.g. with make check-conformance
12. make fullclean