Gitlab runner container pr 1.x (#30)

* Adding a build for a prebuilt CI image.

* Adding the docker SSH entrypoint script.

* Updating container image name.

* Making apt handling more robust.

* Moving the CI container code to just be documentation.

* Updating docs for using ce-dev to build CI containers.

Author: gregharvey

docker-images/controller-ci/Dockerfile

@@ -19,11 +19,32 @@ RUN \
   pip3 install ansible boto3 && \
   git lfs install --skip-repo && \
   update-alternatives --install /usr/bin/python python /usr/bin/python3 1 && \
+  useradd -m controller && \
+  usermod -a -G controller ce-dev && \
+  mkdir -p /home/controller/.ssh && \
+  chmod -R 700 /home/controller/.ssh && \
+  echo "Host remotehost\n\tStrictHostKeyChecking no\n" >> /home/controller/.ssh/config && \
+  mkdir -p /home/ce-dev/.ssh && \
+  chmod -R 700 /home/ce-dev/.ssh && \
+  echo "Host remotehost\n\tStrictHostKeyChecking no\n" >> /home/ce-dev/.ssh/config && \
   rm -rf \
   /var/lib/apt/lists/* \
   /var/log/* \
   /tmp/*
 
+# The keys gets copied in place by before_script in .gitlab-ci.yml
+COPY id_rsa* /home/ce-dev/.ssh/
+COPY id_rsa* /home/controller/.ssh/
+
+RUN \
+  set -x && \
+  export DEBIAN_FRONTEND=noninteractive && \
+  chown -R ce-dev:ce-dev /home/ce-dev/.ssh && \
+  chmod 600 /home/ce-dev/.ssh/id_rsa && \
+  chmod 644 /home/ce-dev/.ssh/id_rsa.pub && \
+  chown -R controller:controller /home/controller/.ssh && \
+  chmod 600 /home/controller/.ssh/id_rsa && \
+  chmod 644 /home/controller/.ssh/id_rsa.pub
 
 RUN su - ce-dev -c "git clone --branch 1.x https://github.com/codeenigma/ce-provision.git /home/ce-dev/ce-provision"
 
@@ -33,7 +54,7 @@ RUN \
   set -x && \
   export DEBIAN_FRONTEND=noninteractive && \
   apt-get update && \
-  su - ce-dev -c "/usr/local/bin/ansible-playbook /home/ce-dev/ce-provision/provision.yml" && \
+  su - ce-dev -c "/usr/local/bin/ansible-playbook --extra-vars=\"{ansible_common_remote_group: controller}\" /home/ce-dev/ce-provision/provision.yml" && \
   rm /home/ce-dev/ce-provision/provision.yml && \
   apt-get clean  && \
   rm -rf \

docker-images/controller-ci/README.md

@@ -1,16 +1,37 @@
 controller-ci
 =============
+Although `ce-dev` is a cli tool first and foremost, it can be used to pack containers to use with `ce-provision` and CI. This is simply an EXAMPLE for provisioning a controller for running `ce-provision` in a container with GitLab CI.
 
-This is simply an EXAMPLE for provisioning a controller for running ce-provision in a container with GitLab CI.
-
-Every organisation wanting to run ce-provision in a container must necessarily make their own container image which incorporates their own version of ce-provision-config and installs the dependencies for their choice of CI. There can be no such thing as a "generic" CI container because it needs to contain secrets and it needs to be tailored to the CI product.
+Every organisation wanting to run `ce-provision` in a container must necessarily make their own container image which incorporates their own version of `ce-provision-config` and installs the dependencies for their choice of CI. There can be no such thing as a "generic" CI container because it needs to contain secrets and it needs to be tailored to the CI product.
 
 As such any generated CI container *must* be private in the container registry, never make them public.
 
-The Dockerfile within is a mix of ce-dev's controller image:
+The Dockerfile within is a mix of `ce-dev`'s controller image:
 * https://github.com/codeenigma/ce-dev/tree/1.x/docker-images/controller
 
 And a GitLab Runner controller based on this project:
 * https://gitlab.com/tmaczukin-test-projects/fargate-driver-debian/-/blob/master/Dockerfile
 
-You can build this container to see how it works, but it will not work with your infra because it currently incorporates an example ce-provision-config repo.
+You can build this container to see how it works, but it will not work with your infra because it currently incorporates an example `ce-provision-config` repo.
+
+# Key handling and user management
+The main complications with using ce-dev to build a CI container are:
+* The SSH private key to access your config repo will need baking in
+* The username for provisioning infrastructure is typically `controller`, whereas the `ce-dev` containers are packed with `ce-dev` as the user
+
+To get around this you can copy an SSH key on to the container, either via a CI variable or from the machine orchestrating the CI. In our example `Dockerfile` we are handling copying a key from the host. Note in particular we create the `controller` user in advance and we add `ce-dev` to the `controller` group. Also, importantly, we this to the `ansible-playbook` command:
+* ` --extra-vars=\"{ansible_common_remote_group: controller}\"`
+
+This is important, so the ownership of tmp files created by `ce-dev` can be changed to use the `controller` group, thus the `controller` user can access them. Otherwise `become: controller` in Ansible will fail.
+
+Steps will vary with CI flavour, but with GitLab CI we use `before_script` and `after_script` to ensure the private and public keys are where `docker` needs them to be, *and* to clean up the private key when we're finished:
+
+```yaml
+before_script:
+  - cp -r /PATH/TO/KEYS/id_rsa* $CI_PROJECT_DIR/docker-images/controller-ci/
+
+after_script:
+  - rm $CI_PROJECT_DIR/docker-images/controller-ci/id_rsa
+```
+
+With these extra changes in place you should be able to pack a `docker` image that uses the `controller` user for executing `ce-provision` and has all the secrets and permissions it needs to build infrastructure.

docker-images/controller-ci/provision.yml

@@ -14,8 +14,8 @@
         config_repository: https://github.com/codeenigma/ce-dev-ce-provision-config.git
         config_repository_branch: 1.x
         config_repository_skip_checkout: false
-        username: ce-dev
-        local_dir: /home/ce-dev/ce-provision
+        username: controller
+        local_dir: /home/controller/ce-provision
         groups: []
         galaxy_custom_requirements_file: ""
   roles: