Dependencies

In this section, we’ll delve into the various out-of-the-box dependencies that each discovered service receives, managed by the platform. These components are an assortment of elements like secrets, service accounts, namespaces, roles, taskruns, and more.

The platform delivers these components with dedicated support to ensure both pipelines and application execution run smoothly, eliminating the need for manual configuration. Let’s explore each of these dependencies in more detail below.

1. Namespaces

For every new service that our platform identifies within our environments, we’ll initiate the creation of two distinct namespaces. These are specifically designed for different purposes - one is allocated for Continuous Integration/Continuous Deployment (CI/CD), while the other caters to application development.

Adhering to a particular naming convention ensures clarity and easy identification. Taking service-1 as an example of a service name, the corresponding namespaces would be established as follows:

• service-1-ci-cd: This namespace is dedicated for Continuous Integration and Continuous Deployment. It’s here where the pipeline associated with service-1 will reside. The CI/CD process that encompasses code commit, build, test, and deployment stages happens within this namespace. Look for the next Pipelines section for more detail.

• service-1-development: This namespace serves as the staging ground for the development of service-1. It’s where developers can build, test, and fine-tune the application in a contained, independent environment, minimizing the risk of interfering with the broader platform.

• service-1-prod: This namespace serves as the production environment of service-1. It’s where customers can access the live, tested and fine-tuned application.

This namespace strategy enhances organization, isolation, and security, making it easy to manage each service’s CI/CD processes and application development independently. Let’s verify:

kubectl get ns -A | grep service-1

service-1-ci-cd              Active   4h
service-1-development        Active   4h
service-1-prod               Active   4h

2. AWS ECR (Elastic Container Registry)

An essential element of our automated infrastructure management is the AWS Elastic Container Registry (ECR). Each of our repositories requires access to pull and push images from and to this image registry, necessitating the use of secrets. However, these secrets (or tokens) provided by AWS tend to expire regularly. To address this, we have a job that runs hourly to keep the tokens updated and functional.

This process is facilitated by an ‘Opaque’ secret that contains the AWS credentials. Thereafter, a Job is run using the xynova/aws-kubectl:latest image. The purpose of this Job is to authenticate with AWS ECR and create a secret called regcred.

The regcred secret

This regcred secret is integral to the operations of our pipeline. In the later stages of the pipeline, Tekton utilizes regcred to push images. Additionally, regcred is required by our applications when they need to run using the images built during the pipeline execution. So let’s investigate this:

kubectl get secret regcred -n service-1-ci-cd && kubectl get secret regcred -n service-1-development

Both regcred secrets should be present, one in the “ci-cd” namespace and the other one in the “development” namespace. If there’s an issue, you can look into the logs of the Job pod for potential errors or failures.

kubectl logs job/create-regcred-ci-cd -n service-1-development
kubectl logs job/create-regcred-dev -n service-1-development

Creating the image registry

But what about the repositories for our image registry? Where do we place the images we construct during our pipelines? We have automated processes to handle that as well. A TaskRun, specifically create-aws-ecr-reg.yaml, is designed to interact with the AWS CLI to generate a new image repository, provided it doesn’t already exist.

The execution of this process is expected to happen only once for each service. You can investigate this by executing the following: kubectl get tr -n service-1-development.

And the response should be something similar to this:

NAMESPACE               NAME                                SUCCEEDED   REASON      STARTTIME   COMPLETIONTIME
service-1-development   create-aws-ecr-reg        True      Succeeded   8m50s       8m42s

For forcing the reexecution of the Jobs, you can delete them:

kubectl delete job/create-regcred-ci-cd -n service-1-development
kubectl delete job/create-regcred-dev -n service-1-development

3. GitHub Webhooks

The initialization of the Tekton task create-github-resources.yaml is a crucial step in the onboarding of a new service onto our platform. This task facilitates the integration of the newly discovered repository with the Tekton EventListener.

On GitHub, every push event trigger dispatches a payload to our platform, thus initiating a PipelineRun. Previously, this process required tedious manual intervention, involving navigation through GitHub interface links. Thankfully, we’ve automated this procedure through a TaskRun.

This TaskRun performs a crucial function - it automatically generates a unique webhook URL (for example, https://eu-north-1.training.dx-book.com/github/dx-book/service-1/webhook) and sets up a new webhook using the GitHub API.

To verify if the TaskRun has been successfully executed, you can use the command kubectl get taskrun -A. You should expect to see an output similar to the following:

NAMESPACE   NAME                                SUCCEEDED   REASON      STARTTIME   COMPLETIONTIME
argocd      create-github-resources             True        Succeeded   71s         9s

To access the logs from this TaskRun, you can run the command:

tkn tr logs service-1-create-github-resources -n argocd

Upon successful execution of the TaskRun, two new webhooks should be created within the service-1 repository on GitHub. You can verify this by navigating to https://github.com/dx-book/service-1/settings/hooks.

The two new webhooks are as follows:

1. https://eu-north-1.training.dx-book.com/github/dx-book/service-1/webhook - This webhook triggers a new PipelineRun for every new commit made.
2. https://argocd.eu-north-1.training.dx-book.com/api/webhook - This webhook ensures a prompt response from ArgoCD whenever code changes are pushed. It enables us to use a push model, allowing for a faster response than the default 3-minute pull refresh model that ArgoCD typically employs.

In the event of a TaskRun failure, you can restart the execution by deleting it using the command:

kubectl delete taskrun/create-github-resources -n argocd