Backend using DevOps.

5 min readOct 15, 2023

Follow steps and run pipeline monitor in Grafana.

Learn How companies work in Backend and theory behind it.

Understanding DevOps

Small intro about DevOps

himavarshahimavarsha.medium.com

This is a demo project that assists in setting up a CI/CD pipeline using Kubernetes. We utilize an AWS server and monitor log analytics and infrastructure to enhance operational efficiency.

Prerequisites:

Download below tools.

EKS
AWS cli
KUBERNATES
Create IAM user after you create AWS account.
GIT BASH
Helm

The primary task is to manage the Kubernetes cluster. For runtime, we rely on AWS EC2 in the backend to handle instances and ensure the clusters run without failures according to our requirements.

Connect AWS to your cli

aws configure

Command to install Kubernetes in local server

2. Create Kubernetes cluster in AWS

eksctl create cluster --name hima1 --region us-east-1

We will create the second cluster

eksctl create cluster --name mycluster1 --region=us-east-1

eksctl get  cluster --name mycluster1 --region=us-east-1

kubectl get nodes

We need to create few dependencies.

kubectl create --filename https://raw.githubusercontent.com/operator-framework/operator-lifecycle-manager/master/deploy/upstream/quickstart/crds.yaml

kubectl create --filename https://raw.githubusercontent.com/operator-framework/operator-lifecycle-manager/master/deploy/upstream/quickstart/olm.yaml

kubectl get catalogsources -n olm

kubectl get packagemanifests -l catalog=operatorhubio-catalog

How do we create Jenkins using Kubernetes?

Kubernetes has a declarative API, and you can convey the desired state using either a YAML or JSON file. For this tutorial, you will use a YAML file to deploy Jenkins. Make sure you have the kubectl command configured for the cluster.

kubectl create namespace lwns

We create namespace to organize the cluster.

Feasibility of Deploying Kubernetes Helm Using Helm Client from Jenkins

helm.exe  repo add jenkins https://raw.githubusercontent.com/jenkinsci/kubernetes-operator/master/chart

Create a repository.

helm.exe install my-jenkins-operator jenkins/jenkins-operator -n lwns --set jenkins.enabled=false

kubectl --namespace lwns get pods -w

Create Jenkins instance file in GitBash.

apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: example
  namespace: lwns
spec:
  configurationAsCode:
    configurations: []
    secret:
      name: ""
  groovyScripts:
    configurations: []
    secret:
      name: ""
  jenkinsAPISettings:
    authorizationStrategy: createUser
  master:
    disableCSRFProtection: false
    containers:
      - name: jenkins-master
        image: jenkins/jenkins:lts
        imagePullPolicy: Always
        livenessProbe:
          failureThreshold: 12
          httpGet:
            path: /login
            port: http
            scheme: HTTP
          initialDelaySeconds: 100
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 5
        readinessProbe:
          failureThreshold: 10
          httpGet:
            path: /login
            port: http
            scheme: HTTP
          initialDelaySeconds: 80
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources:
          limits:
            cpu: 1500m
            memory: 3Gi
          requests:
            cpu: "1"
            memory: 500Mi
  seedJobs:
    - id: jenkins-operator
      targets: "cicd/jobs/*.jenkins"
      description: "LW Jenkins Operator repository"
      repositoryBranch: master
      repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.gitkubectl create -f jenkins_instance.yaml

This command will help you to jenkins instance in your local pc.

kubectl create -f jenkins_instance.yaml

kubectl --namespace lwns get pods -w

Create user and password for jenkins

kubectl --namespace lwns get secret jenkins-operator-credentials-example -o 'jsonpath={.data.user}' | base64 -d
kubectl --namespace lwns get secret jenkins-operator-credentials-example -o 'jsonpath={.data.password}' | base64 -d

Run Jenkins in your host server.

kubectl --namespace lwns port-forward jenkins-example 8080:8080
https://localhost:8080

The job has been created and the pipeline is running.

Grafana Operations

Grafana is an open-source analytics and interactive visualization web application. It provides charts, graphs, and alerts for the web when connected to supported data sources1. Grafana is a visualizing tool with a dashboard that provides a lot of options with graphs to see the data and understand them2. It can be used to view metrics, do queries, and get alerts of logs being generated2. Grafana provides tools to turn your time-series database (TSDB) data into beautiful graphs and visualizations.

helm.exe repo add grafana https://grafana.github.io/helm-charts

helm.exe repo update


helm.exe upgrade --install loki grafana/loki-stack  --set grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=false

kubectl get pods

kubectl patch svc loki-grafana -p '{"spec": {"type": "LoadBalancer"}}'

Get the username and password to login to Grafana.

kubectl get svc loki-grafana -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'
kubectl get secret loki-grafana -o go-template='{{range $k,$v := .data}}{{printf "%s: " $k}}{{if not $v}}{{$v}}{{else}}{{$v | base64decode}}{{end}}{{"\n"}}{{end}}'