Home Containers Install Nginx Ingress Controller on Kubernetes using Helm Chart

Install Nginx Ingress Controller on Kubernetes using Helm Chart

November 9, 2023

35982

The standard way of exposing applications that are running on a set of Pods in a Kubernetes Cluster is by using service resource. Each Pod in Kubernetes has its own IP address, but a set of Pods can have a single DNS name. Kubernetes is able to load-balance traffic across the Pods without any modification in the application layer. A service, by default is assigned an IP address (sometimes called the “cluster IP“), which is used by the Service proxies. A Service is able to identify a set of Pods using label selectors.

What’s in this guide?

What is an Ingress Controller?

Before you can answer this question, an understanding of Ingress object in Kubernetes is important. From the official Kubernetes documentation, an Ingress is defined like as:

An API object that manages external access to the services in a cluster, typically HTTP.

Ingress may provide load balancing, SSL termination and name-based virtual hosting.

An Ingress in Kubernetes exposes HTTP and HTTPS routes from outside the cluster to services running within the cluster. All the traffic routing is controlled by rules defined on the Ingress resource. An Ingress may be configured to:

Provide Services with externally-reachable URLs
Load balance traffic coming into cluster services
Terminate SSL / TLS traffic
Provide name-based virtual hosting in Kubernetes

An Ingress controller is what fulfils the Ingress, usually with a load balancer. Below is an example on how an Ingress sends all the client traffic to a Service in Kubernetes Cluster:

For the standard HTTP and HTTPS traffic, an Ingress Controller will be configured to listen on ports 80 and 443. It should bind to an IP address from which the cluster will receive traffic from. A Wildcard DNS record for the domain used for Ingress routes will point to the IP address(s) that Ingress controller listens on.

It is fact that Kubernetes adopts a BYOS (Bring-Your-Own-Software) approach to most of its addons and it doesn’t provide a software that does Ingress functions out of the box. You can choose from the plenty of Ingress Controllers available. Kubedex does a good job as well on summarizing list of Ingresses available for Kubernetes.

With all the basics on Kubernetes Services and Ingress, we can now plunge into the actual installation of NGINX Ingress Controller Kubernetes.

Install Nginx Ingress Controller in Kubernetes

We shall consider two major deployment options captured in the next sections.

Option 1: Install without Helm

With this method you’ll manually download and run deployment manifests using kubectl command line tool.

Step 1: Install git, curl and wget tools

Install git, curl and wget tools in your Bastion where kubectl is installed and configured:

# CentOS / RHEL / Fedora / Rocky
sudo yum -y install wget curl git

# Debian / Ubuntu
sudo apt update
sudo apt install wget curl git

Step 2: Apply Nginx Ingress Controller manifest

The deployment process varies depending on your Kubernetes setup. My Kubernetes will use Bare-metal Nginx Ingress deployment guide. For other Kubernetes clusters including managed clusters refer to below guides:

The Bare-metal method applies to any Kubernetes clusters deployed on bare-metal with generic Linux distribution(Such as CentOS, Ubuntu, Debian, Rocky Linux) e.t.c.

Download Nginx controller deployment for Baremetal:

controller_tag=$(curl -s https://api.github.com/repos/kubernetes/ingress-nginx/releases/latest | grep tag_name | cut -d '"' -f 4)
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/${controller_tag}/deploy/static/provider/baremetal/deploy.yaml

Rename deployment file:

mv deploy.yaml nginx-ingress-controller-deploy.yaml

Feel free to check the file contents and modify where you see fit:

vim nginx-ingress-controller-deploy.yaml

Apply Nginx ingress controller manifest deployment file:

$ kubectl apply -f nginx-ingress-controller-deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created

You can update your current context to use nginx ingress namespace:

kubectl config set-context --current --namespace=ingress-nginx

Run the following command to check if the ingress controller pods have started:

$ kubectl get pods -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx --watch
NAME                                        READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create--1-hpkzp     0/1     Completed   0          43s
ingress-nginx-admission-patch--1-qnjlj      0/1     Completed   1          43s
ingress-nginx-controller-644555766d-snvqf   1/1     Running     0          44s

Once the ingress controller pods are running, you can cancel the command typing Ctrl+C.

If you want to run multiple Nginx Ingress Pods, you can scale with the command below:

$ kubectl -n ingress-nginx scale deployment ingress-nginx-controller --replicas 2
deployment.apps/ingress-nginx-controller scaled

$ kubectl get pods  -n ingress-nginx
NAME                                        READY   STATUS              RESTARTS   AGE
ingress-nginx-admission-create-lj278        0/1     Completed           0          18m
ingress-nginx-admission-patch-zsjkp         0/1     Completed           0          18m
ingress-nginx-controller-6dc865cd86-n474n   0/1     ContainerCreating   0          119s
ingress-nginx-controller-6dc865cd86-tmlgf   1/1     Running             0          18m

Option 2: Install using Helm

If you opt in for the Helm installation method then follow the steps provided in this section.

Step 1: Install helm 3 in our workstation

Install helm 3 in your Workstation where Kubectl is installed and configured.

cd ~/
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

The installer script works for both Linux and macOS operating systems. Here is a successful installation output:

Downloading https://get.helm.sh/helm-v3.13.1-linux-amd64.tar.gz
Verifying checksum... Done.
Preparing to install helm into /usr/local/bin
helm installed into /usr/local/bin/helm

Let’s query for helm package version to validate working installation:

$ helm version
version.BuildInfo{Version:"v3.13.1", GitCommit:"3547a4b5bf5edb5478ce352e18858d8a552a4110", GitTreeState:"clean", GoVersion:"go1.20.8"}

Step 2: Deploy Nginx Ingress Controller

Download latest stable release of Nginx Ingress Controller code:

controller_tag=$(curl -s https://api.github.com/repos/kubernetes/ingress-nginx/releases/latest | grep tag_name | cut -d '"' -f 4)
wget https://github.com/kubernetes/ingress-nginx/archive/refs/tags/${controller_tag}.tar.gz

Extract the file downloaded:

tar xvf ${controller_tag}.tar.gz

Switch to the directory created:

cd ingress-nginx-${controller_tag}

Change your working directory to charts folder:

cd charts/ingress-nginx/

Create namespace

kubectl create namespace ingress-nginx

Now deploy Nginx Ingress Controller using the following commands

helm install -n ingress-nginx ingress-nginx  -f values.yaml .

Sample deployment output

NAME: ingress-nginx
LAST DEPLOYED: Thu Nov  9 15:13:21 2023
NAMESPACE: ingress-nginx
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The ingress-nginx controller has been installed.
It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status by running 'kubectl --namespace ingress-nginx get services -o wide -w ingress-nginx-controller'

An example Ingress that makes use of the controller:
  apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    name: example
    namespace: foo
  spec:
    ingressClassName: nginx
    rules:
      - host: www.example.com
        http:
          paths:
            - pathType: Prefix
              backend:
                service:
                  name: exampleService
                  port:
                    number: 80
              path: /
    # This section is only required if TLS is to be enabled for the Ingress
    tls:
      - hosts:
        - www.example.com
        secretName: example-tls

If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:

  apiVersion: v1
  kind: Secret
  metadata:
    name: example-tls
    namespace: foo
  data:
    tls.crt: <base64 encoded cert>
    tls.key: <base64 encoded key>
  type: kubernetes.io/tls

Check status of all resources in ingress-nginx namespace:

kubectl get all -n ingress-nginx

Checking runningPods in the namespace.

$ kubectl get pods -n ingress-nginx
NAME                                        READY   STATUS    RESTARTS   AGE
ingress-nginx-controller-6f5844d579-hwrqn   1/1     Running   0          23m

To check logs in the Pods use the commands:

$ kubectl -n ingress-nginx  logs deploy/ingress-nginx-controller
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:       v1.9.4
  Build:         846d251814a09d8a5d8d28e2e604bfc7749bcb49
  Repository:    https://github.com/kubernetes/ingress-nginx
  nginx version: nginx/1.21.6

-------------------------------------------------------------------------------

W1109 12:13:39.757581       7 client_config.go:618] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I1109 12:13:39.757725       7 main.go:205] "Creating API client" host="https://10.233.0.1:443"
I1109 12:13:39.764721       7 main.go:249] "Running in Kubernetes cluster" major="1" minor="27" git="v1.27.7" state="clean" commit="07a61d861519c45ef5c89bc22dda289328f29343" platform="linux/amd64"
I1109 12:13:39.891007       7 main.go:101] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
I1109 12:13:39.907459       7 ssl.go:536] "loading tls certificate" path="/usr/local/certificates/cert" key="/usr/local/certificates/key"
I1109 12:13:39.916401       7 nginx.go:260] "Starting NGINX Ingress controller"
I1109 12:13:39.921967       7 event.go:298] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"d3b1fbda-6bcc-441a-bedf-4aa5f762c02b", APIVersion:"v1", ResourceVersion:"7163", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I1109 12:13:41.118796       7 nginx.go:303] "Starting NGINX process"
I1109 12:13:41.119018       7 leaderelection.go:245] attempting to acquire leader lease ingress-nginx/ingress-nginx-leader...
I1109 12:13:41.119352       7 nginx.go:323] "Starting validation webhook" address=":8443" certPath="/usr/local/certificates/cert" keyPath="/usr/local/certificates/key"
I1109 12:13:41.119662       7 controller.go:190] "Configuration changes detected, backend reload required"
I1109 12:13:41.130405       7 leaderelection.go:255] successfully acquired lease ingress-nginx/ingress-nginx-leader
I1109 12:13:41.130530       7 status.go:84] "New leader elected" identity="ingress-nginx-controller-75967d99c9-hpvfh"
I1109 12:13:41.182271       7 controller.go:210] "Backend successfully reloaded"
I1109 12:13:41.182378       7 controller.go:221] "Initial sync, sleeping for 1 second"
I1109 12:13:41.182488       7 event.go:298] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-75967d99c9-hpvfh", UID:"09b0b595-6305-4c6d-a4cc-fe020ef3fbc0", APIVersion:"v1", ResourceVersion:"7212", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration

To follow logs as they stream run:

kubectl -n ingress-nginx  logs deploy/ingress-nginx-controller -f

Upgrading Helm Release

I’ll set replica count of the controller Pods to 2:

$ vim values.yaml 
controller:
  replicaCount: 3

We can confirm we currently have one Pod:

$ kubectl -n ingress-nginx  get deploy
NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
ingress-nginx-controller   1/1     1            1           43m

Upgrade ingress-nginx release by running the following helm commands:

$ helm upgrade -n ingress-nginx ingress-nginx -f values.yaml .
Release "ingress-nginx" has been upgraded. Happy Helming!
NAME: ingress-nginx
LAST DEPLOYED: Thu Nov  9 15:13:21 2023
NAMESPACE: ingress-nginx
STATUS: deployed
REVISION: 5
TEST SUITE: None
NOTES:
The ingress-nginx controller has been installed.

Check current number of pods after the upgrade:

$ kubectl -n ingress-nginx  get deploy
NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
ingress-nginx-controller   3/3     3            3           45m

Uninstalling the Chart

To remove nginx ingress controller and all its associated resources that we deployed by Helm execute below command in your terminal.

$ helm -n ingress-nginx uninstall ingress-nginx
release "ingress-nginx" uninstalled

Configure Nginx Ingress Controller on Kubernetes

With Ingress Controller installed, we need to configure external connectivity method. For this we have two major options.

Option 1: Using Load Balancer (Highly recommended)

Load balancer is used to expose an application running in Kubernetes cluster to the external network. It provides a single IP address to route incoming requests to Ingress controller application. In order to successfully create Kubernetes services of type LoadBalancer, you need to have the load balancer implementation inside / or outside Kubernetes.

When a service is deployed in cloud environment, Load Balancer will be available to your service by default. Ingress service should get the LB IP address automatically. But for Baremetal installations you’ll need to deploy Load Balancer implementation for Kubernetes, we recommend MetalLB, use guide in link below to install it.

Deploy MetalLB Load Balancer on Kubernetes Cluster

1. Setting Nginx Ingress to use MetalLB

Check Nginx Ingress service.

$ kubectl get svc -n ingress-nginx
NAME                                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.108.4.75      <none>        80:30084/TCP,443:30540/TCP   5m55s
ingress-nginx-controller-admission   ClusterIP   10.105.200.185   <none>        443/TCP                      5m55s

If your Kubernetes cluster is a “real” cluster that supports services of type LoadBalancer, it will have allocated an external IP address or FQDN to the ingress controller.

Use the following command to see that IP address or FQDN:

$ kubectl get service ingress-nginx-controller --namespace=ingress-nginx
NAME                       TYPE       CLUSTER-IP    EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller   NodePort   10.108.4.75   <none>        80:30084/TCP,443:30540/TCP   7m53s

Patch ingress-nginx-controller service by setting service type to LoadBalancer.

kubectl -n ingress-nginx patch svc ingress-nginx-controller --type='json' -p '[{"op":"replace","path":"/spec/type","value":"LoadBalancer"}]'

Confirm successful patch of the service.

service/ingress-nginx-controller patched

Service is assigned an IP address automatically from Address Pool as configured in MetalLB.

$ kubectl get service ingress-nginx-controller --namespace=ingress-nginx
NAME                       TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)                      AGE
ingress-nginx-controller   LoadBalancer   10.108.4.75   192.168.1.30   80:30084/TCP,443:30540/TCP   10m

2. Mapping DNS name for Nginx Ingresses to LB IP

We can create domain name, preferably wildcard for use when creating Ingress routes in Kubernetes. In our cluster, we have k8s.example.com as base domain. We’ll use a unique wildcard domain *.k8s.example.com for Ingress.

The mapping is *.k8s.example.com pointing to IP address 192.168.1.30 (Nginx Ingress LB IP).