Using NGINX Ingress in EKS with TLS Termination in a Network Load Balancer

“File:VaticanMuseumStaircase.jpg” by Andreas Tille is licensed under CC BY-SA 4.0

This article details the installation of the Kubernetes-managed NGINX Ingress Controller for use with a Network Load Balancer (NLB) in an EKS cluster. After considering the recently announced AWS Load Balancer Controller, we went with the NGINX Controller to take advantage of the scalability of load balancing TCP traffic as well as TLS termination.

This guide is for the Kubernetes-managed NGINX Ingress Controller (kubernetes/ingress-nginx), but the same principles may be applied to the NGINX-managed NGINX Ingress Controller (nginxinc/kubernetes-ingress). View a breakdown of the differences between these two controllers.

Prerequisites

• EKS cluster with version > 1.14
• ACM certificate for TLS
• kubernetes/ingress-nginx repo cloned locally

Installation

Edit the deployment manifest

Edit deploy-tls-termination.yaml found in deploy/static/provider/aws/

1. Edit the LoadBalancer service with the following annotations. Be sure to add the ACM certificate ARN to the aws-load-balancer-ssl-cert annotation. If you do not have any public subnets available (like us), you will also need to add and set the aws-load-balancer-internal annotation to true. Here is a full list of relevant annotations for the load balancer service.

NOTE: the targetPort values for the https and http ports in the spec are meant to be http since TLS is terminating in the NLB, otherwise these values would not be changed from their defaults.

Also NOTE: externalTrafficPolicy MUST be set to Cluster (not Local) or the health checks on the NLB target groups will fail.

2. Remove proxy-real-ip-cidr from the ConfigMap and set use-forwarded-headers to false. use-forwarded-headers is for OSI L7 load balancers (e.g. ALBs), and NLBs are L4 so we can set this to false, and since that is disabled, proxy-real-ip-cidr becomes unnecessary.

3. Deploy the controller.

$ kubectl apply -f deploy/static/provider/aws/deploy-tls-termination.yaml

Upon creation, an NLB will begin provisioning and the service will not be available until it is complete. You can check its provisioning status with:

$ kubectl describe svc/ingress-nginx-controller -n ingress-nginx

Create a service and ingress

Create a simple service and ingress to test the functionality of the controller.

1. Deploy two simple applications. These applications will show text on a webpage when hit.

2. Create a service to expose the application

3. Create an Ingress resource that the controller will detect. This Ingress will route requests like this:

https://chocolate.habeebmohammed.com → chocolate-service → chocolate-app:80

https://chocolate.habeebmohammed.com/vanilla → vanilla-service → vanilla-app:80

4. Add the NLB DNS name (AWS Console → EC2 → Load Balancers → Details) to Route53 or other DNS service.

5. Navigate to your new application. In this case, https://chocolate.habeebmohammed.com

That’s it! You’ve successfully created an NGINX ingress controller that uses an NLB with LB-side TLS termination.