Are there two Load Balancer Controllers with EKS ?

Load Balancer Controller — Which one you are using ?

I stumbled once again when I was exposing the Kubernetes service via Network Load Balancer and discovered two ways of creating NLB in EKS.

Have you ?

Why I stumbled upon confluence between AWS Load Balancers and Kubernetes Service ?

Scenario

• EKS cluster is setup
• AWS Load Balancer Controller is deployed
• Deployed some nginx pods
• Deployed service to be accessible outside the EKS cluster via Network Load Balancer (NLB)

Configuration 1 — Service Type : LoadBalancer with No Annotations

#nginx-service-aws-legacy-controller.yaml
apiVersion: v1
kind: Service
metadata:
 name: nginx-service
spec:
 ports:
 — port: 80
 targetPort: 80
 protocol: TCP
 type: LoadBalancer
 selector:
 app: nginx

Deploy the service.

kubectl apply -f nginx-service-aws-legacy-controller.yaml

The load balancer is created with following details.

Classic Load Balancer

Tags on classic load balancer

Observations

• Classic type load balancer is created
• Load Balancer DNS name starts with adxx-141xxx-ap-south-1.elb.amazonaws.com
• Tags are added e.g. kubernets.io/service-name : default/nginx-service

Configuration 2 — Using aws-load-balancer-type: nlb annotation

#nlb-nginx-service-aws-legacy-controller.yaml
apiVersion: v1
kind: Service
metadata:
 name: nginx-service
 annotations:
 service.beta.kubernetes.io/aws-load-balancer-type: nlb
spec:
 type: LoadBalancer
 selector:
 app: nginx
 ports:
 — protocol: TCP
 port: 80
 targetPort: 80

Create service in EKS cluster.

kubectl apply -f nlb-nginx-service-aws-legacy-controller.yaml

The following gets created.

Network type load balancer

Tags

Observations

• Network load balancer is created
• Load Balancer DNS name starts with adxx-16xxx.elb.ap-south-1.amazonaws.com
• Note the tag name — kubernets.io/service-name : default/nginx-service

Configuration 3 — Using aws-load-balancer-type: “external” annotation

apiVersion: v1
kind: Service
metadata:
  name: nlb-nginx-service
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "external"
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: "ip"
    service.beta.kubernetes.io/aws-load-balancer-scheme: "internet-facing"
spec:
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP
  type: LoadBalancer
  selector:
    app: nginx

It creates the following load balancer in AWS.

Observations

• Creates Network load Balancer
• Load Balancer DNS Name is very different compared to earlier ones (it starts with k8s-default-xxx, the dns name is having xx.elb.region.xx format as opposed to xx.region.elb.xx format as seen in earlier configrations)

K8s-default-nlbnginx-bb9e563aad-6f889376a2258726.elb.ap-south-1.amazonaws.com

• Tags are also quite different as seen below.

Note the difference in tags

What is going on here ?

It looks like these load balancers are created bit differently in configurtion 3…

The external value for aws-load-balancer-type is what causes the AWS Load Balancer Controller, rather than the AWS cloud provider load balancer controller, to create the Network Load Balancer.

Two different load balancer controllers !

Wait a minute…so it implies there are two different types of Controllers creating the load balancers in the EKS cluster.

The Configuration 1 and Configuration 2 kicks in AWS cloud provider load balancer controller (I tend to call it legacy now ) where as the Configuration 3 kicks in the AWS Load Balancer Controller (new one) !

Summary

Even if you use AWS Load Balancer Controller add-on with EKS the AWS cloud provider load balancer controller (legacy ) still kicks in based on what is specified in the service annotation !

Annotation + Type of LB + Load Balancer Controller

https://www.buymeacoffee.com/amodkadam