Azure Pipelines

intro

In this post we will go through the deployment of the Application Gateway Ingress Controller (AGIC) into AKS and the solution it provides.

This is an attempt to simplify its deployment by providing scripts and resources which are available on my Github Repo Adeelku/aks-agic and will be demonstrated in this post.

AGIC’s Benefits

It allows the use of a single Application Gateway Ingress Controller to control multiple AKS clusters. It also helps eliminate the need to have another load balancer/public IP in front of AKS cluster and avoids multiple hops before requests reach the AKS cluster. Application Gateway talks to pods directly using their private IP and does not require NodePort or KubeProxy services. This also increases the deployment’s performance.

Ingress Controller is exclusively supported by Standard_v2 and WAF_v2 SKUs, which also brings autoscaling benefits. Application Gateway can react in response to an increase or decrease in traffic load and scale accordingly, without consuming any resources from the AKS cluster.

Using Application Gateway helps protect the AKS cluster by providing TLS policy and Web Application Firewall (WAF) functionality.

Ingress Controllers

What is an Ingress Controller?

An Ingress is a Kubernetes API object that manages external access to the services in a cluster, typically HTTP. It can be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name based virtual hosting. An Ingress controller is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic.

An ingress controller is a piece of software that provides reverse proxy, configurable traffic routing, and TLS termination for Kubernetes services. Kubernetes ingress resources are used to configure the ingress rules and routes for individual Kubernetes services. Using an ingress controller and ingress rules, a single IP address can be used to route traffic to multiple services in a Kubernetes cluster.

Popular ingress controllers are NGINX, Traefik, HAProxy, Envoy.

Ingress Controllers for AKS

it is possible to configure Ingress Cootrollers for AKS via different solutions:

HTTP application routing

The HTTP application routing add-on simplifie access to applications deployed to Azure Kubernetes Service (AKS) cluster. When the solution’s enabled, it configures an Ingress controller in the AKS cluster. As applications are deployed, the solution also creates publicly accessible DNS names for application endpoints.

⚠️ The HTTP application routing add-on is designed to enable Ops quickly create an ingress controller and access the applications. This add-on is not recommended for production use.

Production-ready Ingress controllers

This is the native Kubernetes solution to expose HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource

AGIC

The Application Gateway Ingress Controller (AGIC) enables exposing applications running within AKS to the Internet by leveraging Azure’s native Application Gateway L7 load-balancer. AGIC monitors the Kubernetes cluster on which it is hosted and continuously updates an Application Gateway so that selected services are exposed to the Internet.

AGIC is configured via the Kubernetes Ingress resource, along with Service and Deployments/Pods.

Deploy AGIC

Pre-requisites

yq installed on your developement machine - used for yaml templating

sudo add-apt-repository ppa:rmescandon/yq
sudo apt install yq

Basic knowledge of Kubernetes commands and components
A working AKS cluster with advanced networking enabled.
- You can use the ./aks/create_aks_adv_net.sh AZ CLI script from my Github repo adeelku/aks-modules
An Azure Application Gateway v2 with public IP in the same vnet where the AKS Cluster lives
- You can use the ./application_gtw/create_aag.sh AZ CLI script from my Github repo adeelku/aks-modules
Helm v3 deployed onto your developement machine - install Procedure

install Helm on Linux (amd64)

VERSION="v3.1.1"
wget https://get.helm.sh/helm-$VERSION-linux-amd64.tar.gz
tar -zxvf helm-$VERSION-linux-amd64.tar.gz
sudo mv linux-amd64/helm /usr/local/bin/helm
rm -rf linux-amd64/
rm helm-$VERSION-linux-amd64.tar.gz

Configure AGIC

Now we will deploy and configure the different components needed to set-up Azure Application Ingress Controller:

Create Azure Managed Identity (needs to be deployed into the MC_ Resource group automatically created when creating AKS Cluster)
This Managed Identity will be used by the AAD Pod Idenity
Grant this Managed Indentity:
- Contributor eole on the Application Gateway
- Reader role on the Application Gateway Resource Group
Deploy the Kubernetes components:
- Deploy the Managed Idenity Controller and Node Managed Identity
- Deploy application gateway ingress controller using Helm

To simplify automation of the deployment and configuration process by retrieving the Azure Resoucres infornation and using them dynamically, I prepared a Git Repo for this purpose.

Clone the repo:

git clone https://github.com/Adeelku/aks-agic.git

.
├── k8s_yaml
│   ├── aadpodidentitybinding.yaml
│   ├── aadpodidentity.yaml
│   └── sample_app.yaml
├── LICENSE
├── README.md
└── scripts
    ├── deploy_into_k8s.sh
    └── params.sh

Update the file ./scripts/params.sh to match your AKS and Application Gateway configuration

export RG_NAME="aks-poc-rg" # AKS Resource Group
export CLUSTER_NAME="aks-poc" # AKS Cluster Name
export MSI_NAME="aks-poc-msi" # Managed Identity Desired Name
export LOCATION="canadacentral" # Location of AKS Cluster
export NET_RG_NAME="net-poc-rg" # Resource Group of Application Gateway
export AAG_NAME="aag-poc" # Application Gateway Name

Deploy using the script ./scripts/deploy_into_k8s.sh

a. Managed identity will be created b. Template files for the Managed Idenity Controller and Node Managed Identity will be populated with the information needed

aadpodidentity.yaml

apiVersion: aadpodidentity.k8s.io/v1
kind: AzureIdentity
metadata:
  name:
spec:
  type: 0
  ResourceID:
  ClientID:

aadpodidentitybinding.yaml

apiVersion: aadpodidentity.k8s.io/v1
kind: AzureIdentityBinding
metadata:
  name: poc-identity-binding
spec:
  AzureIdentity:
  Selector: poc-msi

c. Azure Identity Installed and bind in Kubernetes d. Application gateway ingress controller deployed using helm and the Azure resources information

helm upgrade -i ingress-agic application-gateway-kubernetes-ingress/ingress-azure \
     --namespace default \
     --debug \
     --set appgw.name=$AAG_NAME \
     --set appgw.resourceGroup=$NET_RG_NAME \
     --set appgw.subscriptionId=$AAG_SUBID \
     --set appgw.shared=false \
     --set appgw.usePrivateIP=false \
     --set armAuth.type=aadPodIdentity \
     --set armAuth.identityResourceID=$MSI_R_ID \
     --set armAuth.identityClientID=$MSI_C_ID \
     --set rbac.enabled=true \
     --set verbosityLevel=3 \
     --set kubernetes.watchNamespace=default \
     --set aksClusterConfiguration.apiServerAddress=$AKS_API

e. Finally, Resetting template files and removing microsoft managed identity info

When the deployment is successfull, all the Kubernetes components will be in a running state:

k8s_Components

Sample applicatiom

Create a sample application to validate that agic is woring properly

kubectl apply -f k8s_yaml/sample_app.yaml 

pod/aspnetapp created
service/aspnetapp created
ingress.extensions/aspnetapp created

Get the Public IP Address of your Aplication Gatway and navigate to the Application

 az network public-ip show -n $pub-aag -g $net-poc-rg --query ipAddress -o tsv

Sample App

And that’s it! you have got agic deployed into AKS.

⚠️ Of Course in Production you want to be using Application Gateway with Firewall enabled (SKU = WAF_v2) and secure traffic using SSL/TLS certificates

AKS Application Gateway Ingress Controller