Setting Up Access Controls for Kubernetes

Report an issue with this page

The Teleport Kubernetes Service is a proxy that sits between Kubernetes users and one or more Kubernetes clusters.

In this guide, we will use a local Kubernetes cluster to show you how to configure Teleport's role-based access control (RBAC) system to manage access to Kubernetes clusers, groups, users, and resources.

How it works

When a user authenticates to Teleport, they receive a kubeconfig that lets them send requests to their authorized Kubernetes clusters via the Teleport Kubernetes Service. The Kubernetes Service can then inspect, modify, or disallow these requests depending on the privileges you have assigned to the Teleport user via their roles.

Prerequisites

• A running Teleport cluster version 17.0.0-dev or above. If you want to get started with Teleport, sign up for a free trial or set up a demo environment.

• The tctl admin tool and tsh client tool.

  Visit Installation for instructions on downloading tctl and tsh.

• To check that you can connect to your Teleport cluster, sign in with tsh login, then verify that you can run tctl commands using your current credentials. For example, run the following command, assigning teleport.example.com to the domain name of the Teleport Proxy Service in your cluster and email@example.com to your Teleport username:

  tsh login --proxy=teleport.example.com --user=email@example.com
  tctl status
  Cluster  teleport.example.com
  Version  17.0.0-dev
  CA pin   sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678

  If you can connect to the cluster and run the tctl status command, you can use your current credentials to run subsequent tctl commands from your workstation. If you host your own Teleport cluster, you can also run tctl commands on the computer that hosts the Teleport Auth Service for full permissions.

To run the local demo environment, ensure that you have the following tools installed on your workstation:

Tool	Purpose	Installation link
minikube	Local Kubernetes deployment tool	Install minikube
Helm	Kubernetes package manager	Install Helm
kubectl	Kubernetes admin CLI	Install kubectl
Docker	Required minikube driver	Get Started With Docker

Step 1/3. Prepare Kubernetes resources

Start minikube

Start minikube with the Docker driver:

minikube start --driver=docker

This command should start a local Kubernetes cluster and set your context to minikube. To verify this, run the following command:

kubectl config current-context
minikube

Deploy demo pods

On your workstation, create a manifest file called pods.yaml with the following content:

apiVersion: v1
kind: Namespace
metadata:
  name: development
  labels:
    name: development
---
apiVersion: v1
kind: Namespace
metadata:
  name: production
  labels:
    name: production
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp
  namespace: development
spec:
  selector:
    matchLabels:
      app: nginx-webapp
  template:
    metadata:
      labels:
        app: nginx-webapp
    spec:
      containers:
        - name: nginx
          image: nginx:1.23
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: webapp
  namespace: production
spec:
  selector:
    matchLabels:
      app: nginx-webapp
  template:
    metadata:
      labels:
        app: nginx-webapp
    spec:
      containers:
        - name: nginx
          image: nginx:1.23
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: loadbalancer
  namespace: development
spec:
  selector:
    matchLabels:
      app: nginx-loadbalancer
  template:
    metadata:
      labels:
        app: nginx-loadbalancer
    spec:
      containers:
        - name: nginx
          image: nginx:1.23
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: loadbalancer
  namespace: production
spec:
  selector:
    matchLabels:
      app: nginx-loadbalancer
  template:
    metadata:
      labels:
        app: nginx-loadbalancer
    spec:
      containers:
        - name: nginx
          image: nginx:1.23

This manifest creates two namespaces, development and production, and deploys two nginx pods into each one: webapp and loadbalancer. Apply the new resources:

kubectl apply -f pods.yaml

Ensure that the resources are deployed:

kubectl -n development get pods
kubectl -n production get pods

You should see both the loadbalancer and webapp pods in each namespace.

Install Kubernetes RBAC resources

Now that we have deployed our webapp and loadbalancer pods in our development and production namespaces, we will create a Kubernetes role that can view all pods in all namespaces. Later in this guide, we will define a Teleport role that further restricts the access Teleport users can have to resources in your cluster.

Create a manifest file called k8s-rbac.yaml with the following content:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: pod-viewer
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: pod-viewer
subjects:
- kind: Group
  name: developers
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: pod-viewer
  apiGroup: rbac.authorization.k8s.io

Apply your changes:

kubectl apply -f k8s-rbac.yaml

Install the Teleport Kubernetes Service

Now that you have some workloads running on Kubernetes and RBAC resources to manage access to them, install the Teleport Kubernetes Service in your demo cluster so you can get more control into the resources that Kubernetes users can access.

Set up the Teleport Helm repository.

Allow Helm to install charts that are hosted in the Teleport Helm repository:

helm repo add teleport https://charts.releases.teleport.dev

Update the cache of charts from the remote repository so you can upgrade to all available releases:

helm repo update

Request a token that the Kubernetes Service will use to join your Teleport cluster:

tctl tokens add --type=kube,app,discovery --ttl=1h --format=text

Copy this token so you can use it when running the Teleport Kubernetes Service.

Install the Teleport Kubernetes Service in your cluster, assigning proxy-address to the host and port of your Teleport Proxy Service (e.g., mytenant.teleport.sh:443) and token to the token you requested earlier:

helm install teleport-agent teleport/teleport-kube-agent \  --set kubeClusterName=minikube \  --set roles="kube\,app\,discovery" \  --set proxyAddr=proxy-address \  --set authToken=token \  --set labels.region=local --set labels.platform=minikube \  --create-namespace \  --namespace=teleport-agent \  --version 17.0.0-dev

This helm install command supplies the soon-to-be-added Kubernetes Service instance with two labels: region:local and platform:minikube. We will use these to configure access controls for the cluster later in this guide.

Verify that the teleport pod is running in your cluster:

kubectl -n teleport-agent get pods

You can check that the Teleport Kubernetes Service registered itself with your Teleport cluster by executing the following command:

tctl get kube_servers

The output should resemble the following:

kind: kube_server
metadata:
  expires: "2023-01-24T16:20:00.571214635Z"
  id: 0000000000000000000
  name: minikube
spec:
  cluster:
    kind: kube_cluster
    metadata:
      labels:
        platform: minikube
        region: local
      name: minikube
    spec:
      aws: {}
      azure: {}
      gcp: {}
    version: v3
  host_id: 00000000-0000-0000-0000-000000000000
  hostname: remote.kube.proxy.teleport.cluster.local
  rotation:
    current_id: ""
    last_rotated: "0001-01-01T00:00:00Z"
    schedule:
      standby: "0001-01-01T00:00:00Z"
      update_clients: "0001-01-01T00:00:00Z"
      update_servers: "0001-01-01T00:00:00Z"
    started: "0001-01-01T00:00:00Z"
  version: 17.0.0-dev
version: v3

Step 2/3. Define a Teleport role

The Teleport Kubernetes Service determines how to proxy a Teleport user's requests to a Kubernetes API server by looking up the user's roles. Based on this information, the Kubernetes Service accepts or denies the request.

For valid requests, the Kubernetes Service rewrites the request headers to impersonate the Teleport user's desired Kubernetes user and groups, and forwards the request to the API server.

In this section, we will define a Teleport role that:

• Authenticates the user to a Kubernetes cluster as a member of the developers group. In the previous section, we authorized members of this group to view pods in all namespaces.
• Enables the user to access webapp pods in the production namespace and all pods in the development namespace.
• Denies the user access to all other pods.

Define a role

Create a file called kube-access.yaml with the following content:

kind: role
metadata:
  name: kube-access
version: v7
spec:
  allow:
    kubernetes_labels:
      'region': '*'
      'platform': 'minikube'
    kubernetes_resources:
      - kind: pod
        namespace: "production"
        name: "^webapp-[a-z0-9-]+$"
      - kind: pod
        namespace: "development"
        name: "*"
    kubernetes_groups:
    - developers
    kubernetes_users:
    - minikube
  deny: {}

In this role, we have defined the following allow rules:

• kubernetes_labels: Allows access to Kubernetes clusters in all regions, but only with the platform:minikube label.
• kubernetes_resources: Allows access to pods in the webapp deployment in the production namespace and all pods in the development namespace. Note the use of a regular expression (beginning ^ and ending in $) to match pod names that Kubernetes generates automatically.
• kubernetes_groups: Authenticates the user to your Kubernetes cluster as a member of the Kubernetes group developers, which we associated with the pod-viewer Kubernetes Role earlier in this guide.
• kubernetes_users: Authenticates the user to your Kubernetes cluster as the default minikube user.

Create the role

Once you have finished configuring the kube-access role, create it using the following command:

tctl create kube-access.yaml

tip

You can also create and edit roles using the Web UI. Go to Access -> Roles and click Create New Role or pick an existing role to edit.

Assign the kube-access role to your Teleport user by running the appropriate commands for your authentication provider:

Local User

1. Retrieve your local user's roles as a comma-separated list:

   ROLES=$(tsh status -f json | jq -r '.active.roles | join(",")')

2. Edit your local user to add the new role:

   tctl users update $(tsh status -f json | jq -r '.active.username') \  --set-roles "${ROLES?},kube-access"

3. Sign out of the Teleport cluster and sign in again to assume the new role.

GitHub

1. Open your github authentication connector in a text editor:

   tctl edit github/github

2. Edit the github connector, adding kube-access to the teams_to_roles section.

   The team you should map to this role depends on how you have designed your organization's role-based access controls (RBAC). However, the team must include your user account and should be the smallest team possible within your organization.

   Here is an example:

     teams_to_roles:
       - organization: octocats
         team: admins
         roles:
           - access
   +       - kube-access
   

3. Apply your changes by saving closing the file in your editor.

4. Sign out of the Teleport cluster and sign in again to assume the new role.

SAML

1. Retrieve your saml configuration resource:

   tctl get --with-secrets saml/mysaml > saml.yaml

   Note that the --with-secrets flag adds the value of spec.signing_key_pair.private_key to the saml.yaml file. Because this key contains a sensitive value, you should remove the saml.yaml file immediately after updating the resource.

2. Edit saml.yaml, adding kube-access to the attributes_to_roles section.

   The attribute you should map to this role depends on how you have designed your organization's role-based access controls (RBAC). However, the group must include your user account and should be the smallest group possible within your organization.

   Here is an example:

     attributes_to_roles:
       - name: "groups"
         value: "my-group"
         roles:
           - access
   +       - kube-access
   

3. Apply your changes:

   tctl create -f saml.yaml

4. Sign out of the Teleport cluster and sign in again to assume the new role.

OIDC

1. Retrieve your oidc configuration resource:

   tctl get oidc/myoidc --with-secrets > oidc.yaml

   Note that the --with-secrets flag adds the value of spec.signing_key_pair.private_key to the oidc.yaml file. Because this key contains a sensitive value, you should remove the oidc.yaml file immediately after updating the resource.

2. Edit oidc.yaml, adding kube-access to the claims_to_roles section.

   The claim you should map to this role depends on how you have designed your organization's role-based access controls (RBAC). However, the group must include your user account and should be the smallest group possible within your organization.

   Here is an example:

     claims_to_roles:
       - name: "groups"
         value: "my-group"
         roles:
           - access
   +       - kube-access
   

3. Apply your changes:

   tctl create -f oidc.yaml

4. Sign out of the Teleport cluster and sign in again to assume the new role.

Step 3/3. Access resources

At this point, you have configured the Teleport Kubernetes Service to give your Teleport user access to the webapp pod in the production namespace. In this step, we will authenticate to your Kubernetes cluster via Teleport and test our new access controls.

List the Kubernetes clusters you can access via Teleport:

tsh kube ls

You should see the minikube cluster you registered earlier:

Kube Cluster Name Labels                         Selected
----------------- ------------------------------ --------
minikube          platform=minikube region=local

To access your Kubernetes cluster via Teleport, authenticate to it and update your kubeconfig:

tsh kube login minikube

When listing pods in all namespaces, the Teleport Kubernetes Service will filter the pods it retrieves to show only those that your Teleport user can access. Run the following command:

kubectl get pods --all-namespaces

The output will show the webapp pod in the production namespace and both the webapp and loadbalancer pods in the development namespace:

NAMESPACE     NAME                           READY   STATUS    RESTARTS   AGE
development   loadbalancer-000000000-00000   1/1     Running   0          36m
development   webapp-0000000000-00000        1/1     Running   0          36m
production    webapp-0000000000-00000        1/1     Running   0          36m

You can access information about the webapp pod in the production namespace:

kubectl -n production get pods/webapp-0000000000-00000 -o json

Also note that the kube-access role we created earlier mapped your Teleport user to the developers Kubernetes group, which has permissions only to view pods:

kubectl auth can-i create pods
no

By configuring Teleport roles and Kubernetes RBAC resources, you can fine-tune the access that users in your organization have to your Kubernetes-based infrastructure.

When you authenticated to your minikube cluster via tsh kube login, Teleport generated a kubeconfig that connects to your cluster via Teleport:

kubectl config current-context
teleport.example.com-minikube

If you want to regain full control of your minikube cluster, you can use the default minikube context instead:

kubectl config use-context minikube

Next steps

For more detailed information on how Teleport RBAC for Kubernetes works, consult the Kubernetes Access Controls Guide. You can leave your minikube cluster running so you can try out different Teleport and Kubernetes RBAC configurations.

Now that you know how to configure Teleport's RBAC system to control access to Kubernetes clusters, learn how to set up Resource Access Requests for just-in-time access and Access Request plugins so you can manage access with your communication workflow of choice.