Kubectl Kubernetes CheatSheet

Common Commands

Name Command
List everything kubectl get all --all-namespaces
List pods with nodes info kubectl get pod -o wide
Validate yaml file with dry run kubectl apply --dry-run --validate -f pod-dummy.yaml
Diff deployed resource with local yaml file kubectl alpha diff -f <yaml file/dir>
Start a temporary pod for testing kubectl run --rm -i -t --image=alpine test-$RANDOM -- sh
Run wget test temporarily kubectl run --rm mytest --image=busybox -it
Run curl test temporarily kubectl run --rm mytest --image=yauritux/busybox-curl -it
kubectl run shell command kubectl exec -it mytest -- ls -l /etc/hosts
Get system conf via configmap kubectl -n kube-system get cm kubeadm-config -o yaml
kubectl run instance with replicas kubectl run my-nginx --image=nginx --replicas=2 --port=80
Explain resource kubectl explain pods=, kubectl explain svc
Get all services kubectl get service --all-namespaces
Query healthcheck endpoint curl -L http://127.0.0.1:10250/healthz
Open a bash terminal in a pod kubectl exec -it storage sh
Check pod environment variables kubectl exec redis-master-ft9ex env
Enable kubectl shell autocompletion echo "source <(kubectl completion bash)" >>~/.bashrc, and reload
Use minikube dockerd in your laptop eval $(minikube docker-env), No need to push docker hub any more
Get services sorted by name kubectl get services --sort-by=.metadata.name
Get pods sorted by restart count kubectl get pods --sort-by='.status.containerStatuses[0].restartCount'
Get logs from init container kubectl logs <pod-name> -c <init-container-2
Drain a nodes kubectl drain <node_name> --delete-local-data --ignore-daemonsets

Check Performance

Name Command
Get node resource usage kubectl top node
Get pod resource usage kubectl top pod
Get resource usage for a given pod kubectl top <podname> --containers
List resource utilization for all containers kubectl top pod --all-namespaces --containers=true

Resources Deletion

Name Command
Delete pod kubectl delete pod/<pod-name> -n <my-namespace>
Force delete kubectl delete pod NAME --grace-period=0 --force
Delete pods by labels kubectl delete pod -l env=test
Delete deployments by labels kubectl delete deployment -l app=wordpress
Delete all resources filtered by labels kubectl delete pods,services -l name=myLabel
Delete resources under a namespace kubectl -n my-ns delete po,svc --all
Delete persist volumes by labels kubectl delete pvc -l app=wordpress
Delete statefulset only (not pods) kubectl delete sts/<stateful_set_name> --cascade=false

Pod

Name Command
List all pods kubectl get pods
List pods for all namespace kubectl get pods -all-namespaces
List all critical pods kubectl get -n kube-system pods -a
List pods with more info kubectl get pod -o wide=, kubectl get pod/<pod-name> -o yaml
Get pod info kubectl describe pod/srv-mysql-server
List all pods with labels kubectl get pods --show-labels
kubectl run command kubectl exec -it -n "$ns" "$podname" -- sh -c "echo $msg >>/dev/err.log"

Pod Advanced

Name Command
Get pod by selector kubectl get pods -n $namespace --selector="app=syslog" -o jsonpath='{.items[*].metadata.name}')
List pods and containers kubectl get pods -o='custom-columns=PODS:.metadata.name,CONTAINERS:.spec.containers[*].name'
List pods, containers and images kubectl get pods -o='custom-columns=PODS:.metadata.name,CONTAINERS:.spec.containers[*].name,Images:.spec.containers[*].image'

Label & Annotations

Name Command
Filter pods by label kubectl get pods -l owner=denny
Manually add label to a pod kubectl label pods dummy-input owner=denny
Remove label kubectl label pods dummy-input owner-
Manually add annonation to a pod kubectl annotate pods dummy-input my-url=https://www.dennyzhang.com

Deployment & Scale

Name Command
Scale out kubectl scale --replicas=3 deployment/nginx-app
Online rolling upgrade kubectl rollout app-v1 app-v2 --image=img:v2
Roll backup kubectl rollout app-v1 app-v2 --rollback
List rollout kubectl get rs
Check update status kubectl rollout status deployment/nginx-app
Check update history kubectl rollout history deployment/nginx-app
Pause/Resume kubectl rollout pause deployment/nginx-deployment=, resume
Rollback to previous version kubectl rollout undo deployment/nginx-deployment

Quota & Limits & Resource

Name Command
List Resource Quota kubectl get resourcequota
List Limit Range kubectl get limitrange
Customize resource definition kubectl set resources deployment nginx -c=nginx --limits=cpu=200m,memory=512Mi

Service

Name Command
List all services kubectl get services
List service endpoints kubectl get endpoints
Get service detail kubectl get service nginx-service -o yaml
Expose deployment as lb service kubectl expose deployment/my-app --type=LoadBalancer --name=my-service
Expose service as lb service kubectl expose service/wordpress-1-svc --type=LoadBalancer --name=wordpress-lb

StatefulSet

Name Command
List statefulset kubectl get sts
Delete statefulset only (not pods) kubectl delete sts/<stateful_set_name> --cascade=false
Scale statefulset kubectl scale sts/<stateful_set_name> --replicas=5

Volumes & Volume Claims

Name Command
Check the mounted volumes kubectl exec storage ls /data
Check persist volume kubectl describe pv/pv0001
List storage class kubectl get storageclass
Copy files kubectl cp /tmp/foo <namespace1>/<pod1>:/tmp/bar

Security

Name Command
List certificates kubectl get csr

Extenstions

Name Summary
List api group kubectl api-versions
List all CRD kubectl get crd

Components & Services

Services on Master Nodes

Name Summary
kube-apiserver exposes the Kubernetes API from master nodes
etcd reliable data store for all k8s cluster data
kube-scheduler schedule pods to run on selected nodes
kube-controller-manager node controller, replication controller, endpoints controller, and service account & token controllers

Services on Worker Nodes

Name Summary
kubelet makes sure that containers are running in a pod
kube-proxy perform connection forwarding
Container Runtime Kubernetes supported runtimes: Docker, rkt, runc and any OCI runtime-spec implementation.

Tools

Name Summary
kubectl the command line util to talk to k8s cluster
kubeadm the command to bootstrap the cluster
kubefed the command line to control a Kubernetes Cluster Federation
Kubernetes Components Kubernetes Components

Other Components

Log files

Name Command
API Server.log in master node /var/log/kube-apiserver.log
Scheduler.log in master node /var/log/kube-scheduler.log
Controller.log in master node /var/log/kube-controller-manager.log
Kubelet.log in worker node /var/log/kubelet.log
Kube Proxy.log in worker node /var/log/kubelet-proxy.log

Events & Metrics

Name Command
View all events kubectl get events --all-namespaces

Namespace & Security

Name Command
List authenticated contexts kubectl config get-contexts
Switch context kubectl config use-context <cluster-name>
Delete the specified context kubectl config delete-context <cluster-name>
List all namespaces defined kubectl get namespaces
kubectl config file ~/.kube/config

Network

Name Command
Temporarily add a port-forwarding kubectl port-forward redis-izl09 6379
Add port-forwaring for deployment kubectl port-forward deployment/redis-master 6379:6379
Add port-forwaring for replicaset kubectl port-forward rs/redis-master 6379:6379
Add port-forwaring for service kubectl port-forward svc/redis-master 6379:6379
Get network policy kubectl get NetworkPolicy

Basic

Key Concepts

Name Summary
CNCF Cloud Native Computing Foundation
CRI Container Runtime Interface
CNI Container Network Interface
CSI Container Storage Interface

Kubernetes Critical Files

Name Comment
Config folder /etc/kubernetes/
Certificate files /etc/kubernetes/pki/
Credentials to API server /etc/kubernetes/kubelet.conf
Superuser credentials /etc/kubernetes/admin.conf
Kubernets working dir /var/lib/kubelet/
Docker working dir /var/lib/docker/
Etcd working dir /var/lib/etcd/
Network cni /etc/cni/net.d/
Docker container log /var/log/containers/
Log files /var/log/pods/
Env export KUBECONFIG=/etc/kubernetes/admin.conf
Env /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

Check status

Name Summary
List everything kubectl get all --all-namespaces
Get cluster info kubectl cluster-info
Get configuration kubectl config view
Get kubectl version kubectl version
Get component status kubectl get componentstatus
Similar to docker ps kubectl get nodes
Similar to docker inspect kubectl describe pod/nginx-app-413181-cn
Similar to docker logs kubectl logs
Similar to docker exec kubectl exec
Get services for current namespace kubectl get svc
Get node status kubectl describe node/<node_name>

Names of certificates files

Names of certificates files:

  • ca.crt, ca.key (CA certificate)
  • apiserver.crt, apiserver.key (API server certificate)
  • apiserver-kubelet-client.crt, apiserver-kubelet-client.key (client certificate for the apiservers to connect to the kubelets securely)
  • sa.pub, sa.key (a private key for signing ServiceAccount )
  • front-proxy-ca.crt, front-proxy-ca.key (CA for the front proxy)
  • front-proxy-client.crt, front-proxy-client.key (client cert for the front proxy client)

Misc scripts

Tail pod log by label

namespace="mynamespace"
mylabel="app=mylabel"
kubectl get pod -l "$mylabel" -n "$namespace" | tail -n1 \
    | awk -F' ' '{print $1}' | xargs -I{} \
      kubectl logs -n "$namespace" -f {}

Get node hardware resource utilization

kubectl get nodes --no-headers \
     | awk '{print $1}' | xargs -I {} \
     sh -c 'echo {}; kubectl describe node {} | grep Allocated -A 5'

kubectl get nodes --no-headers | awk '{print $1}' | xargs -I {} \
    sh -c 'echo {}; kubectl describe node {} | grep Allocated -A 5 \
     | grep -ve Event -ve Allocated -ve percent -ve -- ; echo'

Apply yaml files recursively

# create
time ls -1 */*.yml | grep -v namespace | xargs -I{} kubectl apply -f {}

# delete
time ls -1r */*.yml | grep -v namespace | xargs -I{} kubectl delete -f {}

Apply the configuration in manifest.yaml and delete all the other configmaps that are not in the file.

kaubectl apply --prune -f manifest.yaml --all --prune-whitelist=core/v1/ConfigMap

List all critical pods

kubectl --namespace kube-system get pods

for pod in $(kubectl --namespace kube-system get pods -o jsonpath="{.items[*].metadata.name}"); do
    node_info=$(kubectl --namespace kube-system describe pod $pod | grep "Node:")
    echo "Pod: $pod, $node_info"
done

Helm: The Kubernetes Package Manager

Name Summary
Helm a chart manager.
Charts packages of pre-configured Kubernetes resources.
Release a collection of Kubernetes resources deployed to the cluster using Helm.
tiller helm server manages releases (installations) of your charts.
Name Summary
helm init
helm list
helm list -a
helm repo update
helm install stable/mysql
helm install --name mysql-release stable/mysql
helm inspect stable/mysql
helm status $helm_name
helm delete $helm_name
helm delete --purge $helm_name
~/.helm/cache/archive

Release, list, inspect, delete, rollback, purge

Metric Server

Metrics Server is a cluster-wide aggregator of resource usage data. Metrics Server registered in the main API server through Kubernetes aggregator.

Name Summary
Core metrics node/container level metrics; CPU, memory, disk and network, etc.
Custom metrics refers to application metrics, e.g. HTTP request rate.

Today (Kubernetes 1.7), there are several sources of metrics within a Kubernetes cluster

Name Summary
Heapster k8s add-on
Cadvisor a standalone container/node metrics collection and monitoring tool.
Kubernetes API does not track metrics. But can get real time metrics

Today, heapster is the source of the time-series data for the Kubernetes Dashboard. All other Kubernetes components will supply their own metrics in a Prometheus format

In logging domain, we can say syslog is the standard format In metric domain, maybe we can choose prometheus as the standard format.

Last Updated: 1/4/2019, 5:58:57 PM