There are several ways to setup a local development environment for Kubernetes, the two most common ones are
Minikube (docs) and
Minikube provisions a local Kubernetes cluster on several Virtual Machines (via VirtualBox or something similar).
docker-desktop on the other hand, sets up Kubernetes cluster in docker containers.
Local Kubernetes cluster using Docker Desktop
In this tutorial, we will base all the installs on Docker Desktop. Even in this case we can use a lightweight minikube setup which gives the same functionality with less impact.
Download and install Docker-Desktop on your laptop. Latest version has an embedded version of Kubernetes so no additional install is needed. Just simply follow the instruction here to get Kubernetes up and running within docker-desktop.
Once Kubernetes is started in docker desktop, you should see something similar below:
This means that:
- Kubernetes is running.
- the command line tool
kubctlis installed in the
- the Kubernetes context is set to
Deploy and access dashboard
After setting up the local Kubernetes you need to deploy the dashboard using the following steps:
- follow the instructions in Kubernetes dashboard doc to deploy the dashboard.
- start the Kubernetes proxy in the background from a terminal to get access on the dashboard on the local host: kubectl proxy &
- access the dashboard at the following URL: clickable link
Access local Kubernetes cluster
The dashboard as deployed in the previous step requires a token or config to sign in. Here we use the token to sign in. The token is generated automatically and can be retrieved from the system.
- retrieve the name of the dashboard token:kubectl -n kube-system get secret | grep kubernetes-dashboard-token
- retrieve the content of the token, note that the token name ends with a random 5 character code and needs to be replaced with the result of step 1. As an example: kubectl -n kube-system describe secret kubernetes-dashboard-token-tf6n8
- copy the token value which is part of the
Datasection with the tag
- select the Token option in the dashboard web UI:
- paste the token value into the input box and sign in:
Local Kubernetes cluster with Minikube
Minikube can be added to an existing Docker Desktop install. Minikube can either use the pre-installed hypervisor or use a hypervisor of choice. These instructions use HyperKit which is embedded in Docker Desktop.
If you want to use a different hypervisor then HyperKit make sure that you follow the generic minikube install instructions. Do not forget to install the correct driver for the chosen hypervisor if required. The basic instructions are provided in the minikube install instructions.
Docker Desktop should have already installed HyperKit. In a terminal run:
hyperkit to confirm. Any response other than
hyperkit: command not found confirms that HyperKit is installed and on the path. If it is not found you can choose a different hypervisor or fix the Docker Desktop install.
- install minikube, you can either use brew or directly via these steps: curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-darwin-amd64chmod +x minikubesudo mv minikube /usr/local/bin
- install HyperKit driver
(required), you can either use brew or directly via these steps:curl -LO https://storage.googleapis.com/minikube/releases/latest/docker-machine-driver-hyperkitsudo install -o root -g wheel -m 4755 docker-machine-driver-hyperkit /usr/local/bin/
- update the minikube config to default to the HyperKit install
minikube config set vm-driver hyperkit
- change docker desktop to use minikube for Kubernetes:
Deploy and access the cluster
After the installation is done you can start a new cluster.
- start the minikube cluster:
minikube start --kubernetes-version v1.14.2
- start the minikube dashboard:
minikube dashboard &
When you create images make sure that the build is run after pointing it to the right environment. Without setting the enviromnent minikube might not find the docker images when deploying the scheduler.
- make sure minikube is started
- in the terminal where you wll run the build execute:
eval $(minikube docker-env)
- run the image build from the yunikorn-k8shim repository root:
- deploy the scheduler as per the normal instructions.
Debug code locally
Note, this instruction requires you have GoLand IDE for development.
In GoLand, go to yunikorn-k8shim project. Then click "Run" -> "Debug..." -> "Edit Configuration..." to get the pop-up configuration window.
Note, you need to click "+" to create a new profile if the
Go Build option is not available at the first time.
The highlighted fields are the configurations you need to add. These include:
- Run Kind: package
- Package path: point to the path of
- Working directory: point to the path of the
confdirectory, this is where the program loads configuration file from
- Program arguments: specify the arguments to run the program, such as
-kubeConfig=/path/to/.kube/config -interval=1s -clusterId=mycluster -clusterVersion=0.1 -name=yunikorn -policyGroup=queues -logEncoding=console -logLevel=-1. Note, you need to replace
/path/to/.kube/configwith the local path to the kubeconfig file. And if you want to change or add more options, you can run
_output/bin/k8s-yunikorn-scheduler -hto find out.
Once the changes are done, click "Apply", then "Debug". You will need to set proper breakpoints in order to debug the program.
Access remote Kubernetes cluster
This setup assumes you have already installed a remote Kubernetes cluster. For a generic view on how to access a multiple cluster and integrate it follow the accessing multiple clusters documentation from Kubernetes.
Or follow these simplified steps:
- get the Kubernetes
configfile from remote cluster, copy it to the local machine and give it a unique name i.e.
- save the
KUBECONFIGenvironment variable (if set)export KUBECONFIG_SAVED=$KUBECONFIG
- add the new file to the environment variableexport KUBECONFIG=$KUBECONFIG:config-remote
- run the command
kubectl config viewto check that both configs can be accessed
- switch context using
kubectl config use-context my-remote-cluster
- confirm that the current context is now switched to the remote cluster config:kubectl config get-contextsCURRENT NAME CLUSTER AUTHINFO NAMESPACEdocker-for-desktop docker-for-desktop-cluster docker-for-desktop* my-remote-cluster kubernetes kubernetes-admin
More docs can be found here