I’ve just come across Antidote a recent project that intends at running networking-oriented labs over Kubernetes. It is developped by members of the Network Reliability Engineering community (Juniper-related, AFAIU), to power the NRE Labs platform.
It looks very similar to other platforms that allow you to run labs remotely in consoles opened on cloud machines, alongside lab instructions.
I find it interesting as the code is published under FLOSS license (Apache), and seems to be runable over any Kubernetes installation: you can test it with Minikube through the selfmedicate repo.
Antidote demo running virtual labs in Kubernetes with selfmedicate/minikube, running locally from Olivier Berger on Vimeo.
Internally, it uses Guacamole to provide the Web consoles connected via SSH to the hosts (or emulated devices) running on the k8s cluster. Each learner will get her own k8s namespace running the corresponding PODs.
In principle, it’s rather easy to package any app that can be used from the CLI to run it over Antidote.
The main drawback I’ve found so far, wrt our experiments with virtual labs, is the limitation to SSH access for a CLI: the Guacamole setup doesn’t provide access to VNC, AFAICS (yet).
Quite interesting and promising project anyway.
I wanted to be able to test deployment of VMs inside a Kubernetes cluster with KubeVirt (and maybe Kata-Containers too), using a VM-based testbed. An additional requirement was trying to get a similar setup script that could be applied to a regular physical host, should the tests be conclusive.
I’ve adapted an existing installation procedure for installation on Vagrant Ubuntu VMs, and the result is there:
I runs a Vagrant VM with libvirt+qemu/kvm, installed with Ubuntu 18.04, and provisions a Kubernetes cluster using kubeadm (no longer minikube, unlike what was done in the original repo I’ve forked from).
Inside the cluster, containers are run with CRI-O, which seems to be compatible with KubeVirt. Packages are “official” k8s packages (except for CRI-O and KubeVirt).
I hope it’ll work for you too, provided that you can run nested virtualization.
Next steps will be trying I’ve also added Kata-Containers to the cluster. It seems that kata-containers and KubeVirt are compatible (all running with libvirt below).
Here’s a screencast :
Updated: I’ve now added Kata-Containers deployment to the provisioning scripts. Also, it runs 18.04 now (LTS too, but more recent than 16.04). Oh, and the scripts seem to be useable outside Vagrant, on a regular Ubuntu server too.
Next steps, testing various workloads on the cluster for virtual labs deployments.