Building an experimental virtual labs cloud with Eclipse Che on a private k8s cloud in the university

I’ve been experimenting with various virtual labs technologies in the last years. Even if the recent months have kept me away from my blog, I think it’s time for an update on what we’re trying to experiment, when the pandemics consequences permit.

We’re aiming at experimenting, for next fall’s back to school, the use of Eclipse Che for computer science labs, for some of our students. I’ve “hired” a team of motivated students to help me with this project, and we bought a nice server to host our experiments.

Eclipse Che provides a “so called” Kubernetes-ready Web IDE and other tools that can be used in programming/development projects (test tools, language integrations, etc.). Why not for the labs in the university.

We’re looking forward to test its use by our students, deployed on a private k8s cluster, to be able to evaluate the UX for our coding labs. This should provide some benefits in terms of ease of deployment for programming environments that our students can use for distance learning, compared to the current situation which requires local installation of an IDE, and language tools. This should save us of having to support such installations on various operating systems (including machines in our traditional labs, inside the school walls).

The pandemics has forced us to switch to distance learning, without sufficient preparation (and resources), which resulted in rather chaotic situations for some students. We seek to evaluate if such “cloud labs” could bring some agility in the rollout of some classes.

Beyond the evaluation of the learner experience, this evaluation also aims at understanding if this brings some benefits for the professors too. I’m thinking of applying some sort of devops-related process to the fabrication of learning materials, for the labs parts. This may result in additional quality. I’ve written before on some of these ideas, if you’re curious. I’m not completely sure of the reception by my faculty colleagues, sometimes afraid by all this cloud buzz.

Another aspect of this evaluation would be on the TCO or other related economic aspects. The impact on our IT department could be meaningful, if we’re changing paradigm.

As we’ll be using Eclipse Che in the novel context of programming labs, this should require the development of some integration with our LMS. The initial step is authenticating students, which should be feasable with Keycloak and Shibboleth/SAML. Next, I’m thinking of an LTI plugin to plug Eclipse Che and Moodle. In the end, I’d like to have some tools to be able to provide traces of activity for the labs, so as to include that in learner activity dashboards. This could be used for formative evaluation, grading and such, but also to assess the calibration of effort of teaching instructions, problems, and other activities which student perform in an unattended way. This is particularly important I think so that labs aren’t performed like in a tunnel, where instructors have poor visibility on what difficulties students are facing. This is even more important in distance learning situations, we’re all confronted to at the moment.
Back to Eclipse Che, it isn’t particularly designed for learning environments, so I hope our use cases will be compatible with the other mainstream uses. I’m a member of the Eclipse community (formally, but not really active), but I hope I’ll have enough time and energy to be able to bring such use cases in the community, and hopefully find other parties interested.

Somehow, the pandemics acts, here also, as a factor of increased digitization, but I’d prefer to be conservative, with a few key principles, in line with our public service nature : using free software/open source technology, hosting on our own private infrastructure (respecting our users privacy), drinking our own champagne (i.e. using the same tools for producing teaching materials, and delivering labs, for instance), and eventually contributing what we’ve assembled as reusable components for others.

In the end, the only measurable success factor should be the increase in learning efficiency, hopefully.

I’ll try to blog on our progress along the way. Stay tuned.

Mixing NRELab’s Antidote and Eclipse Che on the same k8s cluster

You may have heard of my search for Cloud solutions to run labs in an academic context, with a focus on free an open source solutions . You may read previous installments of this blog, or for a shorter, check the presentation I’ve recorded last week.

I’ve become quite interested, in the latest month, in 2 projects: NRELab’s Antidote and  Eclipse Che.

Antidote is the software that powers NRELabs, a labs platform for learning network automation, which runs on top of Kubernetes (k8s). The interesting thing is that for each learner, there can be a dedicated k8s namespace with multiple virtual nodes running on a separate network. This can be used in the context of virtual classes/labs where our students will perform network labs in parallel on the same cluster.

Eclipse Che powers Eclipse “on the Cloud”, making available software development environments, for developers, on a Kubernetes Cloud. Developers typically work from a Web page instead of installing local development tools.

Both projects seem quite complementary. For one, we both teach networks and software developments. So that would naturally appeal for many professors.

Furthermore, Eclipse Che provides a few features that Antidote is lacking : authenticating users (with keycloak), and persisting their work in workspaces, between work sessions. Typically what we need in our academic context where students will work on the same labs during scheduled classes, week after week, during or off-hours.

Thus it would be great to have more integration between the 2 environments.

I intend to work on that front, but that takes time, as running stuff on Kubernetes isn’t exactly trivial, at least when you’re like me and want to use a “vanilla” kubernetes.

I’ve mainly relied on running k8s inside VMs using Vagrant and/or minikube so far.

A first milestone I’ve achieved is making sure that Antidote and Eclipse Che aren’t incompatible. Antidote’s “selfmedicate” script was actually running inside a Vagrant VM, where I had difficulties installing Eclipse Che (probably because of old software, or particular networking setup details). I’ve overcome this hurdle, as I’m now able to install both environments on a single Kubernetes VM (using my own Vagrant setup).

Running Eclipse Che (alongsite Antidote) on a k8s Vagrant VM.

This proves only that there’s no show stopper there, but a lot of work remains.

Stay tuned.

Update: I’ve finally managed to get it to work on the antidote-selfmedicate base too. See my branch at:

Running networking labs over Kubernetes with Antidote

I’ve just come across Antidote a recent project that intends at running networking-oriented labs over Kubernetes. It is developped by members of the Network Reliability Engineering community (Juniper-related, AFAIU), to power the NRE Labs platform.

It looks very similar to other platforms that allow you to run labs remotely in consoles opened on cloud machines, alongside lab instructions.

I find it interesting as the code is published under FLOSS license (Apache), and seems to be runable over any Kubernetes installation: you can test it with Minikube through the selfmedicate repo.

Antidote demo running virtual labs in Kubernetes with selfmedicate/minikube, running locally from Olivier Berger on Vimeo.

Internally, it uses Guacamole to provide the Web consoles connected via SSH to the hosts (or emulated devices) running on the k8s cluster. Each learner will get her own k8s namespace running the corresponding PODs.

In principle, it’s rather easy to package any app that can be used from the CLI to run it over Antidote.

The main drawback I’ve found so far, wrt our experiments with virtual labs, is the limitation to SSH access for a CLI: the Guacamole setup doesn’t provide access to VNC, AFAICS (yet).

Quite interesting and promising project anyway.

Testing KubeVirt + Kata-Containers for running VMs inside Kubernetes in a Vagrant + qemu VM (updated)

I wanted to be able to test deployment of VMs inside a Kubernetes cluster with KubeVirt (and maybe Kata-Containers too), using a VM-based testbed. An additional requirement was trying to get a similar setup script that could be applied to a regular physical host, should the tests be conclusive.

I’ve adapted an existing installation procedure for installation on Vagrant Ubuntu VMs, and the result is there:

I runs a Vagrant VM with libvirt+qemu/kvm, installed with Ubuntu 18.04, and provisions a Kubernetes cluster using kubeadm (no longer minikube, unlike what was done in the original repo I’ve forked from).

Inside the cluster, containers are run with CRI-O, which seems to be compatible with KubeVirt. Packages are “official” k8s packages (except for CRI-O and KubeVirt).

I hope it’ll work for you too, provided that you can run nested virtualization.

Next steps will be trying I’ve also added Kata-Containers to the cluster. It seems that kata-containers and KubeVirt are compatible (all running with libvirt below).

Here’s a screencast :


Updated: I’ve now added Kata-Containers deployment to the provisioning scripts. Also, it runs 18.04 now (LTS too, but more recent than 16.04). Oh, and the scripts seem to be useable outside Vagrant, on a regular Ubuntu server too.

Next steps, testing various workloads on the cluster for virtual labs deployments.