FPGA Device Plugin

A Daemonset deployed on the Kubernetes (k8s) cluster for the discovery of FPGAs inserted in each node of the cluster and running FPGA accessible containers in the k8s cluster.

Asset Description

The Xilinx FPGA device plugin for Kubernetes is a Daemonset deployed on the Kubernetes(k8s) cluster which allows you to:

• Discover the FPGAs inserted in each node of the cluster and expose information about these resources such as number of FPGAs, Shell (Target Platform) type, etc.
• Run FPGA accessible containers in the k8s cluster.

Technical Information

To offer FPGA resources to Pods running in Kubernetes we can leverage Device Plugins. This is a framework provided by Kubernetes that can be used to advertise system hardware resources to the Kubelet. Instead of customizing the code for Kubernetes itself, vendors can implement a device plugin that can be deployed as a DaemonSet.

Xilinx implemented its own device plugin to manage PCI-express connected FPGAs to servers. However, in the case of the DAIS project, the device used is a Xilinx embedded FPGA: Zynq UltraScale+ MPSoC ZCU102, with an arm64 +FPGA. The Xilinx device plugin implementation did not offer support for the ZCU102 board, so we had to modify it with the purpose of including support for this specific device.

Although a pull request has been made to request Xilinx to add this feature to their official repository, it has not been approved at the time being. They do not use Github Workflows in their development, so they have to test it internally before accepting the contribution.

Therefore, the modified FPGA device plugin is currently published in the following GitHub repository within the Gradiant Organization.

Thus, this work allows to distinguish the Kubernetes Nodes within a cluster that have access to a FPGA, and to manage the orchestration of workloads in a way that the FPGA resources are automatically detected in the cluster and are exclusively allocated to a single workload at a time. To this end, this FPGA as a Service device plugin detects and uses a zocl driver and a XRT (Xilinx Runtime).

Usage

The instructions that have to be followed in order to use this device plugin are the same that Xilinx provides in its official documentation.

However, for using the modified version, we have to clone Gradiant’s GitHub repository. The Docker image must be built using the Dockerfile provided in this repo and the corresponding files. Then, to deploy the DaemonSet on the Kubernetes cluster, the provided fpga-device-plugin.yml has to be used instead of Xilinx’s one.

Besides using this work directly, it can be easily modified to add support for new devices and boards.

Licence

Apache License Version 2.0. License link here

Resources

• Available in Github: k8s-fpga-device-plugin
• Created by: - Gradiant (Centro Tecnológico de Telecomunicaciones de Galicia)

Acknowledgement

This asset was created in the context of the DAIS (Distributed Artificial Intelligence Systems) project, which is a pan-European project bringing faster, more secure and energy efficient data processing solutions through the development of edge AI software and hardware components.