Skip to content

Commit

Permalink
Rename 'pflex' device to 'crand' to reflect the usage of /dev/random
Browse files Browse the repository at this point in the history
Signed-off-by: Ivan Sim <[email protected]>
  • Loading branch information
ihcsim committed Sep 2, 2024
1 parent f1e5ad1 commit 65a74df
Show file tree
Hide file tree
Showing 10 changed files with 63 additions and 52 deletions.
4 changes: 3 additions & 1 deletion .goreleaser.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
version: 2

builds:
- env:
- main: ./cmd/crand
binary: dp-crand
env:
- CGO_ENABLED=0
goos:
- linux
Expand Down
12 changes: 7 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,13 @@ GOARCH ?= amd64

# see /proc/devices for the major number of /dev/random
# on your machine
DEVRAND_MAJOR_VERSION ?= 1
DEVRAND_MINOR_VERSION ?= 8
MAJOR_VERSION_CRAND ?= 1
MINOR_VERSION_CRAND ?= 8

build: tidy
GOOS=$(GOOS) GOARCH=$(GOARCH) go build -o ./bin/device-plugin main.go
build: crand

crand: tidy
GOOS=$(GOOS) GOARCH=$(GOARCH) go build -o ./bin/dp-crand cmd/crand/main.go

lint: tidy
golangci-lint run .
Expand All @@ -21,7 +23,7 @@ test: tidy
go test -v -race -cover ./...

run: build
sudo ./bin/device-plugin
sudo ./bin/device-plugin-crand

.PHONY: kubelet
kubelet:
Expand Down
45 changes: 26 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
# kubelet-plugin

An example of a kubelet plugin that can be used to expose node devices to a
kubelet. For more information on Kubernetes device plugins, see
This repo contains examples of kubelet plugins that can be used to expose the
following node devices to the Kubelet:

* [Character device file](https://man7.org/linux/man-pages/man2/mknod.2.html).
The `crand` plugin manages character special files pointing to the local
`/dev/random`.
* KVM device file. The `kvm` plugin points to the local `/dev/kvm`.

For more information on Kubernetes device plugins, see
https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/.

## CDI Configuration - containerd
Expand Down Expand Up @@ -42,13 +49,13 @@ Run the plugin against the kubelet:
make run
```

Expect the plugin `pflex.io/block` to register successfully with the kubelet:
Expect the plugin `github.com.ihcsim/crand` to register successfully with the kubelet:
```sh
I0823 09:22:35.030339 1309759 server.go:144] "Got registration request from device plugin with resource" resourceName="pflex.io/block"
I0823 09:22:35.030374 1309759 handler.go:95] "Registered client" name="pflex.io/block"
I0823 09:22:35.030966 1309759 manager.go:238] "Device plugin connected" resourceName="pflex.io/block"
I0823 09:22:35.030339 1309759 server.go:144] "Got registration request from device plugin with resource" resourceName="github.com.ihcsim/crand"
I0823 09:22:35.030374 1309759 handler.go:95] "Registered client" name="github.com.ihcsim/crand"
I0823 09:22:35.030966 1309759 manager.go:238] "Device plugin connected" resourceName="github.com.ihcsim/crand"
# ...
I0823 09:04:14.473749 1352537 setters.go:329] "Updated capacity for device plugin" plugin="pflex.io/block" capacity=3
I0823 09:04:14.473749 1352537 setters.go:329] "Updated capacity for device plugin" plugin="github.com.ihcsim/crand" capacity=3
```

Deploy the provided busybox pod to the kubelet as a static pod:
Expand All @@ -57,30 +64,30 @@ Deploy the provided busybox pod to the kubelet as a static pod:
make deploy
```

The kubelet logs shows that a `pflex.io/block` device is allocated to the pod:
The kubelet logs shows that a `github.com.ihcsim/crand` device is allocated to the pod:

```sh
I0823 20:09:35.750554 1358062 kubelet.go:2407] "SyncLoop ADD" source="file" pods=["default/busybox-pflex-localhost"]
I0823 20:09:35.750607 1358062 manager.go:836] "Looking for needed resources" needed=1 resourceName="pflex.io/block"
I0823 20:09:35.750642 1358062 manager.go:576] "Found pre-allocated devices for resource on pod" resourceName="pflex.io/block" containerName="busybox" podUID="a9dc80a0d8f74cefb3be144bbfc1b898" devices=["pfl 2117 ex1"]
I0823 20:09:35.750554 1358062 kubelet.go:2407] "SyncLoop ADD" source="file" pods=["default/busybox-crand-localhost"]
I0823 20:09:35.750607 1358062 manager.go:836] "Looking for needed resources" needed=1 resourceName="github.com.ihcsim/crand"
I0823 20:09:35.750642 1358062 manager.go:576] "Found pre-allocated devices for resource on pod" resourceName="github.com.ihcsim/crand" containerName="busybox" podUID="a9dc80a0d8f74cefb3be144bbfc1b898" devices=["pfl 2117 ex1"]
# ...
I0823 20:09:58.293380 1358062 kubelet.go:1758] "SyncPod enter" pod="default/busybox-pflex-localhost" podUID="a9dc80a0d8f74cefb3be144bbfc1b898"
I0823 20:09:58.293433 1358062 kubelet_pods.go:1774] "Generating pod status" podIsTerminal=false pod="default/busybox-pflex-localhost"
I0823 20:09:58.293490 1358062 kubelet_pods.go:1787] "Got phase for pod" pod="default/busybox-pflex-localhost" oldPhase="Running" phase="Running"
I0823 20:09:58.293629 1358062 status_manager.go:691] "Ignoring same status for pod" pod="default/busybox-pflex-localhost" status={"phase":"Running","conditions":[{"type":"PodReadyToStartContainers","status":"True","lastProbeTime":null,"lastTransitionTime":"2024-08-24T03:09:35Z"},{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2024-08-24T03:09:35Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2024-08-24T03:09:35Z"},{"type":"ContainersReady","status":"True","lastProbeTime":null,"lastTransitionTime":"2024-08-24T03:09:35Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2024-08-24T03:09:35Z"}],"podIP":"172.16.16.4","podIPs":[{"ip":"172.16.16.4"}],"startTime":"2024-08-24T03:09:35Z","containerStatuses":[{"name":"busybox","state":{"running":{"startedAt":"2024-08-24T02:11:58Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"docker.io/library/busybox:latest","imageID":"docker.io/library/busybox@sha256:9ae97d36d26566ff84e8893c64a6dc4fe8ca6d1144bf5b87b2b85a32def253c7","containerID":"containerd://72ebbaf688f4454f47eec5991d36ec02fa82299e92ff6f849751c828f3c69ac0","started":true,"allocatedResourcesStatus":[{"name":"pflex.io/block","resources":[{"resourceID":"pflex1","health":"Healthy"}]}]}],"qosClass":"BestEffort"}
I0823 20:09:58.293380 1358062 kubelet.go:1758] "SyncPod enter" pod="default/busybox-crand-localhost" podUID="a9dc80a0d8f74cefb3be144bbfc1b898"
I0823 20:09:58.293433 1358062 kubelet_pods.go:1774] "Generating pod status" podIsTerminal=false pod="default/busybox-crand-localhost"
I0823 20:09:58.293490 1358062 kubelet_pods.go:1787] "Got phase for pod" pod="default/busybox-crand-localhost" oldPhase="Running" phase="Running"
I0823 20:09:58.293629 1358062 status_manager.go:691] "Ignoring same status for pod" pod="default/busybox-crand-localhost" status={"phase":"Running","conditions":[{"type":"PodReadyToStartContainers","status":"True","lastProbeTime":null,"lastTransitionTime":"2024-08-24T03:09:35Z"},{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2024-08-24T03:09:35Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2024-08-24T03:09:35Z"},{"type":"ContainersReady","status":"True","lastProbeTime":null,"lastTransitionTime":"2024-08-24T03:09:35Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2024-08-24T03:09:35Z"}],"podIP":"172.16.16.4","podIPs":[{"ip":"172.16.16.4"}],"startTime":"2024-08-24T03:09:35Z","containerStatuses":[{"name":"busybox","state":{"running":{"startedAt":"2024-08-24T02:11:58Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"docker.io/library/busybox:latest","imageID":"docker.io/library/busybox@sha256:9ae97d36d26566ff84e8893c64a6dc4fe8ca6d1144bf5b87b2b85a32def253c7","containerID":"containerd://72ebbaf688f4454f47eec5991d36ec02fa82299e92ff6f849751c828f3c69ac0","started":true,"allocatedResourcesStatus":[{"name":"github.com.ihcsim/crand","resources":[{"resourceID":"crand1","health":"Healthy"}]}]}],"qosClass":"BestEffort"}
```

With the `ResourceHealthStatus` feature gate enabled, the kubelet also reports
the `allocatedResourcesStatus` field in the pod status container status,
showing that the healthy device `pflex.io/block=pflex1` is allocated to the pod:
showing that the healthy device `github.com.ihcsim/crand=crand1` is allocated to the pod:

```json
"allocatedResourcesStatus": [
{
"name": "pflex.io/block",
"name": "github.com.ihcsim/crand",
"resources": [
{
"resourceID": "pflex1",
"resourceID": "crand1",
"health": "Healthy"
}
]
Expand All @@ -90,7 +97,7 @@ showing that the healthy device `pflex.io/block=pflex1` is allocated to the pod:

## Development

To build the plugin:
To build the plugins:

```sh
make build
Expand Down
32 changes: 16 additions & 16 deletions cdi/pflex.yaml → cdi/github.com.ihcsim.crand.yaml
Original file line number Diff line number Diff line change
@@ -1,42 +1,42 @@
cdiVersion: 0.8.0
kind: pflex.io/block
kind: github.com.ihcsim/crand
devices:
- name: pflex0
- name: crand0
containerEdits:
env:
- CDI_DEVICE_NAME=pflex0
- CDI_DEVICE_NAME=crand0
deviceNodes:
- path: /dev/sdg0
hostPath: /dev/pflex0
hostPath: /dev/crand0
type: block
major: 25
minor: 25
major :${DEVICE_MAJOR_VERSION}
minor: ${DEVICE_MINOR_VERSION}
permissions: "rw"
uid: 5000
gid: 5000
- name: pflex1
- name: crand1
containerEdits:
env:
- CDI_DEVICE_NAME=pflex1
- CDI_DEVICE_NAME=crand1
deviceNodes:
- path: /dev/sdg1
hostPath: /dev/pflex1
hostPath: /dev/crand1
type: block
major: 25
minor: 25
major :${DEVICE_MAJOR_VERSION}
minor: ${DEVICE_MINOR_VERSION}
permissions: "rw"
uid: 5000
gid: 5000
- name: pflex2
- name: crand2
containerEdits:
env:
- CDI_DEVICE_NAME=pflex2
- CDI_DEVICE_NAME=crand2
deviceNodes:
- path: /dev/sdg2
hostPath: /dev/pflex2
hostPath: /dev/crand2
type: block
major: 25
minor: 25
major :${DEVICE_MAJOR_VERSION}
minor: ${DEVICE_MINOR_VERSION}
permissions: "rw"
uid: 5000
gid: 5000
Expand Down
4 changes: 2 additions & 2 deletions main.go → cmd/crand/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ import (
"syscall"
"time"

"github.com/ihcsim/kubelet-plugin/pkg/plugins/pflex"
"github.com/ihcsim/kubelet-plugin/pkg/plugins/crand"
"github.com/rs/zerolog"
)

func main() {
var (
log = logger()
plugin = pflex.NewPlugin(log)
plugin = crand.NewPlugin(log)
)

ctx, cancel := context.WithCancel(context.Background())
Expand Down
4 changes: 2 additions & 2 deletions pkg/plugins/pflex/device.go → pkg/plugins/crand/device.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package pflex
package crand

import (
"context"
Expand Down Expand Up @@ -48,7 +48,7 @@ func (p *DevicePlugin) Allocate(ctx context.Context, r *v1beta1.AllocateRequest)
p.log.Debug().Msg("calling DevicePlugin.Allocate()")
resp := &v1beta1.AllocateResponse{}
for _, allocateRequest := range r.ContainerRequests {
// see cdi/pflex.yaml for pflex CDI configuration
// see cdi/crand.yaml for crand CDI configuration
car := &v1beta1.ContainerAllocateResponse{}
for _, id := range allocateRequest.DevicesIDs {
p.log.Info().Str("name", id).Msg("allocating CDI device")
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package pflex
package crand

import (
"os"
Expand Down Expand Up @@ -30,7 +30,7 @@ func (p *DevicePlugin) discoverDevices() (map[string]*Device, []*Device, error)
continue
}

if strings.Contains(entry.Name(), "pflex") {
if strings.Contains(entry.Name(), "crand") {
id := entry.Name()
device := &Device{
ID: id,
Expand Down
2 changes: 1 addition & 1 deletion pkg/plugins/pflex/grpc.go → pkg/plugins/crand/grpc.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package pflex
package crand

import (
"context"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package pflex
package crand

import "k8s.io/kubelet/pkg/apis/deviceplugin/v1beta1"

Expand Down
6 changes: 3 additions & 3 deletions pkg/plugins/pflex/plugin.go → pkg/plugins/crand/plugin.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package pflex
package crand

import (
"context"
Expand All @@ -14,8 +14,8 @@ import (
)

const (
socketName = "pflex.sock"
resourceName = "pflex.io/block"
socketName = "crand.sock"
resourceName = "github.com.ihcsim/crand"
)

var (
Expand Down

0 comments on commit 65a74df

Please sign in to comment.