add support for host devices

[proppy]

It would be nice if the container api payload had support for exposing host devices to the container (like docker run --device does).

The kubelet could pass it go-dockerclient once they add support for it (fsouza/go-dockerclient#241), or create container with the docker remote api by passing an addition member in the /create HostConfig payload:

{
    "PathOnHost": "/dev/deviceName",
    "PathInContainer": "/dev/deviceName",
    "CgroupPermissions": "mrw"
}

[proppy]

So it's now fixed on the go-dockerclient side fsouza/go-dockerclient@e4fcc92

[therc]

We need to expose GPUs to the containers. I can write the PR and (given my previous experience with a types.go change) rebase it over and over. What are the odds of it getting accepted? The first thought that comes to mind is how to secure it, as in @erictune's linked issue.

[therc]

I could place it behind a kubelet or apiserver flag, which is off by default.

[osterman]

I need this feature so we can run s3fs inside of k8s. Will have to use fleet for now :(

[praoreo]

@proppy , @fsouza
Hi,

What is the syntax to mention device information in yaml/json file? I tried giving the below in .json file, but got "found invalid field device for v1.PodSpec" error. I am using 1.3.6 kubernetes version.

                                "device": {
                                        "PathOnHost": "/dev"
                                },
                                "nodeSelector": {

[maci0]

I don't think it has been implemented yet. But it seems what @therc wants to do has.
https://github.com/kubernetes/kubernetes/blob/master/pkg/api/types.go has some nvidia stuff.

It's still missing support for other devices.
It used to work in docker with volume mounts, my guess it when they introduced --device they locked down the volume mounts using device cgroups
https://www.kernel.org/doc/Documentation/cgroup-v1/devices.txt

[farmdawgnation]

Hey what's the status here?

We need this as well. I haven't contributed to Kubernetes before, but it looks like a lightweight way to provide some (initial) support for this is to support a container annotation for it. That looks like it would get piped into the PodSandboxContext which I could in turn use to pass the requisite arguments into the host config for the Docker container creation.

[drekle]

I also believe that I need this. I will be trying to deploy a pod to a specific node which uses a napatech card or other networking devices.

[jbiel]

FWIW the following was working for me to pass through a sound card device ~3 months ago. Privileged mode was the key and according to the docs it looks like it should still work.

      containers:
      - name: foo
        ...
        volumeMounts:
        - mountPath: /dev/snd
          name: dev-snd
        securityContext:
          privileged: true
      volumes:
      - name: dev-snd
        hostPath:
          path: /dev/snd

[farmdawgnation]

Yeah, that'll work with privileged mode. The rub is we run code in a multi-tenant environment so that's a non-starter for our security requirements. Mounting devices using --device is safer.