Support read-only subset of Virtio-FS file system

[wkozaczuk]

Some resources:

• https://virtio-fs.gitlab.io/index.html#faq
• https://stefanha.github.io/virtio/virtio-fs.html#x1-41500011 (look for virtio-fs)
• https://virtio-fs.gitlab.io/howto-qemu.html
• https://lists.oasis-open.org/archives/virtio-dev/201812/msg00073.html

This would require implementing a new virtio device and a new filesystem. The challenge is that Linux implementation (https://gitlab.com/virtio-fs/linux/blob/virtio-fs/fs/fuse/virtio_fs.c and https://gitlab.com/virtio-fs/linux/blob/virtio-fs/fs/fuse/inode.c) uses FUSE layer which is OSv does not implement and never will probably. For starters, we could limit the initial implementation to a read-only part.

[wkozaczuk]

Even though the commit bae4381 added initial VirtFS implementation, there still many outstanding elements missing or needing improvements:

• Implement virtiofs_readdir() - implemented with a16e5c9
• Implement virtiofs_statfs()
• Make implementation of virtiofs_read() more efficient it terms of exists by adding some kind of caching logic possibly like ROFS has
• Implement DAX Windows (see https://virtio-fs.gitlab.io/design.html) using FUSE_SETUPMAPPING/FUSE_REMOVEMAPPING commands - implemented (7a2eaf2 and other commits)
• Allow booting OSv with VirtioFS mount as root (just like ZFS, ROFS or RAMS can now) - implemented with f398a3b

[foxeng]

Updating the list of pending items, feature-wise:

• mmap support. See

  osv/fs/virtiofs/virtiofs_vnops.cc

  Line 405 in 5a2353c

  virtiofs_arc, /* arc */ //TODO: Implement to allow memory re-use when

• statfs. See

  osv/fs/virtiofs/virtiofs_vfsops.cc

  Line 172 in 5a2353c

  static int virtiofs_statfs(struct mount* mp, struct statfs* statp)

Gap-fillings, some improvements etc. are on the way. Unfortunately, due to current lack of time, this will be a somewhat long way (read: couple of months?).

[wkozaczuk]

@foxeng I think I will close this story and create new ones for each pending item. Also regarding the mmap support, I think it would be nice to implement its to directly map host pages as Linux is doing it. Please see this Linux patch - https://lore.kernel.org/lkml/20200819221956.845195-1-vgoyal@redhat.com/ and these comments:

"This patch series adds DAX support to virtiofs filesystem. This allows
bypassing guest page cache and allows mapping host page cache directly
in guest address space.

When a page of file is needed, guest sends a request to map that page
(in host page cache) in qemu address space. Inside guest this is
a physical memory range controlled by virtiofs device. And guest
directly maps this physical address range using DAX and hence gets
access to file data on host.

This can speed up things considerably in many situations. Also this
can result in substantial memory savings as file data does not have
to be copied in guest and it is directly accessed from host page
cache."

[foxeng]

Totally agreed on replacing this with more fine-grained and easier to address issues.

Regarding mmap, the comment from the linux patch above is actually the description of how the DAX window works in the context of virtio-fs (and linux specifically, when referring to the guest page cache). Right now I am lacking the specifics around mmap in OSv (it's been a number of months since I looked into it and didn't end up implementing it back then), so I am not sure how this applies.

It might be of value, mainly for future reference, to leave a short overview of our current status here. Assuming a virtio-fs device supporting the DAX window:

• We try to satisfy all reads using the window. If for any reason that fails, we fall back to issuing a FUSE_READ to the host. See

  osv/fs/virtiofs/virtiofs_vnops.cc

  Line 258 in ef62b24

  if (dax_mgr) {

• The first thing the DAX window manager (fs/virtiofs/virtiofs_dax.{hh,cc}) does is to check if the requested data are already in the window. If so ("cache hit"), it copies them back directly, satisfying the request. See

  osv/fs/virtiofs/virtiofs_dax.cc

  Line 34 in ef62b24

  // Requested data (at least some initial) is already mapped

• If the data are not found in the window ("cache miss") and the manager decides to bring them in, it issues a FUSE_SETUPMAPPING request, the data are mapped in the window and then copied back like above. See

  osv/fs/virtiofs/virtiofs_dax.cc

  Line 43 in ef62b24

  // Map file

The ideal scenario for mmap IMHO would be to provide access directly to the DAX window, thus avoiding the (intra-OSv) copy that's part of read's semantics.