Test build with Travis CI

[nyh]

Travis-CI offers free CI (builds, tests, etc.) services to open-source projects. I opened an OSv page there:

https://travis-ci.org/cloudius-systems/osv

By adding a properly-formatted travis.yml project (see explanation in https://docs.travis-ci.com/user/tutorial/) we could easily ask Travis to automatically try to build OSv after every commit. Even more interestingly, we can ask it to build on several different Linux distributions, which can help us make sure that OSv still builds properly on older distros which the developers no longer have or test.

In the long run, it would be nice for these automated build to also do "make check" (i.e., run the tests), but these will be very slow because Travis doesn't support nested virtualization so everything will be run in qemu's emulator. But OSv's build itself should be quick it enough (it only minimally uses qemu to build the image - this will be slower than usual but probably not prohibitively slow).

[nyh]

Another option to consider is circleci - https://circleci.com/, which allows to use docker. In Seastar we switched from travis to circleci: scylladb/seastar@34b8eda.

[PabstMatthew]

Hi,

A couple of peers and I from the University of Texas at Austin are interested in contributing to OSv, and we thought a good place to start would be implementing some basic CI infrastructure (#1044). A couple of us have experience with CI/DevOps activities, so it shouldn't be too difficult.

We have a few concerns though:

• The Dockerfile is missing make, so we'd have to add that dependency to run the build scripts
• When we try to run scripts/build check with the latest commit (cccc7d4035f8c81e263340df1ba4c5db305714a1), tracing_smoke_test fails on the tcpdump section
• The testing script fails immediately if any test fails, rather than collecting the output of all tests

Discussion at: (https://groups.google.com/d/msg/osv-dev/dqca0wjU2Lo/qApxTAmFBQAJ)

[wkozaczuk]

Ideally, we would want to make a new process repeatable. By that I mean we should have an ability to create test images/packages independently from running them. To achieve we might have 2 docker files - one to build and another one to run the tests.

Also, the test results should be collected and published somewhere. To achieve this we could use capstan. The test packages could be uploaded to github release assets unless there is a better idea.

Recently I have pushed the number of shell and python scripts aiming to help test a variety of workloads/apps on OSv. These also illustrate how capstan can be used above. Potentially we could reuse some of these scripts from build/run docker file. For more information about automated testing framework please read this.

[wkozaczuk]

Completed with 7305d0f