TestGrid Open-Sourcing Planning

[michelle192837]

ETA (7/25/19): We've got some updates! Check it below

tl;dr: If you're interested in TestGrid being open-sourced, add your comments, support, and use cases here!

ETA: While TestGrid open-source is in the works, you can add your results to testgrid.k8s.io by following the normal process. Your results will need to conform to expected results for now; I'll update when more possibilities are available.

I'm pulling together a roadmap for TestGrid development in 2019! (Since I'll be on vacation in the latter half of December, I'm drafting it this week, and the final one will get posted in January). There's been scattered support, comments, and bugs for TestGrid open-sourcing, but I want to collect all that in one bug for convenience's sake.

So! Comments! If you have details on what you need specifically (i.e. what parts are most important to prioritize, like being able to stand up a separate frontend, upload results, run your own updaters, etc.), or on what you'd be using TestGrid for, note that too. The more comments the better!

Related issues: #3324, #3323

/kind feature
/area testgrid

[BenTheElder]

cc @munnerz @stevekuznetsov

[stevekuznetsov]

/cc @smarterclayton

[stevekuznetsov]

Talked with @michelle192837 in person today - would be great to have a quick overview of the architecture of TestGrid and current progress.

From a naive opinion it seems like we could process our test results and upload our own proto for a third party front end to use, that would need minimal effort to get us to dashboards.

[donaldguy]

👋 I work on AKS; (likely @juan-lee and) I would be interested in using it very much like upstream for our staging edge

Btw, I am also happy to talk about getting our public face on "real" test-grid

I suspect @jackfrancis and co might be similarly interested vis-a-vis Azure/aks-engine

[rdrgmnzs]

I would be in using testgrid internally, currently to be able to visualize the current integration tests I have setup to run against my clusters. The alerting portion if tests fail consecutively is also something I'm looking forward to having.

[neolit123]

would testgrid go in a separate repo?

[michelle192837]

Thanks for the comments so far all! Keep adding if you've got them. ^^

@neolit123 I think ideally, since conceptually it's not k8s-specific? No idea on if/when it would be in a separate repo though.

[neolit123]

i think i see it as a kubernetes/testgrid repo.
my concern here is to avoid the potential bump in volume of test-infra, since it's already one of the higher volume repos under the k org, after k/k and k/website.

[michelle192837]

Thanks for comments so far all! (Feel free to continue adding to this, or encourage others to).

Just got back from vacation, so I'll be pulling together the plan for early 2019 and will publish that publicly when it's more finalized.

[ffledgling]

Adding my use case here after discussion with @michelle192837/@amerai I have a bunch of clusters that running internally and am starting to run the kubernetes e2e conformance tests against them periodically. There are also internal tests added into the same framework to test for custom stuff. I'd like very much to use testgrid as the overview dashboard for these periodic tests and as an indicator of overall cluster health and compliance. A self-hostable version of testgrid would be very very appreciated (even if it initially requires setting up a private GCP buckets etc).

I'd be happy to act as beta/alpha tester for on-perm/self-hosted testgrid in case that helps!

[jstrachan]

would love to use testgrid too (in our case to vsualise all the different system tests running on different kubernetes clusters & configs in Jenkins X).

Even before the code becomes OSS, if there are public usable docker images for the testgrid containers I’d be happy to try create helm charts to make them easy to use on k8s (though I may need a hand figuring out how to wire it all together)

[michelle192837]

Just a quick update; I don't have much externally for a 2019 roadmap atm (been finishing up some work from last year, and going on vacation for 3 weeks in Feb), but I promise I'm still working on this and do intend to do some work in GitHub when I'm back in March.

Thanks again! ^^

(In the meanwhile, if someone's particularly itching to enhance TestGrid, feel free to take a look at #10701 )