Use k8s.io/apimachinery/pkg/util/sets package

[tmshort]

          Nit: I personally love using maps with empty value for sets, but I previously received feedback suggesting to use `sets` package (`k8s.io/apimachinery/pkg/util/sets`).

Originally posted by @m1kola in #846 (comment)

[itroyano]

/assign