Wrappers for standard datasets

[theotherphil]

Using any dataset requires some initial mucking about to parse whatever custom formats the raw data and metadata (e.g. truth files) are in and get it into a suitable shape for processing. It would be great if this library (or an associated one) had a set of scripts/tools for working with a few of the most popular datasets.

[theotherphil]

I'm happy to help with this, but don't have that much spare time at the moment, so I'll do the cross validation issue first and see how it goes.

[AtheMathmo]

Thanks for opening this!

To be sure I totally understand - you mean that it would be nice to more easily use some common data sets. If so there are a few ways I think we can handle this:

1. Some data sets within this repository - and a small crate to load them (separate from rusty-machine).
2. Use a separate crate to handle this entirely - I know of this for example.
3. Place within rusty-machine behind a feature gate.

For me I think the second is the probably the best - though I'm not sure which data structure we'd use to store the data.

Just in case you're talking about some more general data handling - I tried my hand at it a while ago without much success. It's pretty tricky and I don't want something subpar inside rusty-machine. I think for now it's best to let users handle the more general data side on their own.

I'll keep this issue open as I think the idea is valuable. Making rusty-machine more accessible is good in my books and this helps with that for sure.

[theotherphil]

I wasn't aware of the library you linked in 2, thanks. That's the sort of thing I had in mind. I imagined something presenting a database-y view of locally downloaded datasets. There'd be a list of dataset metadata (including where the data's available from), and a local dataset repository supporting adding (downloading and converting to a standard format in a local directory) datasets from the list, removing them (deleting local caches), and loading the data (both the raw inputs, and e.g. ground truths) into a standard form. So that the user can acquire a dataset, load it and test some models, in a few lines.

[sinhrks]

I'm +1 to option 3 ATM.