Add a FileSet class

[escowles]

The works extension includes a FileSet class that represents an original file and other files derived from it. The Hydra implementation has found this to be a very useful structure, and the key to separating Objects that represent component parts of other Objects from groupings of Files.

Should we add a FileSet class to the core ontology?

See #53 for preliminary discussion.

[ruebot]

👎 If it is required.

Further discussion in this thread: https://groups.google.com/forum/#!msg/pcdm/Ep1Cty2JDx4/Hdw8I5pbDwAJ

[whikloj]

I've said it before, but I think enforcing an additional resource for the (I have no numbers to back this up...but) small percentage of use cases that will need multiple file sets is not to the general benefit.

I understand the opinion around consistency for development, but I don't find this particularly persuasive. It seems that we can find some way to allow for the need of multiple FileSets without requiring them for all PCDM based resources.

[mjgiarlo]

@whikloj Can you say more about your use cases? What kinds of content do you need to support?

[ruebot]

@mjgiarlo at bare minimum, the content we support now in Islandora 1.x; which, I am near 100% sure, does not have any public use cases from the Islandora Community on the need for multiple FileSets as described. Should we need FileSets, we'd be perfectly happy using the FileSets extension.

👎 forcing FileSets

[acoburn]

If the PCDM vocabulary did not use predicates such as pcdm:hasMember, pcdm:hasFileSet and pcdm:hasFile (and encouraged people to use ore:aggregates), then the model would support both the simple and complex use case. (With or without FileSets).

[whikloj]

@mjgiarlo I think @ruebot and @acoburn have covered what I would say (and better than I could). Essentially, we could use pcdm:FileSets but for every object in our repository it would be a single lonely pcdm:FileSet and therefore extraneous data with no benefit. So could we not make it optional.

[tpendragon]

@acoburn

If the PCDM vocabulary did not use predicates such as pcdm:hasMember, pcdm:hasFileSet and pcdm:hasFile (and encouraged people to use ore:aggregates), then the model would support both the simple and complex use case. (With or without FileSets).

#63

[whikloj]

@mjgiarlo sorry our data is 90% newspapers. But the question is more about the possibility of multiple FileSets. We scan and, for better or worse, never scan again. So a second FileSet is extremely unlikely in most all of our use cases.

But perhaps I am misunderstanding the use cases that drove this need.

[tpendragon]

@whikloj @ruebot @awoods The majority of the objects in your repository don't store derivatives? The goal of FileSets is "here's a grouping of binaries and a description about why they're grouped."

If FileSets aren't required, then I assume you just never have multiple FileSets, and your object representing "Page 1" just has three files hooked to it and there's no reason to group the master with the derivatives because only one of those is a master and you can identify it?

So, is optional filesets something we can imagine an ingest routine for? "If it uses hasFileSet, go there for files, if it just hasFile, go there, there's only one grouping, if it has both...I dunno, count it as three filesets?"

[tpendragon]

Ah, forgot to mention, the reason multiple filesets became a thing was there were institutions who had multiple masters for a single "page".

[DiegoPino]

@tpendragon if i understand correctly, then the use case that motivated this new rdfs Class is having multiple Masters, with multiple derivatives each one for a same real world entity(page 1 of book 1 for example).

by the way Islandora(and @whikloj) do store Derivatives and a lot of them. I don't feel our use cases are that different, but our approaches are.

Grouping multiple binary resources together under a FileSet Class can be solved(without FileSet) by linking (via a specific predicate, not necessarily in PCDM space) a given Master binary to it's derivatives. You could even make another construct, via proxy's that point to the Binary you want to consider your canonical Master. Just one of many ways. How do you solve the "which of the many masters" problem with FileSet?

I also start to understand that some of this needs are based on programming paradigms, which is maybe the miss understanding we have here, and has probably something to do with how Hydra does data modelling hardcoding structure (based on rdf class to ruby class matching, just guessing) versus what we want to do (trying to extract structure, constraints and requirements from Ontologies and triple store)

If it uses hasFileSet, go there for files, if it just hasFile, go there, there's only one grouping, if it has both...I dunno, count it as three filesets?".

That is what i mean with hardcoded. The ontology itself (the semantic definition of the class + object properties allowed and their target classes) should give us that info instead of the code. I feel that is the idea of using Linked data. I'm not saying one approach is better than other. I'm saying we defer probably on how we deal with the logic over the structure.

[tpendragon]

based on rdf class to ruby class matching, just guessing

Nah, it's totally arbitrary. We store a model statement on the resource to map to the ruby class.

The ontology itself (the semantic definition of the class + object properties allowed and their target classes) should give us that info instead of the code.

I suppose my hypothesis is that this specification is only as good as the tools we can build which utilize it. Arbitrary ingest was just an example. Someone somewhere will have to build that logic, if we can encode that logic into the ontology via restraints that they the developer will have to follow, then great.

Grouping multiple binary resources together under a FileSet Class can be solved(without FileSet) by linking (via a specific predicate, not necessarily in PCDM space) a given Master binary to it's derivatives.

So I think in order to do this we'd have to redefine what a pcdm:File is, because right now it's a binary file. Bits don't have RDF statements - they can have nodes which describe them, but that's not part of the ontology now (except in the case of FileSets).

[tpendragon]

Ah, I may be letting Fedora leak into my previous argument. To be clear, you're proposing:

GET x
<x> <hasFile> <y>
<x> <hasFile> <z>
<y> <type> <master>
<z> <type> <derivative>
<z> <derivedFrom> <y>

Which I think solves that case, yeah.

[tpendragon]

We've always said Files only have access and technical metadata. What's derivedFrom?