Determine LMS Content Data Model Design

[ormsbee]

This is a discovery ticket that would end in multiple ADRs in this repo. The goal is to determine the basic data structures and relationships needed to store content data for the LMS with the following goals:

1. Handle all existing course use cases.
2. Handle v2 content libraries, and library resources shared across multiple courses, with different policies/overrides.
3. Handle potentially non-XBlock content.
4. Allow for fast publishing.
5. Allow for atomic publishing.
6. Allow for easy export.
7. Allow third party applications to build their own advanced data structures on top of it.
8. Allow for efficient querying.
9. Minimize the amount of wasted space caused by nearly identical versions caused by minor edits.

There will likely be some prototyping involved in this, as well as a lot of discussion.

These ADRs would include:

• Determine high level approach to splitting reusable content and learning context specific policy.
• Determine high level approach to versioning and incremental publishing.
• Determine leaf XBlock-level modeling.
• Determine unit-level modeling.
• Determine sequence-level modeling.

---

Open edX's ModuleStore has explicit versioning capabilities (though they're not really used from LMS most of the time). Blockstore has versioning deeply baked into the design, but we've generally avoided encoding multi-version support into the post-publish data stores we build during course publish (e.g. CourseOverview, Block Transformers/Course Blocks API, etc.)

I think we've come to a point where we really do need to cross that divide and start introducing content versioning concepts to the LMS more generally. Some motivating reasons:

1. It's difficult to preview changes before publishing if the LMS can only display the published version.
2. The design of content libraries assumes that multiple versions are available simutaneously.
3. Building all these separate stores of published data takes time, and parts may fail. When this happens, our course is in an inconsistent half-published state. For example, the rendered course content may be updated but the course outline generation may have failed so that nobody can see the new content. Ideally, we'd want to give these systems time to build up the data that they need, and then change everything atomically at once.
4. Other systems like scoring and state storage would benefit from being able to store the actual version it was created against.

Storage Scaling Issues

SplitMongo ModuleStore wasted a lot of space with old version data, eventually leading us to create a separate cleanup script for it. There were a number of reasons why disk usage got so bad with this system:

1. Structural data was stored inefficiently.
2. New versions were being published for tiny changes.
3. There was no cleanup.
4. Historical course data had very little use.

Every time there was even a small change (e.g. the title of a Unit), we ended up writing a document with all settings-scoped data for the course. This happened all the time in Studio, so that for every one version that is of interest to us for viewing or preview purposes, there would be dozens of almost-identical intermediate versions. We ended up in a place where the majority of course storage was wasted in this way.

There are a few ways we can make this much better:

• Isolate changes better
  Most content doesn't change very much from one version to the next, so we should break up the course into more granular pieces and track their changes individually.
• Support simple cleanup policies
  There will be intermediate versions that can be almost immediately discarded (like the last state of the Studio draft). We should have obvious cleanup facilities for getting rid of those as soon as they are not needed.
• Support a simplified data model for clients.
  Thinking about versions is hard, and unintuitive. We should have a set of primitives that help people model version-awareness into their content without having to overly complicate their data models.

Modeling Versioning in a way that scales (i.e. both features and users)

The following is a disorganized set of thoughts:

Entities (the names are bad, I'm just trying to get down the ideas)

• LearningObject
• LearningObjectVersion (with sub-types made with joined tables that represent things like Units, Blocks).
• Bundles (?) of LearningObjectVersions. I wanted to resist adding a separate layer here, but I realized that without this, we'd have to echo out all the (LearningObjectVersion/LearningContextVersion) entries with each new version, even if the libraries that a course is using isn't changing at all. Which would be really bad for encouraging library use. I really don't like the reuse of a Blockstore term and concept though.
• LearningContextVersion can contain multiple Bundles (e.g. a Course consists of versioned bundles for its "own" resources, as well as the Bundles of various other things like Libraries).
• LearningContextBranch enforces that at any given point, there is one live version per branch (e.g. "draft", "live")
• Some sort of registry where any app that has data related to a version has a chance to put the status of it (i.e. is it ready?) (How to handle the case where a process dies?)

I'm fiddling with some of these ideas in the learning_publishing app's models.py file at the moment.

Concern: How is it different than Blockstore? A: It's going to be much more relational data model that you hang other relational models (e.g. XBlock content, scheduling information) off of. Also, it's going to have zero intelligence about cycle detection or dependencies. It's also going to have explicit measures for cleanup of unused versions, which is not a thing in Blockstore.

[ormsbee]

Okay, I think I'm overloading LearningContext with concepts of content ownership and student state association, and I'm not sure how to reconcile that cleanly. I feel like there's something missing that I don't have the right vocabulary for.

Some rambling thoughts:

• The same library content consumed by the same student in different courses may or may not have shared state, depending on the desired use case.
• My implicit assumption with courses is that there is one course-controlled learning context, and then a bunch of libraries. But is it necessary that there's a central "course" learning context? If I had four sections of a "course" (or "pathway") with four differently versioned pieces of content that were strung together, isn't that fine? We just need a way to navigate it.

So it's almost like there's a PublishingContext (each library, the course, some part of a course, some part of a pathway, etc.), which is a group of content that can be versioned and published together. And then there's a LearningContextVersion, which has one or more PublishingContextVersions batched together.

So maybe Learning Contexts have:

• Publishing Contexts
• Navigation
• Student State

That could match up to a Course Run in the LMS. But is that over-complicating things to introduce a distinction between a publishing context and a learning one? If we want to shift how courses are authored to something more granular, it might be worthwhile.

[ormsbee]

FYI @kdmccormick, @feanil, @Carlos-Muniz, @arbrandes, @bradenmacdonald, @doctoryes, @saksham115, @jristau1984: Some thoughts I've been having lately about data modeling content in the LMS, with possible implications for content libraries v2 and unit composition in the LMS. It's not very well organized, and no action is required–there's a lot to unpack here that's beyond the current scope of BD-14. I just wanted to be transparent about where my head was at.

[ormsbee]

Thinking on this a bit more, this might be a viable way to model something like CCX, where the LearningContext becomes the CCX course key, and it has the PublishingContext for the base course. Policy things like dates would joins between LearningContexts and PublishingContexts.

This would also allow for how we model default behavior in libraries. The default values becomes the policy tables that join the library's LearningContext and the library's PublishingContext. The overrides that are specific to courses using the library become joins between the course's LearningContext and the library's PublishingContext.

[ormsbee]

That's actually pretty exciting to me, because it gives us a place to more cleanly apply policy-level separation in a way that decouples content authoring and administration–and does so in a way that unifies CCX and libraries override handling.

[arbrandes]

I'm not going to pretend I have the time to dive into this in the near future, but I certainly like it that you're giving it serious thought, Dave. :)

Nevertheless, here's a use case I want to bring up for consideration.

4. Historical course data had very little use.

This is because Studio never offered a wiki-like history (including grouping changes, who made them, etc) that a course team could navigate. I'm positive authors would love this: I know because I went as far as versioning OLX on git just so I could have all this stuff. I've seen this pop up in the forum on occasion, and if memory serves, MIT used to do this a lot.

I realize what you're discussing here is at a deeper level than exposing diffs on a frontend. But, again, it might be a use case worth considering as you model the data.

[ormsbee]

@arbrandes: Right. I guess it's more accurate to say that historical course data had very little use in the LMS. The main use there would be for fast rollback. Studio could benefit from it, though we didn't manage to prioritize any of that work at edX.

[ormsbee]

In this view of the world, we'd conceptually have:

LearningObject

• Example: ProblemBlock, Unit, Sequence
• Some addressable piece of content
• This model holds almost no data, it's just an immutable ID to build joins off of for more specific models.
• A snapshot of this is a LearningObjectVersion

PublishingContext

• Example: Content Library
• Content that is owned and versioned/released together.
• A snapshot of this is a PublishingContextVersion.
• PublishingContextVersions are M:M with LearningObjectVersions.

LearningContext

• Example: Course Run, Content Library
• A grouping to store content-related student state against: enrollments, grades, progress, etc.
• A grouping to store content-related policy against: schedule, default values
• A snapshot of this is a LearningContextVersion.

Relationship between LearningContexts and PublishingContexts

• A LearningContextVersion has one or more PublishingContextVersions.
• A LearningContextVersion may contain multiple versions of the same PublishingContext (e.g. two problems from two different versions of the same content library).

Next question: How is policy (e.g. defaults, scheduling, partitioning) stored against these data models?

[kdmccormick]

@ormsbee I found your latest comment (the directly above the one I'm typing) to be illuminating. Your distinction between LearningContext and PublishingContext is something that I've wondered about but hadn't been able to crystalize.

I agree with the overall direction you are taking in this app. I haven't taken a deep look a the other two apps yet but will soon.

I think we've come to a point where we really do need to cross that divide and start introducing content versioning concepts to the LMS more generally.

Yeah :/ I was really hoping we could hold onto "LMS doesn't do versions" but I admit I don't have any robust solutions for the pitfalls you listed.

---

Now some more in-the-weeds reactions/ramblings:

The same library content consumed by the same student in different courses may or may not have shared state, depending on the desired use case.

Hm, this sounds complicated. I had always thought that student state would remain isolated between LearningContexts, and by consequence, Content Libraries would carry no student state other than maybe author-preview state in Studio.

I guess I view Content Libraries as a more generalized type of LearningContext, one that should hold policy but not state. Would that function as a helpful or even sensical simplifying assumption? Would it lock Content Libraries out of an interesting use case?

Going another direction, what if policy itself were a LearningObject within the PublishingContext? At that point, could we say that a Content Library is a PublishingContext but not a LearningContext in the eyes of the LMS?

LearningObject, PublishingContext, LearningContext

I know you said the names aren't final. I'm still going to poke at them because it helps me think about the architecture, and also I can't help myself :)

• "Learning object" is an industry term that is unfortunately closer to our Sequences or Sections.
• In terms of "context", I think we get to use that term once ("learning context") before folks start getting confused about what "context" means in different contexts.
• How about "Versioned{Entity}" instead of "{Entity}Version"? "PublishingContextVersions" makes me think of multiple versions of the same PublishingContext, whereas "VersionedPublishingContexts" doesn't.
• Does the LearningContext model belong in this package? Seems like a very core model that isn't publishing-specific.

So, maybe:

• a ContentObject is an addressable piece of content, with each snapshot a VersionedContentObject.
• a ContentPackage is content that is owned and versioned/released together as a VersionedContentPacakge.
• a VersionedLearningContext joins ContentPackage(s) and student state.

keeping in mind that these are models are all namespaced under openedx_learning.publishing.

[ormsbee]

@kdmccormick: I'm going to chew on the first half of your reactions for a while, but w.r.t. the naming suggestions:

I know you said the names aren't final. I'm still going to poke at them because it helps me think about the architecture, and also I can't help myself :)

FWIW, in case it wasn't clear, my naming attempt was a desperate cry for help. 😛 Thank you for suggesting new ones.

"Learning object" is an industry term that is unfortunately closer to our Sequences or Sections.
a ContentObject is an addressable piece of content, with each snapshot a VersionedContentObject.

I had originally dabbled with the idea of intentionally using LO since some of them will be LOs in the general industry sense, but I like your framing of ContentObject better. I would still prefer ContentObjectVersion over VersionedContentObject because to me "VersionedContentObject" implies that there is a separate model with versions–i.e. I should be able to call versioned_content_obj.versions and get back a list of other models with version information. I think that ContentObjectVersion more clearly conveys that this is the data for a single version.

I also like ContentPackage as a way to not use the word "context" so much, and I do think it's simpler and clearer.

I'm in the middle of writing a model for encoding Units, and I'll try to do so in a way that incorporates some of this new language and addresses your other questions/suggestions above. Thank you for looking at this!

[ormsbee]

Implementing Unit Composition

Let's imagine what Unit Composition would look like it if was built on top of a data model like this. In this framing, all the ContentObject and ContentPackage stuff mentioned above would be part of the learning_publishing app, and most of this would be a layer above that as the learning_composition app. Given the data model I'm suggesting, even the LearningContext model might move up to this level.

---

Idea: ContentPackages represent raw pools of ContentObjects, and composition of those COs only exists within a LearningContext.

This sounds a bit weird, since moving a Block from one Unit to another should surely count as a change in content. But a few thoughts I had around this:

1. The LMS has a sort of "compiled" version of this content. Studio/Blockstore will be aware of the parent/child hierarchy and can provide when necessary.
2. We still have LearningContextVersion to encode when these things change.
3. We need to do composition at the LearningContext level anyway, to string together content from multiple ContentPackages (e.g. a Course Run referencing Library content). Doing it at the ContentPackage level would give us two semi-redundant ways to specify hierarchy.

---

These entities live at the LearningContext level.

Model that joins LearningContextVersion and ContentObjectVersion: "Block"?

• This creates the instance of this piece of content as it exists in this version of the context, and gives it an identifier.
• This is where UsageKeys would be mapped onto content.
• This would allow for multiple instances of the same ContentObjectVersion to be brought into a course with different usage keys.
• This needs a better name than LearningContextVersionContentObjectVersion 😛
• Maybe we just call this "Block"? That would be more familiar terminology for people used to working with edx-platform today.
• If usage keys are only established here, is it meaningful to have history in ContentPackages? Are ContentObjects just anonymous, hashed data buckets, like split modulestore definitions are today?

Unit

• we can either try to be smart about collapsing duplicates by having a value hash for this, or we can be really good about cleanup by tying them to a LearningContextVersion and deleting them when that LearningContextVersion is itself deleted.

UnitBlocks

• join between Unit and Block, with a column for ordering.

There's a weird sort of language flip with these names in that we're taking things that are versioned but talking about them

---

Edge Case: Nested blocks in Units

Units are flat lists. There are two scenarios I know of in which we have nesting within units right now:

1. Blocks that render different children to different users.
   These include the LibraryContentBlock, SplitTestBlock, and ConditionalBlock. I think we can add these inline as part of the list, along with their content group settings, so that we can efficiently select the subset that our student cares about.
2. The ProblemBuilder block that uses a container block and different problem-type sub-blocks.
   We might be able to just collapse this into a single Block as far as this layer of abstraction is concerned, and let the XBlock runtime deal with the nesting.

Right now, when you make a SplitTestBlock, with two things in it, you’re actually ending up with a hierarchy in ModuleStore that looks like this:

Unit (Vertical)
- Video
- HTML
- SplitTest
  - Vertical
    - Problem 1a
    - Problem 2a
  - Vertical
    - Problem 1b
    - Problem 2b

So the hierarchy looks like: Course -> Section -> Sequence -> Vertical -> SplitTest -> Vertical -> ProblemBlock. We've had a number of bugs along the lines of “nobody actually thinks this kind of structure is a thing on our platform", because they don't realize that extra level of hierarchy is allowed to exist.

But this kind of nesting for SplitTest only exists to generate what is a conceptually flat list–what we’re really trying to do is render Problem 1a+2a for one group and Problem 1b+2b for another group. The nesting is just because that’s how XBlocks work, and implementing them as children lets the SplitTestBlock control which one to show when rendering. But the mechanism SplitTestBlock uses underneath is user partitioning.

So Studio can keep this view of the world, but there’s no reason we need to encode it like that for the LMS.

The LMS can have a version of the data that just looks like:

Unit
- Video
- HTML
- Problem 1a (Partition Foo, Group A)
- Problem 2a (Partition Foo, Group A)
- Problem 1b (Partition Foo, Group B)
- Problem 2b (Partition Foo, Group B)

[ormsbee]

Okay, I've rambled a lot here, but I'm going to actually try to code this up in models and see how it holds together.

[ormsbee]

Side note: Explicit branches (draft vs. published) vs. having entirely different LearningContexts to encode drafts for previewing (or even sending explicit versions during preview)?

[ormsbee]

Latest wrinkle issue is: How important is it to be able to change the identifier for a piece of content over time? So in other words, can I change the UsageKey for something and still keep student state/history associated with it?

The data model implication if we wanted that kind of capability would be:

1. We'd have a sort of abstract representation of a piece of content across different versions of a LearningContext, using a UUID.
2. We'd associate customized identifiers (e.g. UsageKeys) at the level of an individual version of content.

It's a bit wasteful of space, since the identifiers are getting repeated with every revision, though it's not that big a deal–we'd use 8 bytes for the foreign key anyway, and identifiers would compress well if we use the right row type.

The bigger thing is how intuitive it is, and how awkward it would be to work with. Certain queries get potentially much more expensive, like "did this identifier ever exist in this course?". There's also potential for more confusion between when you're working with this abstract, version-less Block, and a concrete, version-of-a-Block.

[ormsbee]

Another weird wrinkle with having an abstract, versionless Block in this scenario is that it makes it harder to enforce uniqueness of UsageKey-style identifier within a LearningContextVersion (since the identifiers would be in the versioned-block table and the f-key to LearningContextVersion would be in the versionless-block table).