Grouping

[Dsan10s]

Here are some preliminary notes to start with, that helped me formulate a better idea of what we have and some important questions

• mv-group essentially seems like mv-sort, with the only differences being:
  a. Use case: mv-group will be used just like mv-sort, but more likely with
  properties that are expected to have a lot of ties
  b. Display: In addition to the newly "sorted" data, we need to add a header
  element in between each of the breaks (places where the property value
  changes)
  - Display friendly name should be the value of the property being input to
  mv-group

Ex:

<div mv-multiple="restaurants" mv-sort="name" mv-group="type">
	<div property="name"></div>
	<div property="type"></div>
</div>

• We expect mv-group to take in a property whos value is repeated among multiple
  elements in the collection. E.g. a restaurant's type could be something like
  "Italian", "Fast food", "Thai", etc. Probably good to promote the setting of
  this property through a dropdown.

• How would the display of an mv-group header look?

  • What kind of styling should it get?
    1. Copy over the style of each mv-multiple's parent element, with some
       additional style that distinguishes the header element as a header and not
       another collection element
    2. Create a default styling applied to each mv-group header

[Dsan10s]

With respect to grouping by multiple properties, I can see how a nested array or nested object approach could both work, but with respect to displaying these group header elements, I'm not quite sure how that would work? We could use indentation to denote children, but in the extreme case where there is a great deal of nesting, what should happen there? I expect the usual use case to be between 1-4 levels of nesting though.

[LeaVerou]

For context, this is @karger’s email comment:

We've talked about easier and harder use cases for grouping. A relatively easy one is where the author has a fixed grouping in mind but wants it to be maintained as items are inserted and deleted. Idea: should there be an GROUP analogue of the SORT function that can be used for this? The sort function takes an array and a list of properties (expressions?) and returns an array sorted be these properties. What should the GROUP function return? One possibility would be an array of arrays of arrays etc. Even better might be a set of nested objects (dictionaries) where the keys are the values of the grouping attribute and the values are the (recursive) groupings of the objects that have that value of the attribute. For example, if I have an array X of person objects that have city, state, country, then

group(X,{country,state,city})--->
{"United States": {
"Massachusetts": {
"Cambridge": [{David Object}, {Danny object}]
"Boston": [{Lea Object}]
....

As with sorting, we would want the grouping algorithm to be "stable" in the sense that it doesn't change the order of items that end up in the same group----so those can be sorted on some other property.

A question I can't answer is, could I write a mavo that would render such a grouped object ? I'm not sure we can, since I don't know how to reference the keys in the grouping object I described above.

[LeaVerou]

@karger The thing is, since grouping augments the template, it cannot be done with the formula language.

[karger]

I'm not sure I follow your arg that grouping augments the template. What I discuss above is grouping as a data action, with no reference to the template. If you want to express how to present the groups you may need some extra template syntax (although we discussed the mvp doing the template augmentation automatically) but that doesn't seem to constrain the data manipulation.

[LeaVerou]

Sure, and I guess you could hardcode the group headings in a template.
But yes, as we discussed it would be more useful to start from the attribute.

[karger]

what do you mean by start from the attribute?

[LeaVerou]

Start by implementing an mv-groupby attribute, just like the mv-sort attribute.
Unless Danny prefers to first experiment with a groupby() function, which would be fine by me.

[LeaVerou]

Thought I just had:

Often one wants to group manually. E.g. recipe ingredients are often a flat list OR grouped (e.g. base, cream, topping). I wonder if we should support this, and if so, how. Also in that case the grouping is semantic, not presentational, i.e. it needs to be reflected in the stored data.

Just something to think about.

[Dsan10s]

Start by implementing an mv-groupby attribute, just like the mv-sort attribute.
Unless Danny prefers to first experiment with a groupby() function, which would be fine by me.

Well, the functionality of mv-groupby is going to have some underlying logic, whether it be through a groupby() function, or group analogue of the sort function that presents the data in it's grouped format. Either way, to get started on the attribute I'll probably have to do something like one of those two ideas.

If you both agree maybe a good next step is figuring out which is more appropriate?

[Dsan10s]

I think I kind of agree with @karger that the nested dictionary approach is probably better than nested lists, but definitely open to hearing any other opinions.

If we do nested dicts though, I think it should be a separate Mavo.Function, instead of a group construct of the sort Mavo.Function, since the return type is vastly different.

[LeaVerou]

I think I kind of agree with @karger that the nested dictionary approach is probably better than nested lists

Who suggested nested lists?!

If we do nested dicts though, I think it should be a separate Mavo.Function, instead of a group construct of the sort Mavo.Function, since the return type is vastly different.

Oh yes, definitely. But it should probably call sort() first, then operate on the sorted data.

Regarding the markup, I envision something like the following:

1. Look for .mv-group-heading element before the collection
2. If it doesn’t exist, create your own, based on the following algorithm:
   1. If mv-multiple is on a <tr>, create a <thead> above it.
   2. If mv-multiple is on an <option> create an <optgroup>
   3. Otherwise, find the biggest heading level above the list and create a <h(n+1)> element. Remember to take sectioning elements and aria-level into account.

Add mv-group-heading and mv-ui classes to your newly created element.

Other caveats:

• If grouping by a number or a date, you probably want to group into ranges (unless there are very few distinct values. You can use Mavo.Functions.unique() for that. That function will actually be useful to you in general for grouping.
• Sometimes when grouping by a string, we actually want to group by start letter (e.g. think of contact managers). I wonder what heuristic would be good for declaring this is desirable. Some thoughts on that:
  • If almost every value has 1 item, then grouping by value is not super useful.
  • If there are way too many groups.

[Dsan10s]

Who suggested nested lists?!

From email:

What should the GROUP function return? One possibility would be an array of arrays of arrays etc.

Oh yes, definitely. But it should probably call sort() first, then operate on the sorted data.

Do you mean that we want the headers to be sorted? If we have the dictionary structure we discussed, we need another way to retain this order, but we can do this in the function that operates on the return value of the groupBy function.

[karger]

If grouping is stable (like sorting) then you can sort before you group.

…

On 11/23/2017 7:26 PM, Danny Sanchez wrote: Who suggested nested lists?! From email: What should the GROUP function return? One possibility would be an array of arrays of arrays etc. Oh yes, definitely. But it should probably call sort() first, then operate on the sorted data. Do you mean that we want the headers to be sorted? If we have the dictionary structure we discussed, we need another way to retain this order, but we can do this in the function that operates on the return value of the |groupBy| function that returns this nested dictionary. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABFpXg6AovkFEDWCX3BBGvOaa9Zi1r1Qks5s5g1NgaJpZM4QVO-V>.

[Dsan10s]

I'm still unsure what you want to be sorted though. If you mean within each group, that the displayed items are in the same order as the original collection, then that is currently achieved.

…

On Nov 23, 2017 11:06 PM, "David Karger" ***@***.***> wrote: If grouping is stable (like sorting) then you can sort before you group. On 11/23/2017 7:26 PM, Danny Sanchez wrote: > > Who suggested nested lists?! > > From email: > > What should the GROUP function return? One possibility would be an > array of arrays of arrays etc. > > Oh yes, definitely. But it should probably call sort() first, then > operate on the sorted data. > > Do you mean that we want the headers to be sorted? If we have the > dictionary structure we discussed, we need another way to retain this > order, but we can do this in the function that operates on the return > value of the |groupBy| function that returns this nested dictionary. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#3 (comment)>, or > mute the thread > <https://github.com/notifications/unsubscribe-auth/ ABFpXg6AovkFEDWCX3BBGvOaa9Zi1r1Qks5s5g1NgaJpZM4QVO-V>. > — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADdHTOSdD0qzOr9DlOPSoCwbbr6tXbYCks5s5kDAgaJpZM4QVO-V> .

[karger]

yes, that was my point---preserving the original sort order is the easy way to permit sorting within groups.

…

On 11/24/2017 12:15 PM, Danny Sanchez wrote: I'm still unsure what you want to be sorted though. If you mean within each group, that the displayed items are in the same order as the original collection, then that is currently achieved. On Nov 23, 2017 11:06 PM, "David Karger" ***@***.***> wrote: > If grouping is stable (like sorting) then you can sort before you group. > > > On 11/23/2017 7:26 PM, Danny Sanchez wrote: > > > > Who suggested nested lists?! > > > > From email: > > > > What should the GROUP function return? One possibility would be an > > array of arrays of arrays etc. > > > > Oh yes, definitely. But it should probably call sort() first, then > > operate on the sorted data. > > > > Do you mean that we want the headers to be sorted? If we have the > > dictionary structure we discussed, we need another way to retain this > > order, but we can do this in the function that operates on the return > > value of the |groupBy| function that returns this nested dictionary. > > > > — > > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub > > <#3 (comment)>, or > > mute the thread > > <https://github.com/notifications/unsubscribe-auth/ > ABFpXg6AovkFEDWCX3BBGvOaa9Zi1r1Qks5s5g1NgaJpZM4QVO-V>. > > > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#3 (comment)>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/ADdHTOSdD0qzOr9DlOPSoCwbbr6tXbYCks5s5kDAgaJpZM4QVO-V> > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABFpXmaXHDv62_FY3-3fSJKVC6KO4Zjsks5s5vmZgaJpZM4QVO-V>.