Lists in Markdown to Word conversions should use "List Paragraph" style instead of "Normal"

[jdanyow]

pandoc 3.7.0.2

Command:

pandoc -s test.md --reference-doc=reference.docx -o test.docx

Source files

• test.md
• reference.docx (just a new blank word doc created using the latest microsoft word)

The list items are using the "Normal" style instead of the expected "List Paragraph" style, which results in extra vertical spacing.

Here is a screenshot of the generated file with the Styles pane visible:

Here is a screenshot where the text was manually entered in Word:

[rnwst]

If your only issue is the paragraph spacing of list items, you could modify the Normal style in your reference-doc.docx and decrease the before and after paragraph spacing values. However, this won't just affect list items, but also anything else where the Normal style is used.

However, I think there would be merit in being able to style list paragraphs independently, so it's probably a good idea to add a List Paragraph style to the reference-doc and to the DOCX Writer (and to update the relevant section of the Manual with the additional style).

[SeanESCA]

Looking at the XML of the DOCX output, the Compact style from the default reference document is used for 'compact' lists, like those in your test.md file. I'm not sure if that style is used for anything else, though. If it isn't, I think the easiest solution would be to start from the default reference document instead of a blank document, and modify the Compact style to your needs.

I also think the documentation could be clearer on what that style is used for so we don't need to dive into the XML to figure that out.

[jgm]

Compact style is used for Plain elements (as opposed to Para). These occur in tight lists but also in tables.

[SeanESCA]

In that case, I agree with @rnwst that we should be able to style lists and tables separately. I can see where the styles are being set in OpenXML.hs and Table.hs, and styles.xml for the reference doc. It looks like a very small change: just need to add a List Paragraph style that matches Compact to maintain the default settings, and set tight lists to use the new style instead. If no work has been started, I can open a PR for this.

[jgm]

just need to add a List Paragraph style that matches Compact to maintain the default settings, and set tight lists to use the new style instead. If no work has been started, I can open a PR for this.

What about loose lists, then?

[SeanESCA]

From the DOCX output, a loose list looks essentially like paragraphs with bullets, unless there's more to it. The style for loose lists isn't set in the XML, so it defaults to the Normal style. The AST showed that each item in a loose list is a Para object, instead of Plain as it would be in tight lists. As such, I think it would be fine to keep loose lists as is. What do you think? @jgm

[jgm]

That sounds reasonable to me. Of course, there may be things I'm not thinking of that would become apparent in trying to implement it.

[SeanESCA]

I would like to ensure that reference docs using only Compact would still work. I'm trying to ride on how custom styles are added, since I am not sure if there is a better solution. Out of curiosity, is there a reason why missing styles in the reference doc are not added to the final output as a fallback? Like in this case with Compact being missing from the given reference doc.

[jgm]

is there a reason why missing styles in the reference doc are not added to the final output as a fallback? Like in this case with Compact being missing from the given reference doc.

That might be a good thing to do. But there might be complications, e.g., some of the missing styles might refer to other styles that are also missing. One also has to ensure that there are no name or id collisions.

If we didn't want to do that, another way to ensure compatibility with older reference docx would be to check the style in the reference docx, and if it does not have First Paragraph, just use Compact as before. The name of the relevant style could be stored in the writer state.

Not sure which is the best approach here.

[SeanESCA]

If I understand the writer correctly, it collects custom paragraph styles explicitly used in fenced divs and checks whether each custom style has been defined in the reference doc. If not, the writer defines the missing style based on the Body Text style. At least for the Compact style, I was thinking of appending Compact List and Compact Table (the new separated styles) to this list of custom paragraph styles from the input file. If any of them are missing, the writer will add them, but they will be based on Compact instead of Body Text. This would ensure the new styles work fine with older reference docs if they have Compact. If an older reference doc didn't, I don't think we need to worry about this case since Compact wouldn't be used anyway.

I agree that this first approach would be too messy for the built-in styles like headings that depend on different styles between Word and Pandoc.

For the second approach, by storing in the writer state, do you mean the writer options? I could try both, but right now, I feel more comfortable with the first approach since I have a clearer idea of how to implement it, and I think it would be better to get consistent output in the document.xml regardless of the styles available.

[jgm]

I think the solution you outlined sounds reasonable. (For the second approach, I did mean writer state - this is an internal, docx writer specific state, as opposed to WriterOptions.)

To be clear, is this your idea? Loose lists -> List Paragraph. Tight lists -> Compact List. Tables where we currently have Compact -> Compact Table. Is Compact Table really necessary? Could Compact be used here?

[SeanESCA]

I'm thinking:

• Loose lists -> Normal, since they're essentially normal paragraphs with bullets
• Tables -> Compact, so no change
• Tight lists -> Compact List

For old reference docs, Compact List will be based on Compact. For the new reference docs, they will be independent styles.

I was originally thinking that Compact List would be based on Compact in all cases. As such, you'd need that additional Compact Table style to make the list and table styles independent. This overcomplicates things.

[jgm]

OK, that's what I had thought before. I think this simpler plan is good.

[SeanESCA]

Sorry, now I don't think we should add the new style. I did not consider how to handle which style should take priority when there are lists in tables, and this would vary. I think it would be better to use a filter to set the styles if they want that level of customisation. I am also not sure why I thought you meant all text in tables is styled with Compact when you can have Para objects in tables as well, which makes the new style feel more pointless.

Compact style is used for Plain elements (as opposed to Para). These occur in tight lists but also in tables.

I did not notice earlier that loose lists currently use Normal instead of Body Text, so I will go ahead with fixing this.