Definition list definitions treat blank lines differently from regular lists

[notriddle]

I'm trying to rationalize the behavior from pulldown-cmark/pulldown-cmark#979, and having trouble seeing it.

HTML In a Definition List
:
  <div>
    <span>One space</span>
  </div>


More HTML In a Definition List
:

  <div>
    <span>Two Spaces</span>
  </div>

The first one does what I would expect: the <div> is inside the definition.

The second one, however, is confusing. commonmark-hs puts the <div> inside the definition, even though itemized lists don't. This inconsistency between definition lists and itemized lists seems like a bug:

- HTML In an Itemized List
-
  <div>
    <span>One space</span>
  </div>
- HTML *Outside* an Itemized List
-

  <div>
    <span>Two Spaces</span>
  </div>

Details

• HTML In an Itemized List
• One space
• HTML Outside an Itemized List

Two Spaces

[jgm]

Yes, it might make sense to make these consistent.

The current behavior is consistent with the definition list spec in commonmark-extensions/test/definition_lists.md.

So, the question arises why the behavior of itemized lists is the way it is?

Oddly, the restriction "A list item can begin with at most one blank line" occurs only in the text introducing one of the examples, and not in the official statement of what counts as a list item. That is something that should be improved in the spec, assuming this feature is kept. Background: commonmark/commonmark-spec#332 [Edit: looking at that issue, I guess the idea was that "a single blank line" excludes the case where it starts with two blank lines.]

[notriddle]

Removing that restriction would make it simpler to specify how a list item is made.

But it would make it more complex to decide whether a list item is loose or tight, so I'm not sure the commonmark spec would be any simpler overall.

[notriddle]

@jgm Is there any blocker for resolving this issue one way or the other?

[jgm]

The blocker is deciding what the best course of action is.

[notriddle]

Is there any way to break this big question down into smaller questions?

• I tried the obvious thing and ran the introductory example code through babelmark3. Only two implementations actually wrote a <dl>: pandoc and multimarkdown. Multimarkdown put the "Two spaces" one outside the list, while pandoc put it inside (presumably, you already knew that).

If you're thinking about whether the one-blank-line restriction can be removed from standard lists, I guess there are a couple of ways to test it:

• The babelmark3 test for the itemized list shows that most implementations that claim to be commonmark agree (except for intellij-markdown), a surprisingly large number of implementations think there's two second-level headings (including, for what it's worth, multimarkdown), and the only ones that actually put both <div>s inside the list are parsedown, intellij-markdown, and pandoc.

• I could try, to gather some data that could help us make this decision, a Crater experiment: a test run with a patched rustdoc run across all the Rust doc comments in crates.io to see how many of them are affected by this change. I've done this twice now: a couple years ago to test an upgrade to pulldown-cmark to see if the bug fixes broke too many existing docs, and about a week ago to test our in-progress implementation of LaTeX-style math. It takes awhile, both to run and to label the data afterward, but the results are often enlightening.

  But I'd rather not waste time on it if the answer's going to be "Rust docs are not necessarily representative of CommonMark usage in general, so the results don't matter."

[jgm]

I suspect these cases will be relatively rare. The main issue for me is that I can't remember why in the original spec I limited list items to one blank line at the start. There may have been a good reason for that. Looking at the code might help jog my memory, but I don't have time at the moment.

It's also possible (likely?) that this is a hangover from an earlier stage of the spec in which two blank lines terminated a list.