The range of a list item depends on whether it is followed by an empty line

[mikemc]

I've been doing some neorg module development (stemming from wanting to solve nvim-neorg/neorg#938), and came across an inconsistency in the range of list items in the tree-sitter tree I think might trip up others. Possibly this is an expected feature, in which case I can code around it, but I wanted to post here in case a change in the tree-sitter parser might be the best ultimate fix. (I'm new to Neorg and to Neovim plugin development, I apologize for misunderstandings I might have in posting this here.)

I'll talk about unordered lists, but the same behavior applies to ordered lists.

Summary of the issue: The range of a node with type unordered_list1 (corresponding to an unordered list item) is 1 line shorter if the list item is followed by an empty line than if the list item is followed by a line with any content or no lines (i.e. end of buffer). For list items that only span a single line each, the situation for list items followed by another list item, a heading, a quote, or end of file are that the start and end rows of the range have a difference of 1; but for a list item followed by an empty line, the range start and end rows are the same. An analogous situation occurs with list items spanning multiple lines; the range spans multiple rows, but spans one less row when the item is followed by an empty line).

This behavior seems undesirable because it requires code that operate on list items to act differently depending on whether the item is followed by an empty line.

I'll illustrate with some small files.

For this file,

* heading
  - item 1
  - item 2

:InspectTree returns the following tree:

(heading1) ; [1:1 - 5:0]
 (heading1_prefix) ; [1:1 - 2]
 title: (paragraph_segment) ; [1:3 - 9]
 content: (generic_list) ; [2:3 - 3:10]
  (unordered_list1) ; [2:3 - 3:0]
   (unordered_list1_prefix) ; [2:3 - 4]
   content: (paragraph) ; [2:5 - 3:0]
    (paragraph_segment) ; [2:5 - 10]
  (unordered_list1) ; [3:3 - 10]                     <----- range only spans row 3
   (unordered_list1_prefix) ; [3:3 - 4]
   content: (paragraph) ; [3:5 - 10]
    (paragraph_segment) ; [3:5 - 10]

For this file (same but without the final empty line)

* heading
  - item 1
  - item 2

:InspectTree returns the following tree:

(heading1) ; [1:1 - 4:0]
 (heading1_prefix) ; [1:1 - 2]
 title: (paragraph_segment) ; [1:3 - 9]
 content: (generic_list) ; [2:3 - 4:0]
  (unordered_list1) ; [2:3 - 3:0]
   (unordered_list1_prefix) ; [2:3 - 4]
   content: (paragraph) ; [2:5 - 3:0]
    (paragraph_segment) ; [2:5 - 10]
  (unordered_list1) ; [3:3 - 4:0]                    <----- range spans rows 3 and 4
   (unordered_list1_prefix) ; [3:3 - 4]
   content: (paragraph) ; [3:5 - 4:0]
    (paragraph_segment) ; [3:5 - 10]

For this file

* heading
  - item 1
  - item 2
* heading

we get the following tree:

(heading1) ; [1:1 - 4:0]
 (heading1_prefix) ; [1:1 - 2]
 title: (paragraph_segment) ; [1:3 - 9]
 content: (generic_list) ; [2:3 - 4:0]
  (unordered_list1) ; [2:3 - 3:0]
   (unordered_list1_prefix) ; [2:3 - 4]
   content: (paragraph) ; [2:5 - 3:0]
    (paragraph_segment) ; [2:5 - 10]
  (unordered_list1) ; [3:3 - 4:0]                    <----- range spans rows 3 and 4
   (unordered_list1_prefix) ; [3:3 - 4]
   content: (paragraph) ; [3:5 - 4:0]
    (paragraph_segment) ; [3:5 - 10]
(heading1) ; [4:1 - 5:0]
 (heading1_prefix) ; [4:1 - 2]
 title: (paragraph_segment) ; [4:3 - 9]