With pandoc-crossref unreferenced/unlabelled tables are included in the table reference count in pdf output via latex

[dnebauer]

When converting to pdf using latex (xelatex or lualatex) I found that unlabelled and unreferenced tables were included in the table reference count, so that visible table numbering 'missed' or 'skipped' numbers. This was not the case for equations or figures, where unlabelled and unreferenced equations and figures were not included in the reference count, and equation and figure numbering did not 'miss' or 'skip' numbers.

When converting to html, epub or docx the unlabelled and unreferenced tables were not included in the reference count, indicating this problem affects latex output only.

Here is a simple test markdown file to demonstrate the problem (requires an image file called 'warn.png'):

---
title: "Test Document"
---

## Check table reference counting

This is an unreferenced and unlabelled table:

+----:+:-------+
|Fruit|Apple   |
+-----+--------+

This is a referenced and labelled table -- see @tbl:table.

+----------------+---------+
|Food group      |Example  |
+===============:+:========+
|Vegetable       |Carrot   |
+----------------+---------+
|Meat            |Lamb     |
+----------------+---------+

: A table {#tbl:table}

## Check figure reference counting

This is an unreferenced and unlabelled image:

![](warn.png)

This is a referenced and labelled figure -- see @fig:figure.

![A figure](warn.png){#fig:figure}

## Check equation reference counting

This is an unreferenced and unlabelled equation:

$$ F=ma $$

This is a referenced and labelled equation -- see @eq:equation.

$$ E=mc^2 $$ {#eq:equation}

Output was generated with commands like:

pandoc --from="markdown" --filter="pandoc-crossref" --to=latex --output='test.pdf' --pdf-engine=xelatex 'test.md'

I've attached screenshots of the pdf, html, epub and docx output below. In the pdf output the first table reference is incorrectly numbered 2. In all other output formats the first table reference is correctly numbered 1. (Equation and figure numbering is correct in all output formats.)

• pdf output:

• docx output:

• epub output:

• html output:

I've also attached the docx and pdf output files themselves: test.pdf, test.docx. (Github won't attach html or epub files).

The templates used are from the jgm/pandoc-templates github repo.

Here is the intermediate tex file (generated using options --embed-resources and --standalone):

% Options for packages loaded elsewhere
\PassOptionsToPackage{unicode}{hyperref}
\PassOptionsToPackage{hyphens}{url}
\documentclass[
]{article}
\usepackage{xcolor}
\usepackage{amsmath,amssymb}
\setcounter{secnumdepth}{-\maxdimen} % remove section numbering
\usepackage{iftex}
\ifPDFTeX
  \usepackage[T1]{fontenc}
  \usepackage[utf8]{inputenc}
  \usepackage{textcomp} % provide euro and other symbols
\else % if luatex or xetex
  \usepackage{unicode-math} % this also loads fontspec
  \defaultfontfeatures{Scale=MatchLowercase}
  \defaultfontfeatures[\rmfamily]{Ligatures=TeX,Scale=1}
\fi
\usepackage{lmodern}
\ifPDFTeX\else
  % xetex/luatex font selection
\fi
% Use upquote if available, for straight quotes in verbatim environments
\IfFileExists{upquote.sty}{\usepackage{upquote}}{}
\IfFileExists{microtype.sty}{% use microtype if available
  \usepackage[]{microtype}
  \UseMicrotypeSet[protrusion]{basicmath} % disable protrusion for tt fonts
}{}
\makeatletter
\@ifundefined{KOMAClassName}{% if non-KOMA class
  \IfFileExists{parskip.sty}{%
    \usepackage{parskip}
  }{% else
    \setlength{\parindent}{0pt}
    \setlength{\parskip}{6pt plus 2pt minus 1pt}}
}{% if KOMA class
  \KOMAoptions{parskip=half}}
\makeatother
\usepackage{longtable,booktabs,array}
\usepackage{calc} % for calculating minipage widths
% Correct order of tables after \paragraph or \subparagraph
\usepackage{etoolbox}
\makeatletter
\patchcmd\longtable{\par}{\if@noskipsec\mbox{}\fi\par}{}{}
\makeatother
% Allow footnotes in longtable head/foot
\IfFileExists{footnotehyper.sty}{\usepackage{footnotehyper}}{\usepackage{footnote}}
\makesavenoteenv{longtable}
\usepackage{graphicx}
\makeatletter
\newsavebox\pandoc@box
\newcommand*\pandocbounded[1]{% scales image to fit in text height/width
  \sbox\pandoc@box{#1}%
  \Gscale@div\@tempa{\textheight}{\dimexpr\ht\pandoc@box+\dp\pandoc@box\relax}%
  \Gscale@div\@tempb{\linewidth}{\wd\pandoc@box}%
  \ifdim\@tempb\p@<\@tempa\p@\let\@tempa\@tempb\fi% select the smaller of both
  \ifdim\@tempa\p@<\p@\scalebox{\@tempa}{\usebox\pandoc@box}%
  \else\usebox{\pandoc@box}%
  \fi%
}
% Set default figure placement to htbp
\def\fps@figure{htbp}
\makeatother
\setlength{\emergencystretch}{3em} % prevent overfull lines
\providecommand{\tightlist}{%
  \setlength{\itemsep}{0pt}\setlength{\parskip}{0pt}}
\makeatletter
\@ifpackageloaded{subfig}{}{\usepackage{subfig}}
\@ifpackageloaded{caption}{}{\usepackage{caption}}
\captionsetup[subfloat]{margin=0.5em}
\AtBeginDocument{%
\renewcommand*\figurename{Figure}
\renewcommand*\tablename{Table}
}
\AtBeginDocument{%
\renewcommand*\listfigurename{List of Figures}
\renewcommand*\listtablename{List of Tables}
}
\newcounter{pandoccrossref@subfigures@footnote@counter}
\newenvironment{pandoccrossrefsubfigures}{%
\setcounter{pandoccrossref@subfigures@footnote@counter}{0}
\begin{figure}\centering%
\gdef\global@pandoccrossref@subfigures@footnotes{}%
\DeclareRobustCommand{\footnote}[1]{\footnotemark%
\stepcounter{pandoccrossref@subfigures@footnote@counter}%
\ifx\global@pandoccrossref@subfigures@footnotes\empty%
\gdef\global@pandoccrossref@subfigures@footnotes{{##1}}%
\else%
\g@addto@macro\global@pandoccrossref@subfigures@footnotes{, {##1}}%
\fi}}%
{\end{figure}%
\addtocounter{footnote}{-\value{pandoccrossref@subfigures@footnote@counter}}
\@for\f:=\global@pandoccrossref@subfigures@footnotes\do{\stepcounter{footnote}\footnotetext{\f}}%
\gdef\global@pandoccrossref@subfigures@footnotes{}}
\@ifpackageloaded{float}{}{\usepackage{float}}
\floatstyle{ruled}
\@ifundefined{c@chapter}{\newfloat{codelisting}{h}{lop}}{\newfloat{codelisting}{h}{lop}[chapter]}
\floatname{codelisting}{Listing}
\newcommand*\listoflistings{\listof{codelisting}{List of Listings}}
\makeatother
\usepackage{bookmark}
\IfFileExists{xurl.sty}{\usepackage{xurl}}{} % add URL line breaks if available
\urlstyle{same}
\hypersetup{
  pdftitle={Test Document},
  hidelinks,
  pdfcreator={LaTeX via pandoc}}

\title{Test Document}
\author{}
\date{}

\begin{document}
\maketitle

\subsection{Check figure reference
counting}\label{check-figure-reference-counting}

This is an unreferenced image:

\pandocbounded{\includegraphics[keepaspectratio]{warn.png}}

This is a referenced figure -- see fig.~\ref{fig:figure}.

\begin{figure}
\centering
\pandocbounded{\includegraphics[keepaspectratio,alt={A figure}]{warn.png}}
\caption{A figure}\label{fig:figure}
\end{figure}

\subsection{Check table reference
counting}\label{check-table-reference-counting}

This is an unreferenced table:

\begin{longtable}[]{@{}
  >{\raggedleft\arraybackslash}p{(\linewidth - 2\tabcolsep) * \real{0.0833}}
  >{\raggedright\arraybackslash}p{(\linewidth - 2\tabcolsep) * \real{0.1250}}@{}}
\toprule\noalign{}
\endhead
\bottomrule\noalign{}
\endlastfoot
Fruit & Apple \\
\end{longtable}

This is a referenced table -- see tbl.~\ref{tbl:table}.

\begin{longtable}[]{@{}
  >{\raggedleft\arraybackslash}p{(\linewidth - 2\tabcolsep) * \real{0.2361}}
  >{\raggedright\arraybackslash}p{(\linewidth - 2\tabcolsep) * \real{0.1389}}@{}}
\caption{\label{tbl:table}A table}\tabularnewline
\toprule\noalign{}
\begin{minipage}[b]{\linewidth}\raggedleft
Food group
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Example
\end{minipage} \\
\midrule\noalign{}
\endfirsthead
\toprule\noalign{}
\begin{minipage}[b]{\linewidth}\raggedleft
Food group
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Example
\end{minipage} \\
\midrule\noalign{}
\endhead
\bottomrule\noalign{}
\endlastfoot
Vegetable & Carrot \\
Meat & Lamb \\
\end{longtable}

\end{document}

I'm hoping there is a way to prevent unlabelled and unreferenced tables from being included in the table reference count when generating pdf output via latex.

I first reported this to the pandoc-crossref project (lierdakil/pandoc-crossref#480). The pandoc-crossref guru @lierdakil confirmed that this is a bug (or "at the very least violates the principle of least astonishment"). He suspects that recent changes to pandoc table handling have introduced this behaviour and encouraged me to report this issue here. I'm hoping the pandoc gurus may be able to shed some light on the cause of this behaviour and have some suggestions on how to fix or work around it.

I'm happy to do basic troubleshooting and experimenting and, as mentioned, can copy the details from the initial issue report here if that is desired.

Versions

• pandoc 3.7.0.2, Features: +server +lua, Scripting engine: Lua 5.4
• pandoc-crossref v0.3.20 git commit UNKNOWN (UNKNOWN) built with Pandoc v3.7.0.2, pandoc-types v1.23.1 and GHC 9.8.4

[jgm]

Is this a pandoc issue or a pandoc-crossref issue? pandoc-crossref is a separate project and you should submit this there, unless it's clear that the issue is with pandoc itself.

[tarleb]

I think it can be treated as a pandoc issue. Shorter example that doesn't involve pandoc-crossref:

This is an unlabelled table:

+----:+:-------+
|Fruit|Apple   |
+-----+--------+

This is a labelled table.

+----------------+---------+
|Food group      |Example  |
+===============:+:========+
|Vegetable       |Carrot   |
+----------------+---------+
|Meat            |Lamb     |
+----------------+---------+

: A table

Result:

Having a Table 2 without an (obvious) Table 1 does seem slightly weird.

[jgm]

Here is pandoc's LaTeX output for that case:

This is an unlabelled table:

+----:+:-------+
|Fruit|Apple   |
+-----+--------+

This is a labelled table.

+----------------+---------+
|Food group      |Example  |
+===============:+:========+
|Vegetable       |Carrot   |
+----------------+---------+
|Meat            |Lamb     |
+----------------+---------+

: A table
This is an unlabelled table:

\begin{longtable}[]{@{}
  >{\raggedleft\arraybackslash}p{(\linewidth - 2\tabcolsep) * \real{0.0833}}
  >{\raggedright\arraybackslash}p{(\linewidth - 2\tabcolsep) * \real{0.1250}}@{}}
\toprule\noalign{}
\endhead
\bottomrule\noalign{}
\endlastfoot
Fruit & Apple \\
\end{longtable}

This is a labelled table.

\begin{longtable}[]{@{}
  >{\raggedleft\arraybackslash}p{(\linewidth - 2\tabcolsep) * \real{0.2361}}
  >{\raggedright\arraybackslash}p{(\linewidth - 2\tabcolsep) * \real{0.1389}}@{}}
\caption{A table}\tabularnewline
\toprule\noalign{}
\begin{minipage}[b]{\linewidth}\raggedleft
Food group
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Example
\end{minipage} \\
\midrule\noalign{}
\endfirsthead
\toprule\noalign{}
\begin{minipage}[b]{\linewidth}\raggedleft
Food group
\end{minipage} & \begin{minipage}[b]{\linewidth}\raggedright
Example
\end{minipage} \\
\midrule\noalign{}
\endhead
\bottomrule\noalign{}
\endlastfoot
Vegetable & Carrot \\
Meat & Lamb \\
\end{longtable}

Is there any problem? The numbers in this case are being assigned by LaTeX itself, not pandoc.

[tarleb]

I thought there might be an easy way to fix this, but it seem like there isn't. Thus closing, as it appears to be a LaTeX problem.

Sorry for sending you from repo to repo, @dnebauer. Maybe report this in at https://github.com/latex3/latex2e/issues.

[jgm]

I found something relevant here:
https://tex.stackexchange.com/questions/32553/longtable-caption-without-incrementing-table
If we use longtable* for non-captioned tables, maybe that fixes this? (Though it seems to require the caption package.) Another alternative is to insert a command that manually decrements the counter.

[lierdakil]

So... without \usepackage{caption}, longtable* plain doesn't exist. And with it, it doesn't seem to do anything (see lierdakil/pandoc-crossref#480 (comment))...

There is a clunky workaround, which boils down to basically this:

\newenvironment{longtable*}{\renewcommand\LTcaptype{}\longtable}{\endlongtable}

Is it worth it? Dunno. At least this one does what it's supposed to...

[lierdakil]

I should note, \LTcaptype thing has been introduced here latex3/latex2e#1398 about a year ago. I'm not sure if it's related, but to me, it seems bit suspicious nobody complained about this issue before.