TeX splitting can cause the last parts of an equation to not be displayed

[kleinesfilmroellchen]

Description of bug / unexpected behavior

In many varying, highly context-sensitive instances, TeX splitting causes the last part(s) of an equation to not be visible. This happens both when using splitting with {{}}, providing multiple strings, and when combining both techniques as in the example.

Expected behavior

No matter the splitting, all parts of an equation should be visible. Or at least there should be an error thrown by the TeX handling system.

How to reproduce the issue

Code for reproducing the problem

class Example(Scene):
    def construct(self):
        template = TexTemplate()
        template.add_to_preamble(r'''\usepackage[english]{babel}
        \usepackage{csquotes}\usepackage{cancel}''')
        lpc_implies_polynomial = MathTex(
            r's[t] = a_1 s[t-1] + a_2 s[t-2] + a_3 s[t-3] + \dots\\', r'{{\Downarrow}}\\', r'\text{Polynomial function}',
            tex_template=template, tex_environment='gather*').scale(0.9).shift(UP*1.5)
        lpc_implies_not_polynomial = MathTex(
            r's[t] = a_1 s[t-1] + a_2 s[t-2] + a_3 s[t-3] + \dots\\', r'\xcancel{ {{\Downarrow}} }\\', r'\text{Polynomial function}',
            tex_template=template, tex_environment='gather*').scale(0.9).shift(DOWN*1.5)
        self.add(lpc_implies_not_polynomial, lpc_implies_polynomial)

Additional media files

Images/GIFs

Logs

Terminal output

Manim Community v0.16.0.post0

[09/28/22 12:04:03] INFO     Writing s[t] = a_1 s[t-1] + a_2 s[t-2] + a_3      tex_file_writing.py:87                             s[t-3] + \dots\\ \Downarrow \\ \text{Polynomial
                             function} to media\Tex\00fe9e1deec76e46.tex
[09/28/22 12:04:05] INFO     Writing s[t] = a_1 s[t-1] + a_2 s[t-2] + a_3      tex_file_writing.py:87                             s[t-3] + \dots\\ to
                             media\Tex\c6261af89b10ed5a.tex
[09/28/22 12:04:06] INFO     Writing \Downarrow to                             tex_file_writing.py:87                             media\Tex\7d700b24d332ca6c.tex
[09/28/22 12:04:08] INFO     Writing \quad\\ to media\Tex\803547458f0d8687.tex tex_file_writing.py:87[09/28/22 12:04:09] INFO     Writing \text{Polynomial function} to             tex_file_writing.py:87                             media\Tex\b83b64ccde753a44.tex
[09/28/22 12:04:10] INFO     Writing s[t] = a_1 s[t-1] + a_2 s[t-2] + a_3      tex_file_writing.py:87                             s[t-3] + \dots\\ \xcancel{  \Downarrow  }\\
                             \text{Polynomial function} to
                             media\Tex\2982feeb65364253.tex
[09/28/22 12:04:12] INFO     Writing \xcancel{} to                             tex_file_writing.py:87                             media\Tex\7b7b3a8e2f424a94.tex
[09/28/22 12:04:13] INFO     Writing {}\\ to media\Tex\9296f7c1398e4f49.tex    tex_file_writing.py:87

System specifications

LaTeX details

• LaTeX distribution (e.g. TeX Live 2020):

pdfTeX 3.141592653-2.6-1.40.24 (TeX Live 2022)
kpathsea version 6.3.4
Copyright 2022 Han The Thanh (pdfTeX) et al.
There is NO warranty.  Redistribution of this software is
covered by the terms of both the pdfTeX copyright and
the Lesser GNU General Public License.
For more information about these matters, see the file
named COPYING and the pdfTeX source.
Primary author of pdfTeX: Han The Thanh (pdfTeX) et al.
Compiled with libpng 1.6.37; using libpng 1.6.37
Compiled with zlib 1.2.11; using zlib 1.2.11
Compiled with xpdf version 4.03

• Installed LaTeX packages: (sorry this list is too long, it seems like I have pretty much all the CTAN packages)

Additional comments

This is actually my third try reporting this issue: In the first two, as soon as I isolated the equations in question, things started to work again. Now I finally have a "tame" reproducer; do note that I have seen things much whackier than this, where sometimes 90% of the equation is missing.

[kleinesfilmroellchen]

FWIW I checked the SVG output of the TeX subsystem and the equations are totally fine there.

[behackl]

The issue is with how TeX splitting is handled internally at the moment: when compiling a split string, Manim first compiles all separate substrings (which includes a somewhat dodgy preprocessing) from which it determines how many SVG paths belong to each substring. The submobject groups in the returned mobject are then determined based on the numbers determined from the compiled substrings.

However, in certain situations, the number of determined group members does not match with the number of SVG paths that actually belong to the substring. A particularly simple example was, for the longest time, MathTex(r"\sqrt{", "x}"), since r"\sqrt{" essentially got compiled as an empty \sqrt{}, which consists of fewer SVG paths than a root with some content. I don't think it is effectively possible to salvage this situation, the current implementation is simply not robust enough.

A possible solution is the implementation of a completely different approach (e.g., the MTex implementation from 3b1b/manim -- which we could port on top of #2898). Would be a breaking change, and things like the double brace notation would have to be reimplemented (or just completely thrown away and replaced by something more intuitive like double/triple spaces ...), but in the long run it would be a good change I believe.

This also is an issue that gets reported occasionally, most recently in #2955.

[kleinesfilmroellchen]

Thank you for the clear explanation, that makes a lot of sense and I see that I have to work around this for the time being. FWIW I'm not attached to the {{ notation, I'd say it's kind of a bad design as you often have to pay attention to not clash with TeX command curly braces.