Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tabu* does not work with alltt-type usage of Verbatim #3

Open
jfbu opened this issue Jan 20, 2019 · 6 comments · May be fixed by #5
Open

tabu* does not work with alltt-type usage of Verbatim #3

jfbu opened this issue Jan 20, 2019 · 6 comments · May be fixed by #5
Labels
old bug bug that has been in the code since 2011

Comments

@jfbu
Copy link

jfbu commented Jan 20, 2019

This MWE does not compile:

\documentclass{article}
\usepackage{fancyvrb}
\usepackage{tabu}
\begin{document}
\begin{tabu*}to\linewidth{|XX|}
A&
\begin{Verbatim}[commandchars=\\\{\}]
And this is a complete
Verbatim environment
\end{Verbatim}
\\
\end{tabu*}
\end{document}

That was already the case with TL2017. I read the manual but not the source code. Ah, I looked a bit now after examining a trace. I see in particular

\iftabuscantokens \endlinechar=10 \obeyspaces \fi

on line 1134 of the code in tabu.pdf, although I did not check if that line is really relevant it immediately brings to mind the well-known problem of scantokens with an active space.

I see now tabu's author is aware, as later on there is section "Compatibility with the fancyvrb package" in commented source code, and he adapts the fancyvrb end-check via \tabu@fancyvrb code. (lines 1912ff)

Thus this does not work if the \ has its native catcode. This is source of problem, which hence is deeply coded into tabu code: there is no provision for it to work with a Verbatim configured à la "alltt"".

Hence raising this issue here, to at least make it clear in the manual, or to handle it as a feature request.

@davidcarlisle
Copy link
Member

Thanks for raising the issue, I think (since issue #2 means tabu is completely broken) I'll push out a 2.10 today without trying to address this now. As you say, it's good to collect the issues here to be handled as anyone finds time.

@jfbu
Copy link
Author

jfbu commented Jan 22, 2019

To correct some bad wording in my post. tabu* does no \scantokens with an active space. It does collect the body with an active space, then executes \scantokens with space again normal. Verbatim will when expanding re-assign the space to active. Each control word (except for the very last one if it ends the body passed over to \scantokens) is now followed by a (normal) space which Verbatim made active. Then each such control word receives an active space as first argument.

@jfbu
Copy link
Author

jfbu commented Jan 22, 2019

It might be that if the "collect body phase" was done with a catcode 12 backslash, then the scantokens will be innocuous and recreate control sequences without an extra space character. If I get time I would have a closer look at tabu to see if it can work to modify the collecting phase this way. (if it works then the tabu patch of fancyvrb for checking end of Verbatim will become unneeded)

jfbu added a commit to jfbu/tabu that referenced this issue Jan 22, 2019
I don't know how one makes up .lvt type of validating file.

	new file:   testfiles/t003.lvt
@jfbu jfbu linked a pull request Jan 22, 2019 that will close this issue
jfbu added a commit to jfbu/tabu that referenced this issue Jan 22, 2019
MEMO: we can't have environment spread across multiple lines
inside alltt-type Verbatim as fancyvrb scopes each input line
@jfbu
Copy link
Author

jfbu commented Jan 23, 2019

My PR #5 implements the "catcode 12 backslash" idea. While keeping close to original code I did need to separate into two distinct branches the body collecting. But thinking about it, I see that simpler code would result from handling tabu* as a verbatim environment i.e. sanitize all catcodes and fetch to the end \end{tabu*} (or \end{longtabu*}). The current documentation I think allows tabu (without a star) to act like tabu* if some boolean is on but says official interfae is tabu*.

But dropping that and requiring either \end{tabu*} or \end{longtabu*}, one thus only needs in this alternative to either fetch all until \end{tabu*} or fetch all until \end{longtabu*}. But it is blocking here that nesting is impossible without extra code, which brings us back into the checking of the balancing of \begin and \end in original. Then we end up being not so different than the current code in PR #5 except that we don't have to worry about \endfoo situation and must count only (verbatimized) \begin{tabu*} occurrences.

jfbu added a commit to jfbu/tabu that referenced this issue Jan 23, 2019
…-use \tabu@rescan in source code

But \tabu@rescan is now \let to \scantokens
@FrankMittelbach
Copy link
Contributor

@jfbu do I understand correctly that this is an old issue (not a recent one)?

@FrankMittelbach FrankMittelbach added the old bug bug that has been in the code since 2011 label Feb 26, 2019
@jfbu
Copy link
Author

jfbu commented Feb 26, 2019

@FrankMittelbach Yes, this is my understanding as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
old bug bug that has been in the code since 2011
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants