Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for externalizing tikz images. #34

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

milesdai
Copy link
Owner

@milesdai milesdai commented Feb 28, 2022

This commit will cause all the tikz images to be generated first in the
tikz-cache folder before being included in the main PDF file. There are
several issues that need to be resolved for this to work:

  • The tikz-cache folder is not auto-generated, so it is being tracked in
    version control to avoid users cloning the repo and seeing mysterious
    error messages when compiling. No cached tikz files should be added to
    this repository.
  • The \newenvironment command used in our custom circuit and plot
    environments does not play well with the externalizing library. These
    were modified to use the environ library which solves the problem as
    described here
  • This caching significantly improves the compile time when figures are
    already cached, but the first compile takes much longer since the image
    compilation is single-threaded. However, it is possible to use
    tikzexternalize in list and make mode which allows you to use make -j to compile all the images using multiple threads. This requires
    running the pdflatex commands manually without some more complex
    configuration of a latexmkrc file. Only attempt this if you know what
    you are doing. Otherwise, just wait a couple minutes for the
    compilation.

@milesdai milesdai added the enhancement New feature or request label Feb 28, 2022
@milesdai milesdai self-assigned this Feb 28, 2022
This commit will cause all the tikz images to be generated first in the
tikz-cache folder before being included in the main PDF file. There are
several issues that need to be resolved for this to work:
- Whatever command is being used to compile the document (e.g. `latexmk`
or `pdflatex` needs to have the `-shell-escape` flag added.
- The tikz-cache folder is not auto-generated, so it is being tracked in
version control to avoid users cloning the repo and seeing mysterious
error messages when compiling. No cached tikz files should be added to
this directory.
- The \newenvironment command used in our custom circuit and plot
environments does not play well with the externalizing library. These
were modified to use the environ library which solves the problem as
described [here](https://tex.stackexchange.com/a/15614)
- This caching significantly improves the compile time when figures are
already cached, but the first compile takes much longer since the image
compilation is single-threaded. However, it is possible to use
tikzexternalize in `list and make` mode which allows you to use `make
-j` to compile all the images using multiple threads. This requires
running the pdflatex commands manually without some more complex
configuration of a latexmkrc file. Only attempt this if you know what
you are doing. Otherwise, just wait a couple minutes for the
compilation.
@milesdai
Copy link
Owner Author

Closes #32

@milesdai
Copy link
Owner Author

milesdai commented Feb 28, 2022

I'm leaving this as a draft PR for now to allow people to test out the workflow and see if they like it. A couple things to point out:

  1. Take a look at the first message above (or the commit message) for an explanation of what this is doing
  2. You will need to add the -shell-escape flag to whatever command you're using to compile latex with
  3. The first compilation will take a long time (probably around 1-3 minutes). If you are using overleaf, you may need to compile things offline first and upload them or compile the images in multiple batches by commenting out certain images. Do not clear out the tikz-cache directory. Its contents are ignored by Git and the caching will handle keeping the images fresh on its own.

Some issues I have observed so far:

  1. If an image is inserted into the middle of the document, all subsequent images will need to be regenerated.
  2. The initial compile is super slow as it is single-threaded

The solution to both problems is probably some combination of switching to make and using the standalone package. This would require some custom build scripts and would break on Overleaf anyway. Not sure if there are any other thoughts on this, but if people have time, it would be good to try and pull this branch and test out the workflow to see if it breaks anything for you locally.

@zoelzw
Copy link
Contributor

zoelzw commented Mar 25, 2022

Hi, could you explain a bit about "add the -shell-escape flag "? I'm using vs code + latex workshop and default keyboard shortcuts. How should I set the flag? Thanks!

@milesdai
Copy link
Owner Author

Hi, could you explain a bit about "add the -shell-escape flag "? I'm using vs code + latex workshop and default keyboard shortcuts. How should I set the flag? Thanks!

For sure! I use the same setup. Basically Latex Workshop uses "recipes" to define how the latex file should be built. The recipe specifies the commands you need to type into the command line to build the project. There were a number of default recipes included for me, and I added a new one to called latexmk-shell-escape where I copied the default latexmk recipe and just add the --shell-escape flag. I specifically wanted this option only for this project since shell escape has some security implications for projects I don't fully trust, so I added the recipe only to my vscode workspace settings.

To do this, I created a workspace-specific settings file at .vscode/settings.json under the project root directory. In that file, I added the following:

{
    "latex-workshop.latex.tools": [
        {
            "name": "latexmk-shell-esc",
            "command": "latexmk",
            "args": [
                "-shell-escape",              // <---- this line is the one that's important
                "-synctex=1",
                "-interaction=nonstopmode",
                "-file-line-error",
                "-pdf",
                "-outdir=%OUTDIR%",
                "%DOC%"
            ],
            "env": {}
        },
    ],
    "latex-workshop.latex.recipes": [
        {
            "name": "latexmk-shell-esc",
            "tools": [
                "latexmk-shell-esc"
            ]
        }
    ]
}

I think I may have needed to reload the extension, but after that, I had the new recipe available to me.

Let me know if this works for you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tex file taking very long to compile/render figures
2 participants