Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent path usage hinders immutability in config directory mode #2433

Open
zner0L opened this issue Oct 7, 2024 · 0 comments
Open

Comments

@zner0L
Copy link

zner0L commented Oct 7, 2024

Problem description

I tried run datasette in immutable and directory mode at the same time. The documentation on this is a little ambiguous, but I figured out that if my configuration directory contains a inspect-data.json, generated by datasette inspect my-immutable.db --inspect-file=inspect-data.json, my-immutable would be loaded in immutable mode.
However, datasette would always load the database in mutable mode.

Problem resolution

After digging through the code, I found the problem. datasette checks which databases to load immutably by comparing just the filename and not the full path:

datasette/datasette/app.py

Lines 287 to 291 in 832f76c

if not immutables:
immutable_filenames = [i["file"] for i in inspect_data.values()]
immutables = [
f for f in self.files if Path(f).name in immutable_filenames
]

But my inspect-data.json contained absolute paths, because I had run datasette inspect /not/my/cwd/*.db --inspect-file=/not/my/cwd/inspect-data.json with absolute paths.

Because the paths are used inconsistently without this being mentioned in the documentation, I couldn’t figure out why the database was loaded mutably. After I changed the path to just the filename, the problem disappeared.

My specific configs can be found at https://github.com/tweaselORG/data.tweasel.org/tree/main/datasette.

/-/versions:

{
    "python": {
        "version": "3.10.12",
        "full": "3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0]"
    },
    "datasette": {
        "version": "0.64.8"
    },
    "asgi": "3.0",
    "uvicorn": "0.30.1",
    "sqlite": {
        "version": "3.37.2",
        "fts_versions": [
            "FTS5",
            "FTS4",
            "FTS3"
        ],
        "extensions": {
            "json1": null
        },
        "compile_options": [
            "ATOMIC_INTRINSICS=1",
            "COMPILER=gcc-11.4.0",
            "DEFAULT_AUTOVACUUM",
            "DEFAULT_CACHE_SIZE=-2000",
            "DEFAULT_FILE_FORMAT=4",
            "DEFAULT_JOURNAL_SIZE_LIMIT=-1",
            "DEFAULT_MMAP_SIZE=0",
            "DEFAULT_PAGE_SIZE=4096",
            "DEFAULT_PCACHE_INITSZ=20",
            "DEFAULT_RECURSIVE_TRIGGERS",
            "DEFAULT_SECTOR_SIZE=4096",
            "DEFAULT_SYNCHRONOUS=2",
            "DEFAULT_WAL_AUTOCHECKPOINT=1000",
            "DEFAULT_WAL_SYNCHRONOUS=2",
            "DEFAULT_WORKER_THREADS=0",
            "ENABLE_COLUMN_METADATA",
            "ENABLE_DBSTAT_VTAB",
            "ENABLE_FTS3",
            "ENABLE_FTS3_PARENTHESIS",
            "ENABLE_FTS3_TOKENIZER",
            "ENABLE_FTS4",
            "ENABLE_FTS5",
            "ENABLE_JSON1",
            "ENABLE_LOAD_EXTENSION",
            "ENABLE_MATH_FUNCTIONS",
            "ENABLE_PREUPDATE_HOOK",
            "ENABLE_RTREE",
            "ENABLE_SESSION",
            "ENABLE_STMTVTAB",
            "ENABLE_UNLOCK_NOTIFY",
            "ENABLE_UPDATE_DELETE_LIMIT",
            "HAVE_ISNAN",
            "LIKE_DOESNT_MATCH_BLOBS",
            "MALLOC_SOFT_LIMIT=1024",
            "MAX_ATTACHED=10",
            "MAX_COLUMN=2000",
            "MAX_COMPOUND_SELECT=500",
            "MAX_DEFAULT_PAGE_SIZE=32768",
            "MAX_EXPR_DEPTH=1000",
            "MAX_FUNCTION_ARG=127",
            "MAX_LENGTH=1000000000",
            "MAX_LIKE_PATTERN_LENGTH=50000",
            "MAX_MMAP_SIZE=0x7fff0000",
            "MAX_PAGE_COUNT=1073741823",
            "MAX_PAGE_SIZE=65536",
            "MAX_SCHEMA_RETRY=25",
            "MAX_SQL_LENGTH=1000000000",
            "MAX_TRIGGER_DEPTH=1000",
            "MAX_VARIABLE_NUMBER=250000",
            "MAX_VDBE_OP=250000000",
            "MAX_WORKER_THREADS=8",
            "MUTEX_PTHREADS",
            "OMIT_LOOKASIDE",
            "SECURE_DELETE",
            "SOUNDEX",
            "SYSTEM_MALLOC",
            "TEMP_STORE=1",
            "THREADSAFE=1",
            "USE_URI"
        ]
    }
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant