v0.6: SQLite observation full text search

See all issues and PRs for 0.6 here

Note: SQLite table schemas have changed. If you've created a SQLite database with a previous version of pyinaturalist-convert, here is a script to add the new columns and indexes:

ALTER TABLE observation ADD COLUMN created_at TEXT;
ALTER TABLE observation ADD COLUMN annotations JSON;
ALTER TABLE observation ADD COLUMN comments JSON;
ALTER TABLE observation ADD COLUMN identifications JSON;
ALTER TABLE observation ADD COLUMN identifications_count INTEGER;
ALTER TABLE observation ADD COLUMN ofvs JSON;
ALTER TABLE observation ADD COLUMN tags TEXT;
CREATE INDEX ix_observation_created_at ON observation (created_at
ALTER TABLE taxon ADD COLUMN leaf_taxa_count INTEGER;
ALTER TABLE taxon ADD COLUMN observations_count INTEGER;
ALTER TABLE taxon ADD COLUMN observations_count_rg INTEGER;
ALTER TABLE taxon ADD COLUMN reference_url TEXT;
ALTER TABLE taxon RENAME COLUMN active TO is_active;
ALTER TABLE photo ADD COLUMN position INTEGER;
ALTER TABLE photo ADD COLUMN uuid TEXT;

v0.5: SQLite aggregated taxonomy data

See all issues and PRs for 0.5 here

v0.4: Taxon full text search, SQLite observation+taxon database, and improved DwC converter

See all issues and PRs for 0.4 here.

Full text search

One of the main features in this release is the ability to build and search a full text search taxonomy database, with both scientific and common names.
For an overview, see: https://pyinaturalist-convert.readthedocs.io/en/stable/modules/fts.html

Files

Two pre-built FTS databases are attached:

• taxon-fts-en.tar.gz: Scientific names + English common names
• taxon-fts-all-languages.tar.gz: Scientific names + Common names for all available languages

Installation

Default install location varies by platform. This can be show via pyinaturalist_convert.DATA_DIR:

python -c "from pyinaturalist_convert import DATA_DIR; print(DATA_DIR);"

Example script to download and install on Linux:

wget https://github.com/pyinat/pyinaturalist-convert/releases/download/v0.4.0/taxon-fts-all-languages.tar.gz
tar -xvf taxon-fts-all-languages.tar.gz
mkdir -p ~/.local/share/pyinaturalist/
mv *.db ~/.local/share/pyinaturalist/

Usage

Usage example:

from pyinaturalist_convert import TaxonAutocompleter

ta = TaxonAutocompleter()
ta.search('aves')

SQLite database

This release also adds the ability to store observation and taxonomy data in SQLite. For details, see: https://pyinaturalist-convert.readthedocs.io/en/stable/modules/db.html