IndexedDB Cache

IndexedDB is a technology to have fairly large no-sql databases on the browser.

It can be used in several ways but in particular we want to use it for caching data. Some of the data used in explore is fairly large (candidate ags stars) or expensive to calculate (ITC)

The logic is fairly simple the application sends a message to a web worker. The Worker checks if the data is available locally and send it back to the app or it requests it, stores it in indexeddb and sends it back to the app

As a good cache data can be deleted at any time and we have a facility to expire old data

Stores

We are using one store cacheStore that contains a key which should be easy to calculate and uniquely identify a resource (for example catalog-HASH and the value contains a json object with a timestamp and a key to another table. cacheStore is easy to index and can be quickly examined to detect stale objects

Another store contains the actual data, e.g. catalogStore and it is indexed by the same key as in cacheStore and the content is specific to the type of data

Storage types

Commonly simple json objects are stored in the data store. That makes it easy to check them and do queries on the objects However in some cases like in the catalog stars, the amount of data starts to be large and repetitive. Therefore we can store the data in a binary encoding. A quick experiment shows a 5x reduction on storage needs

Transfer types

On the case of binary encoded data we can send the data from the worker in binary format using a Transferable object which is a lot faster