Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add worker queue #630

Merged
merged 1 commit into from
Nov 30, 2023
Merged

Add worker queue #630

merged 1 commit into from
Nov 30, 2023

Commits on Nov 30, 2023

  1. Add job queue infrastructure

    We want to process bulk uploads asynchronously, and so this means we
    need a worker queue.  Since this initial task is going to be infrequent,
    we felt it did not warrant adding a new dependency to our stack (e.g.
    reddis or rabbitmq), and so we're going to just leverage postgres as our
    backend.
    
    I explored two tools for this purpose: pg-boss, and graphile-worker.
    Both packages appeared to meet our technical needs, and both seem to be
    maintained and similarly popular.  I decided to go with graphile-worker
    as it appears to be part of a suite of related tools and the primary
    developer appears to have built a community around that suite of tools
    (and earns some income to support the development of the project).
    
    Issue #559 Set up a queue / processing system for handling bulk proposal uploads
    slifty committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    00bc5ce View commit details
    Browse the repository at this point in the history