Skip to content

Commit

Permalink
Deploy to staging (#264)
Browse files Browse the repository at this point in the history
* Run storybook tests with playwright (#249)

* resolve conflicts

* test cleanup

* add post deployment action for testing

* update lockfile to fix failing action

* update action name

* correct config dir path

* add workflow dispatch for testing

* add push action for wip branch

* fix failing test for radio group pattern edit

* update documentation and add script for storybook testing with playwright

* update lockfile

* update path in action

* set endpoint in action

* update workflow endpoint and run mode

* install playwright in gh action

* create a self-contained image that runs through the tests on docker build

* use docker in post-deploy action

* clean up logging for docker build

* simplify end to end tests

* debugging tests

* rename test

* install unfiltered deps in docker--no-verify

* change end-to-end back to imported convention

* test deliberate failure in pipeline

* remove deliberately failing test

* update adr for test strategy

* add drag and drop test

* dry up test code

* add mouse interaction test for drag-and-drop

* fix flaky test

* remove flaky test

* remove commented code

* ignore pnpm cache dir

* use netcat instead of sleep to start server. pare down the copied files in the base image.

* use netcat instead of sleep to start server. pare down the copied files in the base image.

* add script to provide config for docker functions

* update documentation and add provisions for running stories locally

* grammar correction

* Add "user story" issue template (#220)

* Add initial "user story" template

* Update .github/ISSUE_TEMPLATE/story.md

* 239 page drag and drop (#267)

* Add PageMenuEdit component to provide page reordering features.

* Add routing to component to prevent page switching on reorder.

The code now uses the 'useSearchParams' hook from 'react-router-dom' to allow PageMenuEdit component to know current page id. This change also introduces a method to update search parameters when the page order is updated, thereby accommodating page navigation within the user interface.

* formatting

* extract update order to its own function

* leave a todo to address before merge

* move pagemenuedit to formedit directory and do some cleanup

* provide pagemenuedit in design package root

* test page reorder with drag and drop

The tests include a setup step with localstorage.

* move test to a new file and clean up storage object

* add parameter check for nav reorder

* a11y fix for drag button

* convert to draggable list to ul > li markup

* Add presentation prop to DraggableList component

The DraggableList component has been updated to include an optional presentation prop. This allows users to specify if they want the 'compact' or 'default' view for the DraggableList. By default, the presentation prop is set to 'default'.

* Update styles and layout in form components

removed styles that were coming back as undefined. Clean up the mobile view. Some simplification of visual presentation.

* update test based on new list structre for draggable

* add stories for draggable list to showcase new presentation prop

* remove unused classes from markup

* move pagemenuedit component to the pagesetedit file to provide more context of usage

* refactor test with page object mode

* border styles for compact draggablelist

* Add PageMenuEdit component to provide page reordering features.

* Add routing to component to prevent page switching on reorder.

The code now uses the 'useSearchParams' hook from 'react-router-dom' to allow PageMenuEdit component to know current page id. This change also introduces a method to update search parameters when the page order is updated, thereby accommodating page navigation within the user interface.

* formatting

* extract update order to its own function

* leave a todo to address before merge

* move pagemenuedit to formedit directory and do some cleanup

* provide pagemenuedit in design package root

* test page reorder with drag and drop

The tests include a setup step with localstorage.

* move test to a new file and clean up storage object

* add parameter check for nav reorder

* a11y fix for drag button

* convert to draggable list to ul > li markup

* Add presentation prop to DraggableList component

The DraggableList component has been updated to include an optional presentation prop. This allows users to specify if they want the 'compact' or 'default' view for the DraggableList. By default, the presentation prop is set to 'default'.

* Update styles and layout in form components

removed styles that were coming back as undefined. Clean up the mobile view. Some simplification of visual presentation.

* update test based on new list structre for draggable

* add stories for draggable list to showcase new presentation prop

* remove unused classes from markup

* move pagemenuedit component to the pagesetedit file to provide more context of usage

* refactor test with page object mode

* border styles for compact draggablelist

* visual polish for demo

* Remove deprecated dependency and update action versions (#282)

* remove deprecated dependency

* remove coverage report for debugging

* turn off gh actions reporter in test command

* version bump pnpm in test

* update action versions to fix deprecation notice

* Login.gov authentication - new packages infra-core, database, auth (#254)

* login.gov sign-in working, roughly.

* Move login and logout to the header menu.

* For testing, trigger deploy on the login-gov branch

* Initial attempt at wiring login.gov credentials to deployed containers

* Add code to help manage secrets to the command-line interface.

* Create service-like command functions for secrets, and unit tests. Wire up in the command-line controller.

* Improve cli app organization

* Add set-login-gov-keys CLI command, use case, and tests.

* Add ADR for secrets management.

* Clarify README comment

* Fix bad comment

* Compare against correct value in test

* Add missing files

* Add key generation mocks in unit test.

* remove vitest reporter that hasn't been working for quite some time

* Set AUTH_SECRET for the auth library we're using

* Explicitly read the AUTH_SECRET in auto.config.ts, rather than rely on the framework to read.

* Move server secrets retrieval into secrets.ts

* experiment to revert - use hardcoded string for auth secret, in attempt to get tests passing in CI

* To improve readability, use literal strings in login.gov unit test

* Fix typing error in secrets package and organize its adapters into a separate subdirectory.

* Add "delete secret" command (so I can clean up mispelled keys)

* Test deployment to dev

* sanity check on reading an environment variable

* Used consistent AUTH_SECRET in CI tests; randomness is unnecessary

* Remove debug exception

* Refashion secrets packages as infra/core

* Remove unused docassemble cli commands

* Update infra references

* fix infra dir in validate workflow

* Fix infra-cdktf filtering in root package.json

* Log in flow wired up w/o sessions. Also, TODO: track the nonce.

* Add database package with an initial migration using Knex, with Kysely for query building.

* Add initial database package

* Organize database and auth code; trying to track down import problems when the Astro build is integrated at the app level. I'll factor out the auth code first.

* Create @atj/auth, and move logic in there.

* Initial auth flow working, but tests of auth services still TODO. Integration tests in the demo servers are still failing.

* Add tests for getProviderCallback and logOut.

* Add nonce

* Add createSession

* Test for processProviderCallback. Includes fix to reuse sqlite3 connection from knex.

* Add tests for process-session-cookie

* Manage database dependencies so the integrating form service apps may import and run the server's express handler. This includes:
- Create helper in @atj/common to create service objects from a collection of service functions
- Remove module-global imports of @atj/database from the @atj/server, replacing with async imports where appropriate
TODO: The app integration tests for Kansas and DOJ are now passing, but the build bundle fails.

* Instantiate SSMClient in the AWSParameterStoreSecretsVault constructor

* Update dependencies, fix issue with dev server mode not working, rename DatabaseService to DatabaseGateway

* Work to get the app integrations running again, which primarily involved using .js imports, so we are consistent with ESM loading rules.

* Trying to get the dependency graph command to work again. Will revisit later.

* Use git commit hash rather than deployment identifier as the deployment's docker tag

* Upgrade more dependencies (trying to get coverage working, and failing)

* Remove tsup usage from libraries

* Fix infra/cdktf filter in build script

* Use correct base tsconfig in the design package.

* Update design testing libraries

* Punt of getting code coverage working, for now.
Context: the design lib has an unhandled exception when coverage is enabled.

* Comment out davelosert/[email protected]

* In the Terraform PR plan comment, use the target branch of the PR rather than a hard-coded "staging" value

* Update e2e docker base image to newer playwright version

* Upgrade Storybook to 8.2. 8.0 caused some problems on one dev machine, so hoping the issues are resolved now.

* Fix missing .js extensions on db adapters

* Upgrade @storybook/test-runner to 0.19.1 (fixes build break)

* Wire client secret from app to login.gov provider in deployed apps.

* Remove passage of private key to auth - it shouldn't be necessary with PKCE. Revisit if we switch to the JWT method.

* Make clientSecret undefined rather than an empty string

* Remove clientSecret from auth test

* Remove clientSecret from LoginGovOptions

* Add extra debug logging for failed authorization code checks

* Remove deploy trigger from login-gov feature branch

* Remove unnecessary login.gov bash script (superceded by CLI commands)

* Remove old comment

* Add comment explaining `createService`

* Remove lucia from web server package, in lieu of the auth package.

* Added the ability to move question (#274)

* Part 1 of moving question to another page

* Part 2 of moving question to another page

* Added the ability to move question to another page

* Added the ability to move question to another page

* fixing get by role selector to remove ambiguity

* removed logging

* Removed some console logs

* Fixed some bugs and accessibility issues

This commit includes a fix for the bug when moving questions back and forth and also a bug with the list of page options. This update also addresses some accessibility issues.

* Fixed some bugs and accessibility issues part 2

* Addressed some bugs and feedback reviews

In this commit I fixed a couple of bugs and made updates per some reviews in the pull request.

* Addressed some bugs and feedback reviews Part 2

* Updated the builder test to account for source page

* Made updates per the reviews in my PR

* Made updates per the reviews in my PR Part 2

* Ensured that page types can't be moved

* A minor update to the store.ts

* A minor update to the store.ts Part 2

---------

Co-authored-by: ethangardner <[email protected]>

* Copy patterns and fieldsets (#295)

* Added the ability to copy patterns and fieldsets

* Updated the builder test

* Updated package.json

* Made updates per PR review request

* Fixes for the merge with main branch

---------

Co-authored-by: Daniel Naab <[email protected]>

* Add Postgres database support (#293)

* Rename database contexts to their method (in memory, filesystem) rather than dev vs test

* Add Postgres Testcontainer, implemented via a describeDatabase() test suite factory that will run tests via both Sqlite and Postgres

* Wire describeDatabase up to each db gateway test, and create db helper module for dealing with date differences between sqlite (INTEGER) and Postgres (TIMESTAMP).

* Add ability to share Postgres container across all the database package tests.

* Get Postgres and its testability wired up to the DOJ demo app. The entrypoint still needs cloud.gov environment settings.

* Add Postgres RDS to cloud.gov environment

* temp: run the deploy on this branch for testing

* Postgres Terraform config + app wiring

* Fix aws-rds VCAP_SERVICES lookup path

* on cloud, use ssl with postgres

* Ignore self-signed cert errors with rejectUnauthorized. Revisit by seeing if we can install the RDS certs on the container

* Hack in Postgres Lucia adapter

* To satisfy Lucia, use TEXT for session and user ID types. We will want to move these queries into the database package so we can get test coverage over them.

* Add allowlists for demo apps

* Remove deploy on feature branch

* Form service backend implemented (#297)

* Refactor form service to use createService utility

* Use injected db gateway in form service functions, without changes to the existing localStorage approach.

* Add form db gateway functions

* Wire form db gateways to application.

* Wire form service to frontend via a client-side proxy and API routes

* Fix a couple build issues

* Move db gateway functions into the service that consumes it.
There are failing tests, due to describe blocks defined in the shared `describeDatabase` helper not getting registered with Vitest. Pushing up to CI to run test there, and rule out a local config issue.

* Get describeDatabase working in the auth and forms packages; mark integration tests of demo apps as "expect failure" due to dependency issues that need to be worked out.

* Temp: deploy on this branch

* Update lockfile

* Always use ssl with demo apps

* Trying to isolate db dependencies; get CLI working with NodeNext modules

* Tweak imports for NodeNext

* Testing

* Builds working with refactored database structure and module settings.
Includes an upgrade of Turborepo and new clean tasks.

* Remove pnpm version in Github Actions script, in lieu of the packageManager key in package.json

* Switch design package to NodeNext module resolution

* Add tests for all the forms services (todo: submitForm).
This necessitated giving the forms library the Rollup treatment, and moving some dependencies around to avoid intermixing nodejs deps with the design lib.

* use createTestFormServiceContext in submitForm test

* Fix build break in spotlight app

* Use NodeNext in the base tsconfig.json

* docassemble updated for nodenext

* Wire existing data flow to submitForm. to be updated for server environment

* Do one more test deploy

* Remove test deploy

* Added homepage content to splash page (#298)

* Updated the splash page content

* Updated the hover color for the video button

* Added a specific class to splash page modal

* Made additional updates per review comments in PR

* Added defer to script and other spacing updates

* Added lazy loading to infographic

* Wrapped the script in an IIFE

* Update a few import statements to new style

* echo environment variables for cg pages debugging

* Add debug logging

* With the switch to nodenext modules, use import.meta.env instead of process.env in astro config file

* more debugging

* debugging

* debugging

* try turbo env

* debugging

* remove debugging

---------

Co-authored-by: Daniel Naab <[email protected]>

* Update DOJ allowlist (#303)

---------

Co-authored-by: Ethan Gardner <[email protected]>
Co-authored-by: Daniel Naab <[email protected]>
Co-authored-by: Natasha Pierre-Louis <[email protected]>
Co-authored-by: ethangardner <[email protected]>
Co-authored-by: Daniel Naab <[email protected]>
  • Loading branch information
6 people authored Sep 5, 2024
1 parent fed86ca commit e7b0add
Show file tree
Hide file tree
Showing 433 changed files with 24,603 additions and 16,416 deletions.
64 changes: 64 additions & 0 deletions .github/ISSUE_TEMPLATE/story.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
name: Story
about: Suggest a user story for this product
title: ''
labels: story
assignees: ''
projects: ['GSA-TTS/17']
---

## Overview

As a _, I would like _, so that I can _.

## Context

*Optional: Any reference material or thoughts we may need for later reference, or assumptions of prior or future work that's out of scope for this story.*

- [ ]

## Acceptance Criteria

*Required outcomes of the story*

- [ ]

## Research Questions

- *Optional: Any initial questions for research*

## Tasks

*Research, design, and engineering work needed to complete the story.*

- [ ]

## Definition of done

The "definition of done" ensures our quality standards are met with each bit of user-facing behavior we add. Everything that can be done incrementally should be done incrementally, while the context and details are fresh. If it’s inefficient or “hard” to do so, the team should figure out why and add OPEX/DEVEX backlog items to make it easier and more efficient.

- [ ] Behavior
- [ ] Acceptance criteria met
- [ ] Implementation matches design decisions
- [ ] Documentation
- [ ] ADRs (`/documents/adr` folder)
- [ ] Relevant `README.md`(s)
- [ ] Code quality
- [ ] Code refactored for clarity and no design/technical debt
- [ ] Adhere to separation of concerns; code is not tightly coupled, especially to 3rd party dependencies; dependency rule followed
- [ ] Code is reviewed by team member
- [ ] Code quality checks passed
- [ ] Security and privacy
- [ ] Automated security and privacy gates passed
- [ ] Testing tasks completed
- [ ] Automated tests pass
- [ ] Unit test coverage of our code >= 90%
- [ ] Build and deploy
- [ ] Build process updated
- [ ] API(s) are versioned
- [ ] Feature toggles created and/or deleted. Document the feature toggle
- [ ] Source code is merged to the main branch

## Decisions

- *Optional: Any decisions we've made while working on this story*
12 changes: 12 additions & 0 deletions .github/workflows/_end-to-end.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
name: End-to-end tests
on:
workflow_call:
jobs:
end-to-end:
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build Docker to run tests
run: |
docker build --platform linux/amd64 --tag 'playwright' . -f ./e2e/Dockerfile --target test
39 changes: 0 additions & 39 deletions .github/workflows/_playwright.yml

This file was deleted.

5 changes: 2 additions & 3 deletions .github/workflows/_terraform-apply.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ jobs:
uses: pnpm/action-setup@v4
id: pnpm-install
with:
version: 8
run_install: false

- name: Get pnpm store directory
Expand All @@ -53,7 +52,7 @@ jobs:

- name: Initialize Terraform CDK configuration
shell: bash
working-directory: infra
working-directory: infra/cdktf
run: |
pnpm cdktf get
pnpm build:tsc
Expand All @@ -68,7 +67,7 @@ jobs:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
shell: bash
working-directory: infra
working-directory: infra/cdktf
run: |
cf api https://api.fr.cloud.gov
DEPLOY_ENV=${DEPLOY_ENV} pnpm cdktf deploy --auto-approve
9 changes: 4 additions & 5 deletions .github/workflows/_terraform-plan-pr-comment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ jobs:
uses: pnpm/action-setup@v4
id: pnpm-install
with:
version: 8
run_install: false

- name: Get pnpm store directory
Expand All @@ -55,7 +54,7 @@ jobs:

- name: Initialize Terraform CDK configuration
shell: bash
working-directory: infra
working-directory: infra/cdktf
run: |
pnpm cdktf get
pnpm build:tsc
Expand All @@ -70,19 +69,19 @@ jobs:
cf api https://api.fr.cloud.gov
- name: Synthesize Terraform configuration
working-directory: infra
working-directory: infra/cdktf
run: |
DEPLOY_ENV=${DEPLOY_ENV} pnpm cdktf synth
- name: Get Terraform stack name
id: get_stack_name
working-directory: infra
working-directory: infra/cdktf
run: |
DEPLOY_ENV=${DEPLOY_ENV} pnpm cdktf output --outputs-file outputs.json
echo "stack_name=$(jq -r 'keys[0]' outputs.json)" >> $GITHUB_OUTPUT
- name: Create Terraform plan
uses: dflook/terraform-plan@v1
with:
path: infra/cdktf.out/stacks/${{ steps.get_stack_name.outputs.stack_name }}
path: infra/cdktf/cdktf.out/stacks/${{ steps.get_stack_name.outputs.stack_name }}
label: ${{ steps.get_stack_name.outputs.stack_name }}
15 changes: 7 additions & 8 deletions .github/workflows/_validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,14 @@ jobs:
run: echo ::set-output name=NODE_VERSION::$(cat .nvmrc)

- name: Install required node.js version
uses: actions/setup-node@v3
uses: actions/setup-node@v4
with:
node-version: ${{ steps.nvmrc.outputs.NODE_VERSION }}

- name: Install pnpm
uses: pnpm/action-setup@v4
id: pnpm-install
with:
version: 8
run_install: false

- name: Get pnpm store directory
Expand All @@ -34,7 +33,7 @@ jobs:
run: |
echo "STORE_PATH=$(pnpm store path)" >> $GITHUB_OUTPUT
- uses: actions/cache@v3
- uses: actions/cache@v4
name: Setup pnpm cache
with:
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
Expand All @@ -56,19 +55,19 @@ jobs:

- name: Run test suite
shell: bash
run: pnpm test:ci
run: AUTH_SECRET=not-super-secret pnpm test:ci

- name: Initialize Terraform CDK configuration
shell: bash
run: |
cd infra
cd infra/cdktf
pnpm cdktf get
pnpm build:tsc
- name: Typecheck source code
shell: bash
run: pnpm typecheck

#- name: Vitest Coverage Report
# if: always()
# uses: davelosert/vitest-coverage-report-action@v2.2.0
# - name: Vitest Coverage Report
# if: always()
# uses: davelosert/vitest-coverage-report-action@v2.5.0
2 changes: 1 addition & 1 deletion .github/workflows/add-terraform-plan-to-pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,4 @@ jobs:
uses: ./.github/workflows/_terraform-plan-pr-comment.yml
secrets: inherit
with:
deploy-env: staging
deploy-env: ${{ github.base_ref }}
2 changes: 1 addition & 1 deletion .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,4 @@ jobs:
uses: ./.github/workflows/_terraform-apply.yml
secrets: inherit
with:
deploy-env: ${{ github.ref_name }}
deploy-env: ${{ github.ref_name }}
3 changes: 1 addition & 2 deletions .github/workflows/run-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,5 @@ jobs:
run-tests:
uses: ./.github/workflows/_validate.yml
e2e:
needs: [run-tests]
uses: ./.github/workflows/_playwright.yml
uses: ./.github/workflows/_end-to-end.yml
secrets: inherit
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
.DS_Store
.env
.pnpm-store/
*.code-workspace
_site
.turbo/
Expand Down
3 changes: 2 additions & 1 deletion .husky/pre-commit
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#!/bin/sh
pnpm lint
pnpm format
pnpm test
pnpm test
2 changes: 1 addition & 1 deletion .nvmrc
Original file line number Diff line number Diff line change
@@ -1 +1 @@
v20.15.0
v20.16.0
8 changes: 3 additions & 5 deletions apps/cli/README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,11 @@
# @atj/cli-app

This package includes a very simple command-line interface.
This package defines the platform's command-line interface.

## Example commands

Examples:
Commands are defined to aid with platform management operations. To see available commands, run:

```bash
pnpm run cli create-workspace-graph
pnpm cli --help
```

## Development
Expand Down
14 changes: 14 additions & 0 deletions apps/cli/build.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
const esbuild = require('esbuild');

esbuild
.build({
bundle: true,
entryPoints: ['./src/index.ts'],
format: 'cjs',
minify: true,
outdir: './dist',
platform: 'node',
sourcemap: true,
target: 'es2020',
})
.catch(() => process.exit(1));
8 changes: 5 additions & 3 deletions apps/cli/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,19 @@
"name": "@atj/cli-app",
"version": "1.0.0",
"description": "10x ATJ command-line interface",
"type": "commonjs",
"type": "module",
"license": "CC0",
"main": "src/index.ts",
"scripts": {
"build": "echo 'skipping...' #tsc -p .",
"cli": "ts-node src/index.ts",
"build": "tsup src/* --format esm",
"clean": "rimraf dist tsconfig.tsbuildinfo coverage",
"cli": "node dist/index.js",
"dev": "tsup src/* --watch",
"test": "vitest run --coverage"
},
"dependencies": {
"@atj/dependency-graph": "workspace:*",
"@atj/infra-core": "workspace:*",
"commander": "^11.1.0"
}
}
62 changes: 0 additions & 62 deletions apps/cli/src/cli-controller.ts

This file was deleted.

Loading

0 comments on commit e7b0add

Please sign in to comment.