LG-14716 Download and store Socure reason codes weekly #11350
Conversation
When Socure responds to requests against its identity verification API it includes "reason codes" to describe why a transaction passed or failed. These reason codes map back to descriptions that are available in the reason code API. Socure has suggested to us that we maintain our own copy of their reason codes. This commit adds a background job that will create a copy of the Socure reason codes and descriptions in our database on a weekly basis. The job fetches the list of codes from the API. It adds any new codes that do not exist in the database. It deactivates any codes that we have in the databsae that we do not observe in the response. The job will log the results when it is done. This includes: 1. A list of reason codes that were added 2. A list of reason codes that were deleted 3. Any exceptions that may have occured when adding the codes. changelog: Internal, Socure, Download and store reason codes weekly
|
I just looked back over the ticket in JIRA and discovered I missed one of the ACs. We want to log the "group" associated with error codes. I'm going to go back around and add that. |
|
Okay, I added support for groups and I added sensitivity comments to address the issue found in CI |
|
|
||
| t.timestamps comment: 'sensitive=false' | ||
|
|
||
| t.index :code, unique: true |
There was a problem hiding this comment.
Do you think there is any benefit to adding an index on deactivated_at, and maybe adding a scope to the model, so we can do like SocureReasonCodes.active? I feel like we're probably going to almost always filter out anything that's been deactivated.
(Or maybe that's a problem that will come when we start using these in code.)
| t.datetime :added_at, comment: 'sensitive=false' | ||
| t.datetime :deactivated_at, comment: 'sensitive=false' | ||
|
|
||
| t.timestamps comment: 'sensitive=false' |
There was a problem hiding this comment.
Should we just rely on the stock created_at this gives us vs. having an added_at column?
| Analytics.new( | ||
| user: AnonymousUser.new, | ||
| request: nil, | ||
| sp: service_provider_issuer, |
There was a problem hiding this comment.
I don't understand where service_provider_issuer is coming from. Most usages I'm seeing are reading it off a ServiceProvider record, or passing it through perform.
I am slightly concerned that this might be calling a non-existent method, and it's masked by using FakeAnalytics in the spec. But it is also extremely possible that I'm just missing the definition of this.
| rescue Faraday::ConnectionFailed, | ||
| Faraday::ServerError, | ||
| Faraday::SSLError, | ||
| Faraday::TimeoutError, | ||
| Faraday::ClientError, | ||
| Faraday::ParsingError => e |
There was a problem hiding this comment.
Absolutely not for your PR here, but I feel like our app is all over the place in how we handle different Faraday errors.
For example, where we stop trying to make fetch happen with LexisNexis:
https://github.com/18F/identity-idp/blob/main/app/services/doc_auth/lexis_nexis/request.rb#L20-L21
Versus what we handle in the RiscDeliveryJob: https://github.com/18F/identity-idp/blob/main/app/jobs/risc_delivery_job.rb#L6-L14
Mostly rhetorically, are these differences intentional? 🤷♂️ I wonder if it would make sense to have an app-wide FATAL_FARADAY_ERRORS or something?
This is way out of scope for your PR obviously.
| Faraday::TimeoutError, | ||
| Faraday::ClientError, | ||
| Faraday::ParsingError => e | ||
| raise ApiClientError, e.message |
There was a problem hiding this comment.
I'm inferring that the idea here is that if the request fails, it's not a big deal because we can either manually re-run it, or just get it next week, versus implementing lots of retry logic? (I'm fine with that, just confirming the plan.)
When Socure responds to requests against its identity verification APIs it includes "reason codes" to describe why a transaction may have failed. These reason codes map back to descriptions that are available in the Reason Code API.
Socure has suggested to us that we maintain our own copy of their reason codes. This commit adds a background job that will create/update a copy of the Socure reason codes and descriptions in our database on a weekly basis.
The job fetches the list of codes from the API. It adds any new codes that do not exist in the database. It deactivates any codes that we have in the database that we do not observe in the response.
The job will log the results when it is done. This includes:
This change includes a
idv_socure_reason_code_download_enabledconfig that controls whether this feature is active. It defaults tofalsesince we are not currently utilizing Socure in production.