Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement bulk upload for lookup-tables and case-data #764

Merged
merged 10 commits into from
Oct 1, 2024

Conversation

hunterachieng
Copy link
Contributor

@hunterachieng hunterachieng commented Sep 30, 2024

Summary

Implemented a bulk upload for allowing to upload and update lookup-tables and case-data.

Fixes # #751

AI Usage

Please disclose how you've used AI in this work (it's cool, we just want to know!):

  • Code generation (copilot but not intellisense)
  • Learning or fact checking
  • Strategy / design
  • Optimisation / refactoring
  • Translation / spellchecking / doc gen
  • Other
  • I have not used AI

You can read more details in our Responsible AI Policy

Review Checklist

Before merging, the reviewer should check the following items:

  • Does the PR do what it claims to do?
  • If this is a new adaptor, added the adaptor on marketing website ?
  • Are there any unit tests?
  • Is there a changeset associated with this PR? Should there be? Note that
    dev only changes don't need a changeset.
  • Have you ticked a box under AI Usage?

Copy link
Collaborator

@josephjclark josephjclark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of quick comments @hunterachieng but I'll let @mtuchi take a closer look at the code

Are we going to deprecate submitXls now?

packages/commcare/src/Adaptor.js Outdated Show resolved Hide resolved
packages/commcare/src/Adaptor.js Outdated Show resolved Hide resolved
Signed-off-by: Hunter Achieng <[email protected]>
Signed-off-by: Hunter Achieng <[email protected]>
Signed-off-by: Hunter Achieng <[email protected]>
Signed-off-by: Hunter Achieng <[email protected]>
hunterachieng and others added 3 commits October 1, 2024 13:41
Copy link
Collaborator

@mtuchi mtuchi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good @hunterachieng 👍🏽 , I have added unit test on bulk operation for checking the error if type is not case-data or lookup-table

@mtuchi mtuchi merged commit cff886e into feat/commcare-request Oct 1, 2024
2 checks passed
@mtuchi mtuchi deleted the feat/commcare-files-function branch October 1, 2024 11:26
mtuchi added a commit that referenced this pull request Oct 1, 2024
* feat: implement bulk upload for lookup-tables and case-data

Signed-off-by: Hunter Achieng <[email protected]>

* deleted imports not required

Signed-off-by: Hunter Achieng <[email protected]>

* fix: remove lookup-table plural

Signed-off-by: Hunter Achieng <[email protected]>

* reduce example and rename test

Signed-off-by: Hunter Achieng <[email protected]>

* update example

Signed-off-by: Hunter Achieng <[email protected]>

* remove try...catch

Signed-off-by: Hunter Achieng <[email protected]>

* fix bulk description

Signed-off-by: Hunter Achieng <[email protected]>

* implement changeset

Signed-off-by: Hunter Achieng <[email protected]>

* add unit test for bulk

---------

Signed-off-by: Hunter Achieng <[email protected]>
Co-authored-by: Emmanuel Evance <[email protected]>
mtuchi added a commit that referenced this pull request Oct 2, 2024
* feat:implement dynamic request function in commcare

Signed-off-by: Hunter Achieng <[email protected]>

* fix pr comments and add changeset

Signed-off-by: Hunter Achieng <[email protected]>

* feat: implement bulk upload for lookup-tables and case-data (#764)

* feat: implement bulk upload for lookup-tables and case-data

Signed-off-by: Hunter Achieng <[email protected]>

* deleted imports not required

Signed-off-by: Hunter Achieng <[email protected]>

* fix: remove lookup-table plural

Signed-off-by: Hunter Achieng <[email protected]>

* reduce example and rename test

Signed-off-by: Hunter Achieng <[email protected]>

* update example

Signed-off-by: Hunter Achieng <[email protected]>

* remove try...catch

Signed-off-by: Hunter Achieng <[email protected]>

* fix bulk description

Signed-off-by: Hunter Achieng <[email protected]>

* implement changeset

Signed-off-by: Hunter Achieng <[email protected]>

* add unit test for bulk

---------

Signed-off-by: Hunter Achieng <[email protected]>
Co-authored-by: Emmanuel Evance <[email protected]>

* format test

* bump version and update changelog

---------

Signed-off-by: Hunter Achieng <[email protected]>
Co-authored-by: Emmanuel Evance <[email protected]>
Copy link
Collaborator

@josephjclark josephjclark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @hunterachieng Just a couple of little comments for future reference.

Nothing here needs to be actioned, but maybe you could check the file name thing and raise an issue? I would be more comfortable if the file name was consistent across the API call. Even if it doesn't matter at the backend, it means the code looks buggy to anyone reading it

path = `/a/${domain}/fixtures/fixapi/`;
file = 'file-to-upload';
// append types and lookup-table name xlsx
Object.keys(json).forEach(sectionName => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for (key in json) { is probably a simply way to to looks like this (you can use async in it as well - not useful here but useful in other places)


const workbook = xlsx.utils.book_new();

if (type.toLowerCase() === 'lookup-table') {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type.toLowerCase() is duplicated. I'd probably prefer either a case-insensitive regex (maybe too complicated in this case), or type being saved out into a new variable and lowercased

});
} else if (type.toLowerCase() === 'case-data') {
path = `/a/${domain}/importer/excel/bulk_upload_api/`;
file = 'file';
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think file could just be a const file in both cases? The actual file name doesn't seem important. If it is important we probably need an option to take the filename from the user

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also we actually call the file data.xlsx anyway 🤔

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants