Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Database work - Fix issues with affiliated committee data #5866

Open
4 tasks
tmpayton opened this issue Jun 11, 2024 · 0 comments
Open
4 tasks

Database work - Fix issues with affiliated committee data #5866

tmpayton opened this issue Jun 11, 2024 · 0 comments

Comments

@tmpayton
Copy link
Contributor

What we’re after

There are data issues blocking the creation of an affiliated committee mv. When we union the three views that contain affiliated committee data there are duplicates. This is due to differences in the data between the views. The research ticket contains an example and an excel spreadsheet that lists the 79 duplicates.

Related ticket(s)

Action item(s)

  • Fix data issues that are causing duplicates within the affiliated committee data

Completion criteria

  • There are no duplicates with affiliated committee data when unioning fec_form_1s_vw, fec_vsum_f1_vw, and fec_vsum_f1z_vw

References/resources/technical considerations

Sql snippet that identifies duplicates:

with aff_cmte_data as 
(SELECT fec_vsum_f1_vw.cmte_id,fec_vsum_f1_vw.affiliated_cmte_id, fec_vsum_f1_vw.affiliated_cmte_nm, fec_vsum_f1_vw.receipt_dt, fec_vsum_f1_vw.affiliated_relationship_cd
           FROM fec_vsum_f1_vw
          WHERE fec_vsum_f1_vw.affiliated_cmte_id IS NOT NULL AND fec_vsum_f1_vw.most_recent = 'Y'
		UNION        
         SELECT fec_form_1s_vw.cmte_id, fec_form_1s_vw.affiliated_cmte_id, fec_form_1s_vw.affiliated_cmte_nm, fec_form_1s_vw.receipt_dt, fec_form_1s_vw.affiliated_relationship_cd
           FROM fec_form_1s_vw
          WHERE fec_form_1s_vw.affiliated_cmte_id IS NOT NULL AND fec_form_1s_vw.most_recent_filing_flag = 'Y'
		UNION
         SELECT fec_vsum_f1z_vw.cmte_id, fec_vsum_f1z_vw.affiliated_cmte_id, fec_vsum_f1z_vw.affiliated_cmte_nm, fec_vsum_f1z_vw.receipt_dt, fec_vsum_f1z_vw.affiliated_relationship_cd
           FROM public.fec_vsum_f1z_vw
          WHERE fec_vsum_f1z_vw.affiliated_cmte_id IS NOT null AND fec_vsum_f1z_vw.most_recent = 'Y')
 select cmte_id, affiliated_cmte_id, count(*) from aff_cmte_data group by cmte_id, affiliated_cmte_id having count(*) > 1;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

1 participant