Dataflow Templates 2024-10-16-00_RC00

Release week of 2024-10-14

Dataflow Templates 2024-10-15-00_RC00

Release Week of 10/15/2024

Improvements

• Adding JSON enum to userOption for MongoDB templates (#1910)
• Add warnings for excess tables (#1900)
• Add several types to PGAdapter (#1887)
• Reducing MySQL SchemaDiscovery error timeout (#1899)
• Update live terraform templates to support end to end migration (#1889)

Bug Fixes

• Fixed extra table in session file (#1928)
• Fixing wrong timestamp logical type conversion for datastream to bigquery (#1922)

Contributors

• mrtkshi
• Deep1998
• VardhanThigle
• thiagotnunes
• bharadwaj-aditya
• yair-harel

Dataflow Templates 2024-10-14-00_RC00

Release week of 2024-10-14

Dataflow Templates 2024-10-11-00_RC00

Release week of 2024-10-07

Dataflow Templates 2024-10-01-00_RC00

Release week of 2024-09-30

Dataflow Templates 2024-09-30-00_RC00

Release week of 2024-09-30

Dataflow Templates 2024-09-26-00_RC00

Release week of 2024-09-23

[DO NOT USE] Test Release

Simplify uw-tests logic (#1884)

Signed-off-by: Jeffrey Kinard <[email protected]>

Dataflow Templates 2024-09-19-00_RC00

Release Week of 9/21/24

Note: This release is being rolled out. It may not be in your region yet.

Improvements

• Add JSON userOption to MongoDbToBigQuery templates (#1855)
• DlqRetry for bulk with the current retry flow (#1870)
• Support populating Spanner source database id, instance id and a custom metadata field in Spanner CDC to Pub Sub template (#1769)
• feat: update pg index discovery query for 9.4+ (#1863)

Bug Fixes

• Fix for custom logical types in DatastreamIO (#1851)
• Fixing isue of discovering unsigned types (#1864)
• Make utc named timezone to "+00:00" (#1860)
• fix: PG index cardinality estimate (#1869)

Contributors

Danny McCormick
Deep1998
Florent Biville
Jeff Kinard
Shuran Zhang
Thiago Nunes
Vardhan Vinay Thigle
Yi Hu
aksharauke
darshan-sj
shreyakhajanchi

Dataflow Templates 2024-09-10-00_RC01

Release Week of 9/10/24

Improvements

• Add support for OAuth authentication to schema registry in the Kafka to Big Query Flex and Kafka to GCS flex templates. (#1816)
• feat: Adds SourceDBToSpanner PostgreSQL Adapter (#1754)

Bug Fixes

• BigTable to Parquet: only set numShards for >0 (#1820)
• Fix for custom logical types in DatastreamIO (#1767)

Contributors

Astha Mohta
Damon
Deep1998
Dylan Hercher
Jeff Kinard
Ron Gal
Shuran Zhang
Thiago Nunes
Vardhan Vinay Thigle
Yi Hu
aksharauke
atask-g
darshan-sj
fozzie15
shreyakhajanchi
yair-harel