Edward Gao (edgao) · 2024-02-15T00:17:50Z

incorporates changes from Gireesh Sreepathi (@gisripa) 's #35342 as well as all the stacked CDK PRs.

from #35342 - move all of the schema change-detection code from sqlgenerator into destinationhandler. Also split out a BigqueryDestinationHandlerTest (unit test) class from the preexisting BigquerySqlGeneratorTest.

as noted in code comments - bigquery doesn't like frequent small updates on tables, so not implementing a true state table.

vercel · 2024-02-15T00:17:52Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Mar 4, 2024 7:14pm

Edward Gao (edgao) · 2024-02-15T00:18:04Z

Destination bigquery: upgrade cdk #35315 👈
Java CDK: Staging destinations include timezone in extracted_at in CSV file #35313 : 3 other dependent PRs (#35308 , #35316 , #35528 )
DV2 destinations: Build DestinationState / Migration framework #35303
DV2 TyperDeduper: Extract migrations to separate method #35376
Destinations CDK: Refactor T+D to gather required world state upfront #35342 : 3 other dependent PRs (#35354 , #35385 , #35456 )
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join Edward Gao (@edgao) and the rest of your teammates on Graphite

github-actions · 2024-02-15T00:18:14Z

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

PR name follows PR naming conventions
Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan.
Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
You've updated the connector's metadata.yaml file any other relevant changes, including a breakingChanges entry for major version bumps. See metadata.yaml docs
Secrets in the connector's spec are annotated with airbyte_secret
All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
Migration guide updated in docs/integrations/<source or destination>/<name>-migrations.md with an entry for the new version, if the version is a breaking change. See migration guide example
If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

Check for hidden checklists in your PR description
Toggle the github label checklist-action-run on/off to re-run the checklist CI.

Signed-off-by: Gireesh Sreepathi <gisripa@gmail.com>

This was referenced Feb 15, 2024

DV2 destinations: Build DestinationState / Migration framework #35303

Merged

Destination Snowflake: Write extracted_at in UTC #35308

Merged

Octavia Squidington III (octavia-squidington-iii) added the area/connectors Connector related issues label Feb 15, 2024

Edward Gao (edgao) mentioned this pull request Feb 15, 2024

Java CDK: Staging destinations include timezone in extracted_at in CSV file #35313

Merged

Octavia Squidington III (octavia-squidington-iii) added the connectors/destination/bigquery label Feb 15, 2024

Edward Gao (edgao) marked this pull request as ready for review February 15, 2024 00:18

Edward Gao (edgao) requested a review from a team as a code owner February 15, 2024 00:18

Edward Gao (edgao) mentioned this pull request Feb 15, 2024

Destination redshift: Upgrade cdk #35316

Merged

Edward Gao (edgao) marked this pull request as draft February 15, 2024 15:28

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone_bigquery branch from 5c1421f to 76bc3be Compare February 15, 2024 15:29

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch from 3829bea to bbe9cde Compare February 21, 2024 00:28

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone_bigquery branch 2 times, most recently from dbf54d6 to 7ff3c33 Compare February 21, 2024 16:11

Octavia Squidington III (octavia-squidington-iii) added the CDK Connector Development Kit label Feb 21, 2024

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch from bbe9cde to 9f7d650 Compare February 21, 2024 16:11

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone_bigquery branch from 7ff3c33 to bcabf0c Compare February 21, 2024 20:34

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch from 9f7d650 to 863ef4a Compare February 21, 2024 20:34

Octavia Squidington III (octavia-squidington-iii) removed the CDK Connector Development Kit label Feb 21, 2024

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone_bigquery branch from bcabf0c to dd5100a Compare February 21, 2024 20:45

Octavia Squidington III (octavia-squidington-iii) added the CDK Connector Development Kit label Feb 21, 2024

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch from 863ef4a to 74528dc Compare February 21, 2024 20:45

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone_bigquery branch from dd5100a to f095f49 Compare February 21, 2024 21:32

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch from 74528dc to eaad0b5 Compare February 21, 2024 21:32

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone_bigquery branch from f095f49 to c352217 Compare February 21, 2024 21:42

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch from eaad0b5 to 9e0fabb Compare February 21, 2024 21:43

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone_bigquery branch from c352217 to c4b5368 Compare February 21, 2024 21:44

Octavia Squidington III (octavia-squidington-iii) removed the CDK Connector Development Kit label Feb 21, 2024

Octavia Squidington III (octavia-squidington-iii) added connectors/source/paypal-transaction connectors/source/pinterest connectors/source/salesforce connectors/source/sendgrid labels Feb 22, 2024

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch from 1a73b86 to 68b2fff Compare February 22, 2024 23:26

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone_bigquery branch from e06e166 to 936cf1b Compare February 22, 2024 23:44

Octavia Squidington III (octavia-squidington-iii) removed area/documentation Improvements or additions to documentation normalization labels Feb 22, 2024

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch from 68b2fff to cc85121 Compare February 22, 2024 23:45

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone_bigquery branch 2 times, most recently from 327f2c7 to 5667569 Compare February 23, 2024 00:54

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch from cc85121 to d692960 Compare February 23, 2024 00:55

Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone_bigquery branch from 5667569 to 4bc6385 Compare February 23, 2024 17:14

Edward Gao (edgao) and others added 10 commits March 4, 2024 11:01

upgrade cdk

1ed06ca

start fixing build

ac35693

fix build

3361320

format

c456d09

switch to sql

8cc7e2c

remove state handling

564f57b

disable test

c13c740

format

b73ecdd

fix compilation post rebase

c001be8

version logistics

64fb4cb

Signed-off-by: Gireesh Sreepathi <gisripa@gmail.com>

Gireesh Sreepathi (gisripa) approved these changes Mar 4, 2024

View reviewed changes

Navigation Menu

Uh oh!

Destination bigquery: upgrade cdk - #35315