8000
Skip to content

Java CDK: Staging destinations include timezone in extracted_at in CSV file - #35313

Merged
Gireesh Sreepathi (gisripa) merged 2 commits into
masterfrom
edgao/staging_csv_timestamp_timezone
Mar 1, 2024
Merged

Java CDK: Staging destinations include timezone in extracted_at in CSV file#35313
Gireesh Sreepathi (gisripa) merged 2 commits into
masterfrom
edgao/staging_csv_timestamp_timezone

Conversation

@edgao
@edgao Edward Gao (edgao) commented Feb 14, 2024 •
edited
Loading
Copy link
Copy Markdown
Contributor

Modify how we write staging files, so that the extracted_at timestamp includes an offset. This fixes destination-snowflake writing extracted_at in local timezone instead of UTC. (see #35308 for the migration code.)

See stacked PRs for tests. I'm just relying on our T+D tests to catch any regressions, since they have assertions for extracted_at being correct.

@vercel
vercel Bot commented Feb 14, 2024 •
edited
Loading
Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Mar 1, 2024 7:11pm

@edgao Graphite App
Edward Gao (edgao) commented Feb 14, 2024 •
edited
Loading
Copy link
Copy Markdown
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join Edward Gao (@edgao) and the rest of your teammates on Graphite Graphite

@github-actions
github-actions Bot commented Feb 14, 2024
Copy link
Copy Markdown
Contributor

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

  • PR name follows PR naming conventions
  • Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan.
  • Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
  • You've updated the connector's metadata.yaml file any other relevant changes, including a breakingChanges entry for major version bumps. See metadata.yaml docs
  • Secrets in the connector's spec are annotated with airbyte_secret
  • All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
  • Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
  • Migration guide updated in docs/integrations/<source or destination>/<name>-migrations.md with an entry for the new version, if the version is a breaking change. See migration guide example
  • If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

  1. Check for hidden checklists in your PR description

  2. Toggle the github label checklist-action-run on/off to re-run the checklist CI.

@edgao
Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch from 027b08c to 45444b4 Compare February 14, 2024 23:40
@edgao Edward Gao (edgao) mentioned this pull request Feb 14, 2024
Merged
@edgao
Edward Gao (edgao) force-pushed the edgao/destination_state_table branch from b7b4c3d to 67f0f17 Compare February 15, 2024 00:14
@edgao
Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch from 350f326 to 9b0bca9 Compare February 15, 2024 00:14
@edgao Edward Gao (edgao) mentioned this pull request Feb 15, 2024
Merged
edgao
Edward Gao (edgao) commented Feb 15, 2024
View reviewed changes
Comment thread ...stinations/src/test/java/io/airbyte/cdk/integrations/destination/s3/csv/S3CsvWriterTest.java Outdated
2b95a13f-d54f-4370-a712-1c7bf2716190,"{""bar"":84}",%s\r
"""
f6767f7d-ce1e-45cc-92db-2ad3dfdd088e,"{""foo"":73}",1970-01-01T00:00:01.234Z\r
2b95a13f-d54f-4370-a712-1c7bf2716190,"{""bar"":84}",1970-01-01T00:00:02.345Z\r
@edgao Edward Gao (edgao) Feb 15, 2024
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note the Z offset suffix. Previously this was not present.

This was referenced Feb 15, 2024
8000
Merged
Merged
@edgao
Edward Gao (edgao) force-pushed the edgao/destination_state_table branch 5 times, most recently from 34a4fb3 to fa52885 Compare February 20, 2024 22:23
@edgao
Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch 3 times, most recently from 9f7d650 to 863ef4a Compare February 21, 2024 20:34
@edgao
Edward Gao (edgao) force-pushed the edgao/destination_state_table branch from be41e16 to b579660 Compare February 21, 2024 20:35
@edgao
Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch 2 times, most recently from 74528dc to eaad0b5 Compare February 21, 2024 21:32
@edgao
Edward Gao (edgao) force-pushed the edgao/destination_state_table branch 2 times, most recently from 3c448f8 to 9e3d10f Compare February 21, 2024 21:43
@edgao
Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch 2 times, most recently from 9e0fabb to d128fa1 Compare February 21, 2024 21:52
@edgao
Edward Gao (edgao) force-pushed the edgao/destination_state_table branch from b274ff7 to 75cc1f1 Compare February 23, 2024 19:12
@edgao
Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch from 96252de to da07c10 Compare February 23, 2024 19:12
@edgao
Edward Gao (edgao) force-pushed the edgao/destination_state_table branch from 75cc1f1 to 73f7fe4 Compare February 23, 2024 21:12
@edgao
Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch 2 times, most recently from a12202f to 51c992b Compare February 23, 2024 21:17
This was referenced Feb 23, 2024
Merged
Closed
@edgao
Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch from 51c992b to e944acd Compare February 23, 2024 21:41
@edgao Edward Gao (edgao) mentioned this pull request Feb 23, 2024
Merged
@edgao
Edward Gao (edgao) force-pushed the edgao/staging_csv_timestamp_timezone branch 4 times, most recently from 291e039 to e074811 Compare February 26, 2024 16:46
This was referenced Feb 26, 2024
Closed
Closed
Closed
@gisripa
Gireesh Sreepathi (gisripa) force-pushed the edgao/destination_state_table branch 5 times, most recently from 5022d37 to d8ce2d2 Compare February 29, 2024 19:35
@gisripa
Gireesh Sreepathi (gisripa) force-pushed the edgao/staging_csv_timestamp_timezone branch from e074811 to 51d3784 Compare February 29, 2024 19:38
@gisripa
Gireesh Sreepathi (gisripa) force-pushed the edgao/destination_state_table branch from d8ce2d2 to d158042 Compare March 1, 2024 18:35
@gisripa
Gireesh Sreepathi (gisripa) commented Mar 1, 2024 •
edited by github-actions Bot
Loading
Copy link
Copy Markdown
Contributor

/publish-java-cdk

🕑 https://github.com/airbytehq/airbyte/actions/runs/8116169407
✅ Successfully published Java CDK version=0.23.11!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers
@gisripa Gireesh Sreepathi (gisripa) gisripa approved these changes
@jbfbell Joe Bell (jbfbell) jbfbell approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

CDK Connector Development Kit connectors/destination/snowflake

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@edgao @gisripa @jbfbell @octavia-squidington-iii
0