8000
Skip to content

Source S3: validate CSV read options and convert options - #22550

Merged
Denys Davydov (davydov-d) merged 6 commits into
masterfrom
ddavydov/#1467-source-s3-validate-config
Feb 9, 2023
Merged

Source S3: validate CSV read options and convert options#22550
Denys Davydov (davydov-d) merged 6 commits into
masterfrom
ddavydov/#1467-source-s3-validate-config

Conversation

@davydov-d
@davydov-d Denys Davydov (davydov-d) commented Feb 8, 2023
Copy link
Copy Markdown
Contributor

What

https://github.com/airbytehq/oncall/issues/1467

How

Add validation for CSV read options and convert options

@octavia-squidington-iii Octavia Squidington III (octavia-squidington-iii) added area/connectors Connector related issues area/documentation Improvements or additions to documentation labels Feb 8, 2023
@davydov-d
Denys Davydov (davydov-d) commented Feb 8, 2023 •
edited by github-actions Bot
Loading
Copy link
Copy Markdown
Contributor Author

/test connector=connectors/source-s3

🕑 connectors/source-s3 https://github.com/airbytehq/airbyte/actions/runs/4122734019
❌ connectors/source-s3 https://github.com/airbytehq/airbyte/actions/runs/4122734019
🐛 https://gradle.com/s/vcprrdt5deqmq

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs2] - Failed: Please check...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:98: The previous and actual specifications are identical.
SKIPPED [6] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:507: The previous and actual discovered catalogs are identical.
============= 1 failed, 107 passed, 7 skipped in 259.51s (0:04:19) =============

@davydov-d
Denys Davydov (davydov-d) commented Feb 8, 2023
Copy link
Copy Markdown
Contributor Author

☝️ the failing test is not related. It's about the actual and expected schema mismatch (avro)

@davydov-d
Denys Davydov (davydov-d) commented Feb 9, 2023 •
edited by github-actions Bot
Loading
Copy link
Copy Markdown
Contributor Author

/test connector=connectors/source-s3

🕑 connectors/source-s3 https://github.com/airbytehq/airbyte/actions/runs/4132391051
❌ connectors/source-s3 https://github.com/airbytehq/airbyte/actions/runs/4132391051
🐛 https://gradle.com/s/cdhjjau5p6duk

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs2] - Failed: Please check...
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:98: The previous and actual specifications are identical.
SKIPPED [6] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:507: The previous and actual discovered catalogs are identical.
============= 1 failed, 107 passed, 7 skipped in 258.65s (0:04:18) =============

@davydov-d
Denys Davydov (davydov-d) commented Feb 9, 2023 •
edited by github-actions Bot
Loading
Copy link
Copy Markdown
Contributor Author

/publish connector=connectors/source-s3 run-tests=false

🕑 Publishing the following connectors:
connectors/source-s3
https://github.com/airbytehq/airbyte/actions/runs/4135402872


Connector Did it publish? Were definitions generated?
connectors/source-s3 ✅ ✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

@airbyteio
Airbyte (airbyteio) temporarily deployed to more-secrets February 9, 2023 15:02 — with GitHub Actions Inactive
@airbyteio
Airbyte (airbyteio) temporarily deployed to more-secrets February 9, 2023 15:02 — with GitHub Actions Inactive
@github-actions
github-actions Bot commented Feb 9, 2023
Copy link
Copy Markdown
Contributor

Airbyte Code Coverage

There is no coverage information present for the Files changed

Total Project Coverage 24.66% ❌

@davydov-d
Denys Davydov (davydov-d) temporarily deployed to more-secrets February 9, 2023 15:51 — with GitHub Actions Inactive
@davydov-d
Denys Davydov (davydov-d) temporarily deployed to more-secrets February 9, 2023 15:51 — with GitHub Actions Inactive
@davydov-d
Denys Davydov (davydov-d) merged commit fcd3b03 into master Feb 9, 2023
@davydov-d
Denys Davydov (davydov-d) deleted the ddavydov/#1467-source-s3-validate-config branch February 9, 2023 16:27
@github-actions github-actions Bot mentioned this pull request Feb 9, 2023
Closed
This was referenced Feb 10, 2023
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lazebnyi Serhii Lazebnyi (lazebnyi) Awaiting requested review from lazebnyi
@pedroslopez Pedro S. Lopez (pedroslopez) Awaiting requested review from pedroslopez
@alafanechere Augustin (alafanechere) Awaiting requested review from alafanechere
1 more reviewer
@roman-yermilov-gl Roman Yermilov [GL] (roman-yermilov-gl) roman-yermilov-gl approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/s3

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@davydov-d @roman-yermilov-gl @octavia-squidington-iii @airbyteio
0