I’ve tried the conversion and it doesn’t work. As regards reformatting the file that’s not practical. I need Airbyte to do this autonomously.
Here’s some detail.
Google drive links to file uploaded in both xls and xlsx formats:
• https://docs.google.com/spreadsheets/d/1BgpLakMqrLP8ut-XTdULAbCVh2NPlk8y/edit?usp=drive_link&ouid=116900388473101210455&rtpof=true&sd=true|dcc-private-hire-vehicle-public-register-april2024.xlsx
• https://docs.google.com/spreadsheets/d/1Z3FDIeunjJA1pPeFJI99-7PSy9EYaiM0/edit?usp=drive_link&ouid=116900388473101210455&rtpof=true&sd=true|dcc-private-hire-vehicle-public-register-april2024.xls
Steps I’ve carried out:
- Uploaded both an xls and xlsx to two different files stored on Google Drive. Permissions set to allow anyone with the link access.
- Setup a new connection to the file using the file connector (xls file). Failed. Error “Configuration check failed. … Failed to load please use the Official Google Sheets Source connector”
- Setup a new connection to the file using the file connector (xlsx file). Failed. Same error.
- Setup a new conection using the Google Sheets connector to the xlsx file
- Failed. Detailed error below.
Reformatting request - This isn’t practical. It necessitates downloading a file manually, modifying the file manually, saving it as an xlsx, uploading it to Google Drive etc. All of this defeats the purpose of using an ELT tool like Airbyte to automate the process. Sorry, I don’t wish to sound mean but it should be obvious this isn’t a solution anyone can use.
Security comment - When using the Google sheets connector I have to authenticate and give permission to See and download all your Google Drive files.. For a commercial use case this isn’t practical. I’ve set permissions on the file to make it viewable to anyone with the link. That means Airbyte can see the file without the need for full access to all other files. Why is this not enough? It results in a far more secure solution for everyone.
Configuration check failed
Config error:
Learn more
Internal message: Config error:
Failure origin: source
Failure type: config_error
Stacktrace
File “/usr/local/lib/python3.9/site-packages/googleapiclient/_helpers.py”, line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File “/usr/local/lib/python3.9/site-packages/googleapiclient/http.py”, line 938, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 400 when requesting https://sheets.googleapis.com/v4/spreadsheets/1BgpLakMqrLP8ut-XTdULAbCVh2NPlk8y?includeGridData=false&alt=json returned “This operation is not supported for this document”. Details: “This operation is not supported for this document”>
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “/airbyte/integration_code/main.py”, line 8, in <module>
run()
File “/airbyte/integration_code/source_google_sheets/run.py”, line 15, in run
launch(source, sys.argv[1:])
File “/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py”, line 235, in launch
for message in source_entrypoint.run(parsed_args):
File “/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py”, line 115, in run
yield from map(AirbyteEntrypoint.airbyte_message_to_string, self.check(source_spec, config))
File “/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py”, line 139, in check
check_result = self.source.check(self.logger, config)
File “/airbyte/integration_code/source_google_sheets/source.py”, line 62, in check
raise AirbyteTracedException(
airbyte_cdk.utils.traced_exception.AirbyteTracedException: Config error:
?