Summary
When trying to connect S3 as a source, encountering an error due to an invalid bucket name format. The error message indicates that the bucket name must match a specific regex pattern.
Question
Hello Team: Trying to connect S3 as my source - I exactly followed as per the instructions but getting error message as below, not sure how to trouble shoot. My AWS User has Full s3 access but still not working. Pls refer the error attached here.Configuration check failed
[‘Traceback (most recent call last):\n File “/airbyte/integration_code/source_s3/v4/stream_reader.py”, line 131, in get_matching_files\n for remote_file in self._page(s3, globs, self.config.bucket, current_prefix, seen, logger):\n File “/airbyte/integration_code/source_s3/v4/stream_reader.py”, line 193, in _page\n response = s3.list_objects_v2(Prefix=prefix, **kwargs) if prefix else s3.list_objects_v2(**kwargs)\n File “/usr/local/lib/python3.9/site-packages/botocore/client.py”, line 553, in _api_call\n return self._make_api_call(operation_name, kwargs)\n File “/usr/local/lib/python3.9/site-packages/botocore/client.py”, line 946, in _make_api_call\n api_params = self._emit_api_params(\n File “/usr/local/lib/python3.9/site-packages/botocore/client.py”, line 1072, in _emit_api_params\n self.meta.events.emit(\n File “/usr/local/lib/python3.9/site-packages/botocore/hooks.py”, line 412, in emit\n return self._emitter.emit(aliased_event_name, **kwargs)\n File “/usr/local/lib/python3.9/site-packages/botocore/hooks.py”, line 256, in emit\n return self._emit(event_name, kwargs)\n File “/usr/local/lib/python3.9/site-packages/botocore/hooks.py”, line 239, in _emit\n response = handler(kwargs)\n File “/usr/local/lib/python3.9/site-packages/botocore/handlers.py”, line 288, in validate_bucket_name\n raise ParamValidationError(report=error_msg)\nbotocore.exceptions.ParamValidationError: Parameter validation failed:\nInvalid bucket name “airbyte-testdata/testdata/”: Bucket name must match the regex “[1]{1,255}$” or be an ARN matching the regex “^arn:(aws).:(s3|s3-object-lambda):[a-z\-0-9]:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-.]{1,63}$|^arn:(aws).*:s3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}$”\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File “/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/availability_strategy/default_file_based_availability_strategy.py”, line 81, in _check_list_files\n file = next(iter(stream.get_files()))\n File “/airbyte/integration_code/source_s3/v4/stream_reader.py”, line 143, in get_matching_files\n self._raise_error_listing_files(globs, exc)\n File “/airbyte/integration_code/source_s3/v4/stream_reader.py”, line 147, in _raise_error_listing_files\n raise ErrorListingFiles(\nairbyte_cdk.sources.file_based.exceptions.ErrorListingFiles: Error listing files. Please check the credentials provided in the config and verify that they provide permission to list files. Contact Support if you need assistance.\nsource=s3 bucket=airbyte-testdata/testdata/ globs=[''] endpoint=\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File “/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/availability_strategy/default_file_based_availability_strategy.py”, line 62, in check_availability_and_parsability\n file = self._check_list_files(stream)\n File “/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/availability_strategy/default_file_based_availability_strategy.py”, line 87, in _check_list_files\n raise CheckAvailabilityError(FileBasedSourceError.ERROR_LISTING_FILES, stream=stream.name) from exc\nairbyte_cdk.sources.file_based.exceptions.CheckAvailabilityError: Error listing files. Please check the credentials provided in the config and verify that they provide permission to list files. Contact Support if you need assistance.\nstream=dummy\n’]
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.
Join the conversation on Slack
["s3-connector", "aws-s3", "invalid-bucket-name", "error-message", "troubleshooting"]a-zA-Z0-9.\-_ ↩︎