Google Search Console - Service Account Key Authentication Failure

danieldiamond · October 12, 2022, 2:04am

Error when attempting to authenticate the google search console source using the Service Account Key Authentication.

Version 0.1.16

"Unable to connect to Google Search Console API with the provided credentials - KeyError('siteEntry')"

Note: Connection was working fine with the OAuth connection. i.e. dates are fine, domains are fine, etc.

marcosmarxm · October 12, 2022, 3:02pm

Hello there! You are receiving this message because none of your fellow community members has stepped in to respond to your topic post. (If you are a community member and you are reading this response, feel free to jump in if you have the answer!) As a result, the Community Assistance Team has been made aware of this topic and will be investigating and responding as quickly as possible.
Some important considerations that will help your to get your issue solved faster:

It is best to use our topic creation template; if you haven’t yet, we recommend posting a followup with the requested information. With that information the team will be able to more quickly search for similar issues with connectors and the platform and troubleshoot more quickly your specific question or problem.
Make sure to upload the complete log file; a common investigation roadblock is that sometimes the error for the issue happens well before the problem is surfaced to the user, and so having the tail of the log is less useful than having the whole log to scan through.
Be as descriptive and specific as possible; when investigating it is extremely valuable to know what steps were taken to encounter the issue, what version of connector / platform / Java / Python / docker / k8s was used, etc. The more context supplied, the quicker the investigation can start on your topic and the faster we can drive towards an answer.
We in the Community Assistance Team are glad you’ve made yourself part of our community, and we’ll do our best to answer your questions and resolve the problems as quickly as possible. Expect to hear from a specific team member as soon as possible.

Thank you for your time and attention.
Best,
The Community Assistance Team

harshith · October 12, 2022, 3:55pm

Hey @danieldiamond I will look into this get back.

danieldiamond · October 13, 2022, 12:08am

@harshith this siteEntry error is also occurring when trying to use the OAuth credentials

danieldiamond · October 13, 2022, 12:11am

Possible offending line added 2 days ago

github.com

airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-google-search-console/source_google_search_console/source.py#L109


      
          @staticmethod
          def validate_site_urls(site_urls, auth):
              if isinstance(auth, ServiceAccountAuthenticator):
                  request = auth(requests.Request(method="GET", url="https://www.googleapis.com/webmasters/v3/sites"))
                  with requests.Session() as s:
                      response = s.send(s.prepare_request(request))
              else:
                  response = requests.get("https://www.googleapis.com/webmasters/v3/sites", headers=auth.get_auth_header())
              response_data = response.json()
          
          
    remote_site_urls = {s["siteUrl"] for s in response_data["siteEntry"]}
              invalid_site_url = set(site_urls) - remote_site_urls
              if invalid_site_url:
                  raise InvalidSiteURLValidationError(f'The following URLs are not permitted: {", ".join(invalid_site_url)}')
          
          
def streams(self, config: Mapping[str, Any]) -> List[Stream]:
              """
              :param config: A Mapping of the user input configuration as defined in the connector spec.
              """
              config = self._validate_and_transform(config)
              stream_config = self.get_stream_kwargs(config)

danieldiamond · October 13, 2022, 12:17am

Confirming when I revert to GSC version 0.1.14 I dont experience this siteEntry Key error.

However, I think this is a misleading error. I believe the credentials I was inputting were invalid, I have managed to successfully connect now with both old and new versions.

harshith · October 13, 2022, 4:05pm

Got it. Thanks a lot for checking this out.

brickel · November 4, 2022, 3:51pm

Hey,

I am on Airbyte 0.40.16 with connector version 0.1.18.

I do get the same error message:

The connection tests failed.
"Unable to connect to Google Search Console API with the provided credentials - KeyError('siteEntry')"

I am using an existing service account that I have setup for Google Ads. I grant it “Domain-wide Delegation ” via the Workspace Admin Console. Then I create a new key and use the service account’s email address together with the new JSON key in the Auth.

Was the issue resolved?

harshith · November 10, 2022, 9:09am

Hey as pointed out by Daniel maybe not. Could you create a github issue on this so that team can fix it?

brickel · November 11, 2022, 9:58am

Well if Daniel is correct than I would need some help/guidance in figuring out what is going on.
Maybe the “Setup Guide” could be revised?

In the meantime I have created a github issue:

github.com/airbytehq/airbyte

Google Search Console - Service Account Key Authentication Failure

opened 09:56AM - 11 Nov 22 UTC

sbrickel-nimble

type/bug community needs-triage team/triage team/use autoteam

## Environment - **Airbyte version**: 0.40.16 - **OS Version / Instance**: Debian GNU/Linux 10 VM instance - **Deployment**: Docker - **Source Connector and version**: Google Search Console 0.1.18 - **Destination Connector and version**: BigQuery 1.2.5 - **Step where error happened**: Setup new connection - ## Current Behavior The connection tests failed. "Unable to connect to Google Search Console API with the provided credentials - KeyError('siteEntry')" ## Expected Behavior Setting up connection ## Logs NA ## Steps to Reproduce 1. Setup GSC as described in Setup Guide 2. Click "Setup source" 3. ## Are you willing to submit a PR? Sure

marcosmarxm · November 11, 2022, 9:59am

Well if Daniel is correct than I would need some help/guidance in figuring out what is going on.
Maybe the “Setup Guide” could be revised?

In the meantime I have created a github issue:

github.com/airbytehq/airbyte

Google Search Console - Service Account Key Authentication Failure

opened 09:56AM - 11 Nov 22 UTC

sbrickel-nimble

type/bug community needs-triage team/triage team/use autoteam

rc1112 · April 11, 2023, 6:45pm

Hello all

I still got the same error below with the latest update. I think sc-domain:mydoamin.com is correct. Can anyone help with my 1st pipeline

Topic		Replies	Views
Google Search Console UI Bad Config Connector Questions & Issues source-google-search-console , getting-started , connectors	9	380	July 14, 2022
Hubspot failing on sync Connector Questions & Issues connectors	2	113	December 23, 2022
Amazon ADS source KeyError Connector Questions & Issues connectors	10	245	July 14, 2022
No records, no list for google ads Connector Questions & Issues source-google-ads , normalization , getting-started , data-loading , source-bigquery	2	216	December 14, 2022
WooCommerce - Not authorizing for API url during sync Connector Questions & Issues connectors	5	259	October 21, 2022

Google Search Console - Service Account Key Authentication Failure

Google Search Console - Service Account Key Authentication Failure

Related Topics