Summary
User is facing an issue with the Google Sheets connector in Airbyte Cloud as it requires access to all Google Drive files, which is not suitable for the organization. User is seeking clarification on the issue.
Question
I am testing Airbyte Cloud to see if it is possible to move everything from (a more expensive tool) to Airbyte Cloud, but I got stopped at the very first connector I tested, namely Google Sheet.
Airbyte Cloud asks to “See and download all your Google Drive files.” That is obviously not something our organization can do.
Are there something I’m missing, am I doing something wrong here?
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.
Join the conversation on Slack
["google-sheets-connector", "airbyte-cloud", "google-drive", "organization"]
This is the nature of the OAuth permissions being granted. But if you want to limit scope, you could create a Service Account or dedicated user and share the files to that with whatever more restricted permissions you want, and then authorize that.
I believe that others like Fivetran sort of force you into this by having you share the file with THEIR service account email address. Some people dislike this for similar reasons (because now you have some third-party service account authorized that doesn’t loose access like an offboarded employee account would). So pros and cons there.
But I personally always prefer using service account users when doing data integrations and only giving them access to the scopes of data I want them to see.
You’ll see this second option on the Authentication section of the Source setup for Google Sheets:
if you’re already using your own service account for the old service (versus sharing to their service account email) . . . you can likely generate a new keypair for the same service account so it has access to the same things.
Then, when you’re ready to disable the old one, you can just nuke their key and not have to re-permission your access to every individual asset
Thanks <@U035912NS77>!
A service account is not feasible for this POC. It would require to get work done by other departments, plus we are not a big GCP house, so every time we need to do something in the GCP it is cumbersome.
A feature request could be that you could share individual Google Sheets, like you can in (a more expensive tool)
How are you currently handing this in the other tool . . . are you sharing the sheet with their service account?
The other tool has a generated email that you can shere the sheet with.
So from that standpoint you could do the same thing—use an additional Google account (whether in or out of the org, could even be a Gmail address for a POC) or service account email (again, inside or outside of the org—it doesn’t actually matter, it could even be a free Sandbox account)
. . . then use that account to authenticate with. That way it limits the scope of the access to just the files explicitly shared.
Unfortunately the OAuth scopes are account-level, there’s no way to authenticate to just a single file. And a lot of orgs don’t like the idea of sharing to third-party service accounts either, so I imagine that for the Airbyte folks it’s sort of a catch-22 from an implementation standpoint.
If you want similar functionality, you could put in a feature request with the Airbyte team. but this would obviously only apply to Airbyte Cloud since it would need to be their service account. I can’t speak for them on when or if that would be implemented, but I don’t think your case of departmental isolation and IT-centric challenges to things like this is unique, so I’d make sure to give a little backstory there as I think it helps the case.
But in the meantime, the only workarounds I can think of for your specific use case would be using another Google account (which depending on the org is sometimes easier to acquire than a service account), a stand-alone Gmail account (or Google account tied to literally any email), or a service account outside your primary org to authenticate.
If none of those are an option and you have to fly this up the flagpole internally, the <Google Sheets | Airbyte Documentation on the Sheets connector> for service accounts are accurate (it’s only a few clicks, but I know that doesn’t always matter in different IT environments 
)
And for what it’s worth, you can know if you’re authenticating to the other tool’s service account if the “generated email” you mention ends in *.<http://iam.gserviceaccount.com|iam.gserviceaccount.com>
While it doesn’t change anything, this may be useful if your internal team pushes back on authenticating to a service account . . . and you can say that you already are, just not one that you control.