Trouble with Datadog connector for Datadog to BigQuery data transfer

Summary

Error message ‘time data does not match format %Y-%m-%dT%H:%M:%SZ’ when using Datadog connector to transfer data to BigQuery. User does not control time data format from Datadog.


Question

Has anyone here had success using the Datadog connector to bring data over from Datadog to BigQuery? I’m getting the below error that I’m not sure how to resolve since I don’t control the time data format for what comes from Datadog. . .
time data '' does not match format '%Y-%m-%dT%H:%M:%SZ'



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.

Join the conversation on Slack

["datadog-connector", "bigquery", "error", "time-format"]

Is that cloud or OSS?

Either way, what version of datadog connector are you using? Would you mind filing an issue for this?

I’d try opening datadog in builder and looking into the record filters. You might want to filter out records where the empty date is in the field you care about (likely the updated_at used as a cursor)

If you can create a reproduction workspace on Airbyte Cloud trial, I’d take a look later over the weekend

<@U065RJ879QT> this is on OSS. I created the issue in Github https://github.com/airbytehq/airbyte/issues/46549|here.
Version is 1.0.6
How do I open in builder? I don’t see the ability to fork that connector in the dropdown.
Happy to create the reproduction workspace if that is useful as well :slightly_smiling_face:

It looks like some data did come over at some point during the weekend. None of them have a “sync_date” populated in BigQuery tho, which is what it looks like is used as the cursor.