Kafka Source Data Sync Issue with BigQuery

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

User reports an issue with Kafka source not syncing data to BigQuery, showing no records loaded despite initial success. The setup involves JSON message format and SASL SSL protocol, deployed on Kubernetes version 1.1.0.


Question

Hi Community !
I am using kafka source with message format JSON ,protocol SASL SSL and destination as Bigquery. The problem here is that data is not syncing it is giving no record loaded. I am using airbyte deployed on kubernetes and version is 1.1.0. intially data was syncing but after some time it stopped.
Anyone here who has ever faced the same issue??

  "status" : "completed",
  "startTime" : 1731476134916,
  "endTime" : 1731476150677,
  "totalStats" : {
    "bytesCommitted" : 0,
    "bytesEmitted" : 0,
    "destinationStateMessagesEmitted" : 0,
    "destinationWriteEndTime" : 1731476150574,
    "destinationWriteStartTime" : 1731476135151,
    "meanSecondsBeforeSourceStateMessageEmitted" : 0,
    "maxSecondsBeforeSourceStateMessageEmitted" : 0,
    "meanSecondsBetweenStateMessageEmittedandCommitted" : 0,
    "recordsEmitted" : 0,
    "recordsCommitted" : 0,
    "replicationEndTime" : 1731476150673,
    "replicationStartTime" : 1731476134916,
    "sourceReadEndTime" : 1731476143219,
    "sourceReadStartTime" : 1731476135152,
    "sourceStateMessagesEmitted" : 0
  },
  "streamStats" : [ ],
  "performanceMetrics" : {
    "processFromSource" : {
      "elapsedTimeInNanos" : 0,
      "executionCount" : 0,
      "avgExecTimeInNanos" : "NaN"
    },
    "readFromSource" : {
      "elapsedTimeInNanos" : 8040281611,
      "executionCount" : 54,
      "avgExecTimeInNanos" : 1.488941039074074E8
    },
    "processFromDest" : {
      "elapsedTimeInNanos" : 0,
      "executionCount" : 0,
      "avgExecTimeInNanos" : "NaN"
    },
    "writeToDest" : {
      "elapsedTimeInNanos": 0,
      "executionCount": 0,
      "avgExecTimeInNanos": "NaN"
    
    }```


<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1731478205978559) if you want
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
['kafka-source', 'bigquery', 'data-sync', 'sasl-ssl', 'json-format', 'kubernetes', 'airbyte']
</sub>