Data not syncing from Kafka source to Bigquery destination

Slack Knowledge Base Q&A Connector Questions
, , , ,
1

Summary

Data is not syncing from Kafka source with JSON message format and SASL SSL protocol to Bigquery destination in Airbyte deployed on Kubernetes. The sync summary shows no records loaded.


Question

Hi Community !
I am using kafka source with message format JSON ,protocol SASL SSL and destination as Bigquery. The problem here is that data is not syncing it is giving no record loaded. I am using airbyte deployed on kubernetes and version is 1.1.0. intially data was syncing but after some time it stopped.
Anyone here who has ever faced the same issue??

  "status" : "completed",
  "startTime" : 1731476134916,
  "endTime" : 1731476150677,
  "totalStats" : {
    "bytesCommitted" : 0,
    "bytesEmitted" : 0,
    "destinationStateMessagesEmitted" : 0,
    "destinationWriteEndTime" : 1731476150574,
    "destinationWriteStartTime" : 1731476135151,
    "meanSecondsBeforeSourceStateMessageEmitted" : 0,
    "maxSecondsBeforeSourceStateMessageEmitted" : 0,
    "meanSecondsBetweenStateMessageEmittedandCommitted" : 0,
    "recordsEmitted" : 0,
    "recordsCommitted" : 0,
    "replicationEndTime" : 1731476150673,
    "replicationStartTime" : 1731476134916,
    "sourceReadEndTime" : 1731476143219,
    "sourceReadStartTime" : 1731476135152,
    "sourceStateMessagesEmitted" : 0
  },
  "streamStats" : [ ],
  "performanceMetrics" : {
    "processFromSource" : {
      "elapsedTimeInNanos" : 0,
      "executionCount" : 0,
      "avgExecTimeInNanos" : "NaN"
    },
    "readFromSource" : {
      "elapsedTimeInNanos" : 8040281611,
      "executionCount" : 54,
      "avgExecTimeInNanos" : 1.488941039074074E8
    },
    "processFromDest" : {
      "elapsedTimeInNanos" : 0,
      "executionCount" : 0,
      "avgExecTimeInNanos" : "NaN"
    },
    "writeToDest" : {
      "elapsedTimeInNanos": 0,
      "executionCount": 0,
      "avgExecTimeInNanos": "NaN"
    
    }```


<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1731478205978559) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["kafka-source", "json-message-format", "sasl-ssl", "bigquery-destination", "airbyte", "kubernetes", "data-syncing-issue"]
</sub>