Summary
The user is facing a java.lang.NullPointerException error when trying to run a sync with incremental streams between AWS DocumentDB and BigQuery using the MongoDB source connector in Airbyte Cloud. The error seems to be related to the resumeToken being null. The user has provided detailed logs and additional information about the setup.
Question
Hello. I am using Airbyte Cloud. I am trying to set a connection between AWS DocumentDB using the MongoDB source connector and BigQuery. When I try to run the sync with incremental streams, I get the following error:
java.lang.NullPointerException: Cannot invoke "org.bson.BsonDocument.containsKey(Object)" because "resumeToken" is null
Here are the full logs:
at io.debezium.connector.mongodb.ResumeTokens.getData(ResumeTokens.java:41)
at io.airbyte.integrations.source.mongodb.cdc.MongoDbDebeziumStateUtil.constructInitialDebeziumState(MongoDbDebeziumStateUtil.java:64)
at io.airbyte.integrations.source.mongodb.cdc.MongoDbCdcInitializer.createCdcIterators(MongoDbCdcInitializer.java:100)
at io.airbyte.integrations.source.mongodb.MongoDbSource.read(MongoDbSource.java:155)
at io.airbyte.cdk.integrations.base.IntegrationRunner.readSerial(IntegrationRunner.kt:313)
at io.airbyte.cdk.integrations.base.IntegrationRunner.runInternal(IntegrationRunner.kt:183)
at io.airbyte.cdk.integrations.base.IntegrationRunner.run(IntegrationRunner.kt:115)
at io.airbyte.integrations.source.mongodb.MongoDbSource.main(MongoDbSource.java:52)
,retryable=<null>,timestamp=1715772715958,streamDescriptor=<null>,additionalProperties={}], io.airbyte.config.FailureReason@3542bdaf[failureOrigin=source,failureType=<null>,internalMessage=Source process exited with non-zero exit code 1,externalMessage=Something went wrong within the source connector,metadata=io.airbyte.config.Metadata@3e46bc5[additionalProperties={attemptNumber=0, jobId=11910178, connector_command=read}],stacktrace=io.airbyte.workers.internal.exception.SourceException: Source process exited with non-zero exit code 1
at io.airbyte.workers.general.BufferedReplicationWorker.readFromSource(BufferedReplicationWorker.java:378)
at io.airbyte.workers.general.BufferedReplicationWorker.lambda$runAsyncWithHeartbeatCheck$3(BufferedReplicationWorker.java:242)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
,retryable=<null>,timestamp=1715772716283,streamDescriptor=<null>,additionalProperties={}]],additionalProperties={}]```
I have been endlessly looking in this Slack, the discussion website, GitHub, etc. for solutions to this issue and I was not able to find anything. Please help me as this will be extremely valuable and vital in my company's decision to use Airbyte. By the way, great work! I love the tool!
More information about DocumentDB:
• Tested with both 4.0 and 5.0.
• I can connect properly to MongoDB.
• The user is read-only and can access the database without a problem.
• I have enabled change streams on all collections, using:
```db.adminCommand({modifyChangeStreams: 1,
database: "<database>",
collection: "",
enable: true});```
I would also greatly appreciate if you could point me in the right direction with some troubleshooting tips.
<br>
---
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1715773298031659) if you want
to access the original thread.
[Join the conversation on Slack](https://slack.airbyte.com)
<sub>
["aws-documentdb", "bigquery", "mongodb-source-connector", "sync-error", "resumeToken", "incremental-streams", "java-exception", "troubleshooting-tips"]
</sub>