Summary
User is experiencing issues with incremental mode using CDC in Airbyte’s Postgres and MongoDB source connectors, where the read command hangs instead of terminating like in full refresh mode. They seek clarification on the behavior and whether manual intervention is required.
Question
Hi! I’m trying to use an Airbyte connector directly and having trouble understanding how it works, specifically with incremental mode using CDC. I am trying with the airbyte postgres and mongodb source connectors. Using full refresh mode it seems the read command terminates with my toy dataset (of like 2 items), but using the incremental mode it seems to just hang there. I’m trying to understand what this means and how I can interpret this behavior. I know that I get a state back as the last message so I could technically take that to get more. But I guess I’d like to understand why it is hanging there instead of terminating on its own like in the full refresh mode. I thought maybe that is because it was waiting indefinitely for more updates, but that doesn’t seem to be the case as no more new records are emitted when I insert new entries. Any ideas or explanations as to what is going on? Am I suppose to manually cut it off? Thank you!
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.