Summary
Issue with cursor rounding in Airbyte OSS v1.1.0 source-mssql connector when using incremental append method, causing data duplication or omission. Cursor date milliseconds are being rounded, impacting sync accuracy.
Question
Hi, I’m having a trouble with airbyte OSS v1.1.0 on the source-mssql, I’m using the incremental append method and it seems that the cursor is not working properly. It seems that airbyte is rounding the milliseconds, so if the cursor date is 2024-09-25T13:39:40.833000, it will be rounded to 2024-09-25T13:39:40.83 and on the next sync all data with 2024-09-25T13:39:40.833000 will be added again. If the cursor is : 2024-09-25T13:39:40.837000, it will be rounded to 2024-09-25T13:39:40.84 and in the next sync no data will be passed, respecting the incremental append. Anyone had this problem and know how to solve it? The data type of the cursor on mssql is datetime
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.