Missing records in Salesforce incremental synchronization

  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Ubuntu
  • Memory / Disk: 4Gb / 1 Tb
  • Deployment: Kubernetes
  • Airbyte Version: 0.44.3
  • Source name/version: Salesforce / 2.0.12
  • Destination name/version: BigQuery / 1.2.13
  • Step: during sync
  • Description:

I am currently working with a Salesforce Contact table.
The synchronization method is Incremental Sync - Deduped History

The Contact table is heavily updated and pulls in tens of thousands of records daily.
So I found that some records are missing.

  1. Synchronization started on 2023-05-28 21:00:00 (State is {SystemModstamp": "2023-05-27T21:15:08.000+0000} at this time)

  2. after synchronization, State is updated to {“SystemModstamp”: “2023-05-28T21:10:08.000+0000”}.

  3. next synchronization, it was found that a few records were missing, records for which SystemModstamp had the following values: {“SystemModstamp”: “2023-05-28T21:10:08.000+0000”}.(I checked the raw table)

  • 2023-05-28 21:15:07 UTC
  • 2023-05-28 21:15:06 UTC

I can see why it is missing because the value of SystemModstamp, which is indeed a curl field, is the value after that at step2.

How can this be resolved?