Summary
The user is inquiring about the usage of Xmin in a Postgres source for incremental updates and whether it helps in recovering the state after a worker failure.
Question
If I run with xmin and incrementally update on TS column with postgres source.
If the worker fails and goes down, will it be able to recover from the state it failed so I don’t have to start over again?
I am trying to understand the doc how to use this xmin I am right now seeing in my logs
2024-02-29 13:32:40 [44msource[0m > INFO i.a.i.s.p.PostgresSource(getIncrementalIterators):488 Xmin Status : {Number of wraparounds: 0, Xmin Transaction Value: 1197157354, Xmin Raw Value: 1197157354
2024-02-29 13:32:40 [44msource[0m > INFO i.a.i.s.p.PostgresQueryUtils(fileNodeForIndividualStream):235 Relation filenode is for stream "public"."table" is 49406
2024-02-29 13:32:40 [44msource[0m > INFO i.a.i.s.p.PostgresQueryUtils(getTableBlockSizeForStream):299 Stream "public"."table" relation size is 9427099648. block size 8192
2024-02-29 13:33:01 [44msource[0m > INFO i.a.i.s.p.PostgresQueryUtils(getTableMaxTupleForStream):348 Stream "public"."table" max tuple is 134
2024-02-29 13:33:01 [44msource[0m > INFO i.a.i.s.p.PostgresSource(getIncrementalIterators):524 Streams to be synced via ctid : 1
2024-02-29 13:33:01 [44msource[0m > INFO i.a.i.s.p.PostgresSource(getIncrementalIterators):525 Streams: public.provider_charge
2024-02-29 13:33:01 [44msource[0m > INFO i.a.i.s.p.PostgresSource(getIncrementalIterators):534 No Streams will be synced via xmin.```
Does this mean `xmin` will not improve anything?
<br>
---
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1709212972185099) if you want to access the original thread.
[Join the conversation on Slack](https://slack.airbyte.com)
<sub>
["xmin", "postgres-source", "incremental-updates", "worker-failure", "recovery"]
</sub>