Custom Replication/Sync Time

Connector Questions & Issues
1

  • Is this your first time deploying Airbyte: Yes

  • OS Version / Instance: Mac O

  • Memory / Disk: 16Gb / 1Tb SSD

  • Deployment: Docker / Kubernetes

  • Airbyte Version: 0.55.55-alpha

  • Source name/version: File 0.24

  • Destination name/version: Snowflake

  • Description: I want to replicate or sync data on Weekly basis but there is no such options how can I do that?

2

Hi @Dev-ManishTomar,
The web interface does not allow to pick a scheduled interval greater than 24h.
But leveraging our API you could:

  • Trigger the sync from a weekly cron with your own script
  • Set the scheduling to 7 days by updating the connection configuration
3

Any Guide which I can follow?

4

We do not have this kind of guide, because it’s rather a UI workaround rather than an optimal solution.

You can basically run a curl command from a cron on this endpoint.

E.G for a weekly trigger in a crontab

0 0 * * 0 curl -X POST http://localhost:8000/api/v1/connections/sync  -H "Content-Type: application/json" -d '{"connectionId": "<your-connection-id>"}'

When this set up is working you can disable the scheduling of your connection from the UI so it will only run when triggered by your cron job (considered as a manual sync by Airbyte).

5

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.