Truncating Job History Configuration

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

Inquiring about configuration option for automatically truncating job history or if it needs to be done with a database query


Question

Can anyone point me to a configuration option for truncating job history automatically? Or does that need to be done with a database query?


This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want to access the original thread.

Join the conversation on Slack

["truncating-job-history", "configuration-option", "database-query"]
2

Hi Sean, we do not have a configuration option for that just yet.
You’ll need to do that through a database query indeed, let me know if you need an example.

3

An example would be very helpful!

4

<@U0304PEJRFH> Greetings. Apologies for tagging you directly; I was hoping you could provide an example query to truncate job history.

5

Hi Sean, sorry I forgot to respond to you, I’m in a meeting right now but I’ll give you an answer in the morning. Ping me again if I forget

6

Much appreciated, thank you.

7

Hi Sean, this would be an example. Give me a bit more time to test, clean up that script a bit more today and remove the redundant date. I should have something cleaned up and tested tomorrow:

-- tables with job_id reference
delete from stream_statuses where job_id in (select id from jobs where created_at &lt; '2023-12-01');
delete from retry_states where job_id in (select id from jobs where created_at &lt; '2023-12-01');

-- tables with attempt_id reference
delete from sync_stats where attempt_id in (select a.id from attempts a inner join jobs j on j.id = a.job_id where j.created_at &lt; '2023-12-01');
delete from stream_stats where attempt_id in (select a.id from attempts a inner join jobs j on j.id = a.job_id where j.created_at &lt; '2023-12-01');
delete from normalization_summaries where attempt_id in (select a.id from attempts a inner join jobs j on j.id = a.job_id where j.created_at &lt; '2023-12-01');

delete from attempts where job_id in (select id from jobs where created_at &lt; '2023-12-01');
delete from jobs where created_at &lt; '2023-12-01';

commit;```
8

Thank you very much!