JobHistoryHandler - "Missing stats for job" warning

Connector Questions & Issues
,
1
  • Is this your first time deploying Airbyte?: No
  • Deployment: Are you using Docker or Kubernetes deployment? K8S GKE Autopilot
  • Airbyte Version: What version are you using now? 0.40.30
  • Step: The issue is happening during sync, creating the connection or a new source? During the execution of any connection.
  • Description: Whenever we run a connection, the connection runs with no error, but in the server there are a lot of error handling but no description for the error:
    The errors look like this:
2023-02-28 16:27:37 ERROR i.a.c.s.h.JobHistoryHandler(listJobsFor):140 - Missing stats for job 2977 attempt 0
2023-02-28 16:27:40 ERROR i.a.c.s.h.JobHistoryHandler(listJobsFor):140 - Missing stats for job 2977 attempt 0
2023-02-28 16:27:42 ERROR i.a.c.s.h.JobHistoryHandler(listJobsFor):140 - Missing stats for job 2977 attempt 0
2023-02-28 16:27:45 ERROR i.a.c.s.h.JobHistoryHandler(listJobsFor):140 - Missing stats for job 2977 attempt 0
2023-02-28 16:27:47 ERROR i.a.c.s.h.JobHistoryHandler(listJobsFor):140 - Missing stats for job 2977 attempt 0

This problem started to happen after adding these variables to the server:

  • STATE_STORAGE_GCS_APPLICATION_CREDENTIALS
  • STATE_STORAGE_GCS_BUCKET_NAME
  • WORKER_STATE_STORAGE_TYPE
  • WORKER_LOGS_STORAGE_TYPE
    We had to add them since we were having a lot of problems when we wanted to get rid of minio to migrate to GCS, they have solved our problem, but we have these error handling now that are polluting the server logs for nothing.
    How can we check the problem efficiently for the JobHistoryHandler?

Thanks

3

Hey @data-bite,

You can take a look at where this is happening in our Airbyte Platform repo: https://github.com/airbytehq/airbyte-platform/blob/876139deba3d60dacc6d5e5511eb7637918f0e5e/airbyte-commons-server/src/main/java/io/airbyte/commons/server/handlers/JobHistoryHandler.java#L144

My understanding is that this piece of code retrieves job attempt statistics from a JobPersistence object and updates corresponding AttemptRead objects. The stat variable refers to a JobPersistence.AttemptStats object containing the statistics for a particular job attempt. I’m not sure if this has to do with the variables you added to the server, although they could be related.

Did you delete any variables and where did you set these variables? Was it this file? https://github.com/airbytehq/airbyte-platform/blob/8ddd99fa7225ba8062af9b3b9f3b274e0cb68464/airbyte-container-orchestrator/src/test/resources/files/envMap.json