Hello team,
While we sync the data from freshdesk, it gets failed but transfers some data, and as per we checked it transferred only half of the data to our application.
let us provide some solution.
mail: gaurav@aivhub.com
Thanks | Gaurav Lohar
AIVHUB
@gauravaivhub it’s possible to upload the sync error log file to identify the root cause?
Hello team,
please, let us provide the solution asap, the logs are shared below, we have removed some logs due to upload character limitations:
2022-03-01 05:24:02 e[32mINFOe[m i.a.s.a.w.WorkerRun(call):49 - Executing worker wrapper. Airbyte version: 0.35.1-alpha
2022-03-01 05:24:03 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):118 - Docker volume job log path: /tmp/workspace/8374/0/logs.log
2022-03-01 05:24:03 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):123 - Executing worker wrapper. Airbyte version: 0.35.1-alpha
2022-03-01 05:24:03 e[33mWARNe[m i.a.d.Databases(createPostgresDatabaseWithRetryTimeout):65 - Waiting for database to become available...
2022-03-01 05:24:03 e[32mINFOe[m i.a.d.i.j.JobsDatabaseInstance(lambda$static$2):25 - Testing if jobs database is ready...
2022-03-01 05:24:03 e[32mINFOe[m i.a.d.Databases(createPostgresDatabaseWithRetryTimeout):90 - Database available!
2022-03-01 05:24:03 e[32mINFOe[m i.a.d.Databases(createPostgresDatabaseWithRetry):48 - Database available!
2022-03-01 05:24:03 e[32mINFOe[m i.a.w.DefaultReplicationWorker(run):99 - start sync worker. job id: 8374 attempt id: 0
2022-03-01 05:24:03 e[32mINFOe[m i.a.w.DefaultReplicationWorker(run):108 - configured sync modes: {null.tickets=full_refresh - overwrite, null.companies=full_refresh - overwrite, null.groups=full_refresh - overwrite, null.agents=full_refresh - overwrite, null.contacts=full_refresh - overwrite, null.roles=full_refresh - overwrite, null.conversations=full_refresh - overwrite, null.satisfaction_ratings=full_refresh - overwrite, null.skills=full_refresh - overwrite, null.time_entries=full_refresh - overwrite, null.surveys=full_refresh - overwrite}
2022-03-01 05:24:15 e[32mINFOe[m i.a.w.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$3):247 - state in DefaultReplicationWorker from Destination: io.airbyte.protocol.models.AirbyteMessage@6611fe9e[type=STATE,log=<null>,spec=<null>,connectionStatus=<null>,catalog=<null>,record=<null>,state=io.airbyte.protocol.models.AirbyteStateMessage@68d60c2c[data={},additionalProperties={}],additionalProperties={}]
2022-03-01 05:24:15 e[43mdestinatione[0m > 2022-03-01 05:24:15 e[32mINFOe[m i.a.i.b.IntegrationRunner(run):133 - {} - Completed integration: io.airbyte.integrations.base.ssh.SshWrappedDestination
2022-03-01 05:24:15 e[43mdestinatione[0m > 2022-03-01 05:24:15 e[32mINFOe[m i.a.i.d.m.MySQLDestination(main):119 - {} - completed destination: class io.airbyte.integrations.destination.mysql.MySQLDestination
2022-03-01 05:24:16 e[32mINFOe[m i.a.w.DefaultReplicationWorker(run):137 - Source and destination threads complete.
2022-03-01 05:24:16 e[32mINFOe[m i.a.w.DefaultReplicationWorker(run):165 - sync summary: io.airbyte.config.ReplicationAttemptSummary@4c725043[status=completed,recordsSynced=0,bytesSynced=0,startTime=1646112243216,endTime=1646112256105]
2022-03-01 05:24:16 e[32mINFOe[m i.a.w.DefaultReplicationWorker(run):172 - Source output at least one state message
2022-03-01 05:24:16 e[32mINFOe[m i.a.w.DefaultReplicationWorker(run):178 - State capture: Updated state to: Optional[io.airbyte.config.State@13e8f70b[state={}]]
2022-03-01 05:24:16 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):144 - Stopping cancellation check scheduling...
2022-03-01 05:24:16 e[32mINFOe[m i.a.c.p.ConfigRepository(updateConnectionState):537 - Updating connection 5bfa84f1-264b-485b-81f3-ea4238486046 state: io.airbyte.config.State@3a058cfe[state={}]
2022-03-01 05:24:16 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):118 - Docker volume job log path: /tmp/workspace/8374/0/logs.log
2022-03-01 05:24:16 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):123 - Executing worker wrapper. Airbyte version: 0.35.1-alpha
2022-03-01 05:24:16 e[33mWARNe[m i.a.d.Databases(createPostgresDatabaseWithRetryTimeout):65 - Waiting for database to become available...
2022-03-01 05:24:16 e[32mINFOe[m i.a.d.i.j.JobsDatabaseInstance(lambda$static$2):25 - Testing if jobs database is ready...
2022-03-01 05:24:16 e[32mINFOe[m i.a.d.Databases(createPostgresDatabaseWithRetryTimeout):90 - Database available!
2022-03-01 05:24:16 e[32mINFOe[m i.a.d.Databases(createPostgresDatabaseWithRetry):48 - Database available!
2022-03-01 05:24:16 e[32mINFOe[m i.a.w.DefaultNormalizationWorker(run):46 - Running normalization.
2022-03-01 05:24:16 e[32mINFOe[m i.a.w.n.DefaultNormalizationRunner(runProcess):122 - Running with normalization version: airbyte/normalization-mysql:0.1.61
2022-03-01 05:24:16 e[32mINFOe[m i.a.c.i.LineGobbler(voidCall):82 - Checking if airbyte/normalization-mysql:0.1.61 exists...
2022-03-01 05:24:16 e[32mINFOe[m i.a.w.p.DockerProcessFactory(create):171 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/8374/0/normalize --network host --log-driver none airbyte/normalization-mysql:0.1.61 run --integration-type mysql --config destination_config.json --catalog destination_catalog.json
2022-03-01 05:24:16 e[32mINFOe[m i.a.c.i.LineGobbler(voidCall):82 - airbyte/normalization-mysql:0.1.61 was found locally.
2022-03-01 05:24:16 e[42mnormalizatione[0m > Running: transform-config --config destination_config.json --integration-type mysql --out /data/8374/0/normalize
2022-03-01 05:24:16 e[42mnormalizatione[0m > Namespace(config='destination_config.json', integration_type=<DestinationType.mysql: 'mysql'>, out='/data/8374/0/normalize')
2022-03-01 05:24:16 e[42mnormalizatione[0m > transform_mysql
2022-03-01 05:24:16 e[42mnormalizatione[0m > Running: transform-catalog --integration-type mysql --profile-config-dir /data/8374/0/normalize --catalog destination_catalog.json --out /data/8374/0/normalize/models/generated/ --json-column _airbyte_data
2022-03-01 05:24:17 e[42mnormalizatione[0m > Processing destination_catalog.json...
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_agents_ab1.sql from freshdesk_agents
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_agents_ab2.sql from freshdesk_agents
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_agents_ab3.sql from freshdesk_agents
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_tables/freshdesk/freshdesk_agents.sql from freshdesk_agents
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_companies_ab1.sql from freshdesk_companies
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_companies_ab2.sql from freshdesk_companies
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_companies_ab3.sql from freshdesk_companies
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_tables/freshdesk/freshdesk_companies.sql from freshdesk_companies
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_contacts_ab1.sql from freshdesk_contacts
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_contacts_ab2.sql from freshdesk_contacts
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_contacts_ab3.sql from freshdesk_contacts
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_tables/freshdesk/freshdesk_contacts.sql from freshdesk_contacts
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_conversations_ab1.sql from freshdesk_conversations
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_conversations_ab2.sql from freshdesk_conversations
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_conversations_ab3.sql from freshdesk_conversations
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_tables/freshdesk/freshdesk_conversations.sql from freshdesk_conversations
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_groups_ab1.sql from freshdesk_groups
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_groups_ab2.sql from freshdesk_groups
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_groups_ab3.sql from freshdesk_groups
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_tables/freshdesk/freshdesk_groups.sql from freshdesk_groups
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_roles_ab1.sql from freshdesk_roles
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_roles_ab2.sql from freshdesk_roles
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_roles_ab3.sql from freshdesk_roles
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_tables/freshdesk/freshdesk_roles.sql from freshdesk_roles
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_satisfaction_ratings_ab1.sql from freshdesk_satisfaction_ratings
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_satisfaction_ratings_ab2.sql from freshdesk_satisfaction_ratings
2022-03-01 05:24:17 e[42mnormalizatione[0m > Truncating _airbyte_freshdesk_satisfaction_ratings_hashid (#46) to _airbyte_freshdesk_sa_action_ratings_hashid (#44)
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_satisfaction_ratings_ab3.sql from freshdesk_satisfaction_ratings
2022-03-01 05:24:17 e[42mnormalizatione[0m > Truncating _airbyte_freshdesk_satisfaction_ratings_hashid (#46) to _airbyte_freshdesk_sa_action_ratings_hashid (#44)
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_tables/freshdesk/freshdesk_satisfaction_ratings.sql from freshdesk_satisfaction_ratings
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_skills_ab1.sql from freshdesk_skills
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_skills_ab2.sql from freshdesk_skills
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_skills_ab3.sql from freshdesk_skills
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_tables/freshdesk/freshdesk_skills.sql from freshdesk_skills
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_surveys_ab1.sql from freshdesk_surveys
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_surveys_ab2.sql from freshdesk_surveys
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_surveys_ab3.sql from freshdesk_surveys
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_tables/freshdesk/freshdesk_surveys.sql from freshdesk_surveys
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_tickets_ab1.sql from freshdesk_tickets
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_tickets_ab2.sql from freshdesk_tickets
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_tickets_ab3.sql from freshdesk_tickets
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_tables/freshdesk/freshdesk_tickets.sql from freshdesk_tickets
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_time_entries_ab1.sql from freshdesk_time_entries
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_time_entries_ab2.sql from freshdesk_time_entries
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_time_entries_ab3.sql from freshdesk_time_entries
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_tables/freshdesk/freshdesk_time_entries.sql from freshdesk_time_entries
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_agents_contact_ab1.sql from freshdesk_agents/contact
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_agents_contact_ab2.sql from freshdesk_agents/contact
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_agents_contact_ab3.sql from freshdesk_agents/contact
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_tables/freshdesk/freshdesk_agents_contact.sql from freshdesk_agents/contact
2022-03-01 05:24:17 e[42mnormalizatione[0m > Ignoring stream 'domains' from freshdesk_companies/domains because properties list is empty
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_skills_agents_ab1.sql from freshdesk_skills/agents
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_skills_agents_ab2.sql from freshdesk_skills/agents
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_skills_agents_ab3.sql from freshdesk_skills/agents
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_tables/freshdesk/freshdesk_skills_agents.sql from freshdesk_skills/agents
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_skills_conditions_ab1.sql from freshdesk_skills/conditions
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_skills_conditions_ab2.sql from freshdesk_skills/conditions
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_skills_conditions_ab3.sql from freshdesk_skills/conditions
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_tables/freshdesk/freshdesk_skills_conditions.sql from freshdesk_skills/conditions
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_surveys_questions_ab1.sql from freshdesk_surveys/questions
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_surveys_questions_ab2.sql from freshdesk_surveys/questions
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_ctes/freshdesk/freshdesk_surveys_questions_ab3.sql from freshdesk_surveys/questions
2022-03-01 05:24:17 e[42mnormalizatione[0m > Generating airbyte_tables/freshdesk/freshdesk_surveys_questions.sql from freshdesk_surveys/questions
2022-03-01 05:24:17 e[42mnormalizatione[0m > Ignoring stream 'to_emails' from freshdesk_tickets/to_emails because properties list is empty
2022-03-01 05:24:17 e[42mnormalizatione[0m > Ignoring stream 'value' from freshdesk_skills/conditions/value because properties list is empty
2022-03-01 05:24:17 e[42mnormalizatione[0m > Ignoring stream 'accepted_ratings' from freshdesk_surveys/questions/accepted_ratings because properties list is empty
2022-03-01 05:24:17 e[42mnormalizatione[0m > detected no config file for ssh, assuming ssh is off.
2022-03-01 05:24:19 e[42mnormalizatione[0m > Running with dbt=0.19.0
2022-03-01 05:24:23 e[42mnormalizatione[0m > [e[33mWARNINGe[0m]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
2022-03-01 05:24:23 e[42mnormalizatione[0m > There are 2 unused configuration paths:
2022-03-01 05:24:23 e[42mnormalizatione[0m > - models.airbyte_utils.generated.airbyte_incremental
2022-03-01 05:24:23 e[42mnormalizatione[0m > - models.airbyte_utils.generated.airbyte_views
2022-03-01 05:24:23 e[42mnormalizatione[0m >
2022-03-01 05:24:24 e[42mnormalizatione[0m > Found 60 models, 0 tests, 0 snapshots, 0 analyses, 441 macros, 0 operations, 0 seed files, 11 sources, 0 exposures
The part of logs you’d shared doesn’t display any errors.