- Is this your first time deploying Airbyte?: Yes
- OS Version / Instance: Debian
- Memory / Disk: 4GB / 50GB
- Deployment: Docker
- Airbyte Version: 0.36.1-alpha
- Source name/version: MySQL 0.5.6
- Destination name/version: BigQuery (denormalized typed struct) 0.3.1
- Step: Sync
Description:
When I first set up my connection, it appeared to hang at the 2022-04-22 14:22:06 source > 2022-04-22 14:22:05 INFO i.a.i.s.m.MySqlSource(main):208 - completed source: class io.airbyte.integrations.source.mysql.MySqlSource step of the initial sync.
I cancelled it after 23 hours to restart a sync from scratch, and it took two attempts for succeeding. I say “succeed” as it’s the final state of the sync, but the last two lines of the logs are the following:
2022-04-23 13:33:07 INFO i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed.
errors: $.credential: is not defined in the schema and the schema does not allow additional properties, $.part_size_mb: is not defined in the schema and the schema does not allow additional properties, $.gcs_bucket_name: is not defined in the schema and the schema does not allow additional properties, $.gcs_bucket_path: is not defined in the schema and the schema does not allow additional properties, $.keep_files_in_gcs-bucket: is not defined in the schema and the schema does not allow additional properties, $.method: must be a constant value Standard
Even since that first sync, none of my subsequent syncs (I’ve set it up as Incremental | Append) appear to work, as I encounter the errors: $.credential: is not defined in the schema and the schema does not allow additional properties, $.part_size_mb: is not defined in the schema and the schema does not allow additional properties, $.gcs_bucket_name: is not defined in the schema and the schema does not allow additional properties, $.gcs_bucket_path: is not defined in the schema and the schema does not allow additional properties, $.keep_files_in_gcs-bucket: is not defined in the schema and the schema does not allow additional properties, $.method: must be a constant value Standard again each time.
I’m attaching success logs for the initial sync (logs-2) along with the logs for the last incremental sync (logs-4).
Thanks!
logs-2.txt (117.4 KB)
logs-4.txt (325.8 KB)
@xavierosee looks to me like a resource issue. Can you try increasing the memory and try syncing it again?
Hi @harshith and thank you!
I’ve upgraded memory to 8GB, and ran a new ìnitial sync` yesterday, 18 hours ago.
It’s now stuck to 2022-04-25 16:02:12 source > 2022-04-25 16:02:11 INFO i.a.i.s.m.MySqlSource(main):208 - completed source: class io.airbyte.integrations.source.mysql.MySqlSource since yesterday. Do you still believe it’s a memory issue ?
Hey @xavierosee do you see DEADLINE_EXCEEDED similar to log_4 ?
@harshith no, but the sync is still on running… I could try cancelling it and relaunching a new one to check…
yeah do you mind retrying it
It did succeed right now, even though I still have an error on the last line 
(logs-9)
I manually triggered another sync right after and that worked as well though still with the same error (logs-10). I’ll keep monitoring but it’s looking like it’s flowing =)
logs-10.txt (219.3 KB)
logs-9.txt (677.3 KB)
@xavierosee is the sync getting successful ?
Yes absolutely.
Do you know what this error is about though ? I don’t understand how it can be a successful sync if they all end with this error…
errors:
$.credential: is not defined in the schema and the schema does not allow additional properties,
$.part_size_mb: is not defined in the schema and the schema does not allow additional properties,
$.gcs_bucket_name: is not defined in the schema and the schema does not allow additional properties,
$.gcs_bucket_path: is not defined in the schema and the schema does not allow additional properties,
$.keep_files_in_gcs-bucket: is not defined in the schema and the schema does not allow additional properties,
$.method: must be a constant value Standard
Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.