MySQL to BigQuery connection failing - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached

  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Ubuntu
  • Memory / Disk: 16Gb
  • Deployment: Docker
  • Airbyte Version: What version are you using now?
  • Source name/version: 0.35.64-alpha
  • Destination name/version: BigQuery 1.0.1
  • Step: Creating the connection
  • Description: Connection fails with the error message can be seen below. Not sure why it’s failing.
INFOe[m i.a.w.w.WorkerRun(call):49 - Executing worker wrapper. Airbyte version: 0.35.64-alpha
INFOe[m i.a.w.t.TemporalAttemptExecution(get):105 - Docker volume job log path: /tmp/workspace/20/1/logs.log
INFOe[m i.a.w.t.TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.35.64-alpha
INFOe[m i.a.w.DefaultReplicationWorker(run):104 - start sync worker. job id: 20 attempt id: 1
INFOe[m i.a.w.DefaultReplicationWorker(run):116 - configured sync modes: {backendmariadb.sales_order=full_refresh - overwrite, backendmariadb.store_website=full_refresh - overwrite, backendmariadb.store_group=full_refresh - overwrite, backendmariadb.store=full_refresh - overwrite, backendmariadb.sales_order_item=full_refresh - overwrite}
INFOe[m i.a.w.p.a.DefaultAirbyteDestination(start):69 - Running destination...
INFOe[m i.a.c.i.LineGobbler(voidCall):82 - Checking if airbyte/destination-bigquery:1.0.1 exists...
INFOe[m i.a.c.i.LineGobbler(voidCall):82 - airbyte/destination-bigquery:1.0.1 was found locally.
INFOe[m i.a.w.p.DockerProcessFactory(create):106 - Creating docker job ID: 20
INFOe[m i.a.w.p.DockerProcessFactory(create):158 - Preparing command: docker run --rm --init -i -w /data/20/1 --log-driver none --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e WORKER_CONNECTOR_IMAGE=airbyte/destination-bigquery:1.0.1 -e WORKER_JOB_ATTEMPT=1 -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e AIRBYTE_VERSION=0.35.64-alpha -e WORKER_JOB_ID=20 airbyte/destination-bigquery:1.0.1 write --config destination_config.json --catalog destination_catalog.json
INFOe[m i.a.c.i.LineGobbler(voidCall):82 - Checking if airbyte/source-mysql:0.5.6 exists...
INFOe[m i.a.c.i.LineGobbler(voidCall):82 - airbyte/source-mysql:0.5.6 was found locally.
INFOe[m i.a.w.p.DockerProcessFactory(create):106 - Creating docker job ID: 20
INFOe[m i.a.w.p.DockerProcessFactory(create):158 - Preparing command: docker run --rm --init -i -w /data/20/1 --log-driver none --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e WORKER_CONNECTOR_IMAGE=airbyte/source-mysql:0.5.6 -e WORKER_JOB_ATTEMPT=1 -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e AIRBYTE_VERSION=0.35.64-alpha -e WORKER_JOB_ID=20 airbyte/source-mysql:0.5.6 read --config source_config.json --catalog source_catalog.json --state input_state.json
INFOe[m i.a.w.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$6):339 - Destination output thread started.
INFOe[m i.a.w.DefaultReplicationWorker(run):158 - Waiting for source and destination threads to complete.
INFOe[m i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$5):279 - Replication thread started.
sourcee[0m > [0.104s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached.
sourcee[0m > Error occurred during initialization of VM
sourcee[0m > java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
sourcee[0m > 	at java.lang.Thread.start0(java.base@17.0.1/Native Method)
sourcee[0m > 	at java.lang.Thread.start(java.base@17.0.1/Thread.java:802)
sourcee[0m > 	at java.lang.ref.Reference.<clinit>(java.base@17.0.1/Reference.java:314)
sourcee[0m > 	at java.security.AccessController.doPrivileged(java.base@17.0.1/AccessController.java:318)
sourcee[0m > 	at java.lang.reflect.AccessibleObject.<clinit>(java.base@17.0.1/AccessibleObject.java:489)
INFOe[m i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed. 
errors: $: unknown found, object expected
ERRORe[m i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$1):70 - Validation failed: null
INFOe[m i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$5):305 - Total records read: 0 (0 bytes)
destinatione[0m > SLF4J: Class path contains multiple SLF4J bindings.
destinatione[0m > SLF4J: Found binding in [jar:file:/airbyte/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
destinatione[0m > SLF4J: Found binding in [jar:file:/airbyte/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
destinatione[0m > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
destinatione[0m > SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
INFOe[m i.a.i.b.IntegrationCliParser(parseOptions):118 - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json}
INFOe[m i.a.i.b.IntegrationRunner(runInternal):121 - Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination
INFOe[m i.a.i.b.IntegrationRunner(runInternal):122 - Command: WRITE
INFOe[m i.a.i.b.IntegrationRunner(runInternal):123 - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'}
WARNe[m c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
WARNe[m c.n.s.JsonMetaSchema(newValidator):338 - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
INFOe[m i.a.i.d.b.BigQueryUtils(getLoadingMethod):284 - Selected loading method is set to: STANDARD
INFOe[m i.a.i.d.b.BigQueryUtils(createPartitionedTable):124 - Partitioned Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=backendmariadb, tableId=_airbyte_tmp_iya_MAGENTO_sales_order}} created successfully
INFOe[m i.a.i.d.b.BigQueryUtils(getLoadingMethod):284 - Selected loading method is set to: STANDARD
INFOe[m i.a.i.d.b.BigQueryUtils(createPartitionedTable):124 - Partitioned Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=backendmariadb, tableId=_airbyte_tmp_pzt_MAGENTO_sales_order_item}} created successfully
INFOe[m i.a.i.d.b.BigQueryUtils(getLoadingMethod):284 - Selected loading method is set to: STANDARD
INFOe[m i.a.i.d.b.BigQueryUtils(createPartitionedTable):124 - Partitioned Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=backendmariadb, tableId=_airbyte_tmp_opg_MAGENTO_store}} created successfully
INFOe[m i.a.i.d.b.BigQueryUtils(getLoadingMethod):284 - Selected loading method is set to: STANDARD
INFOe[m i.a.i.d.b.BigQueryUtils(createPartitionedTable):124 - Partitioned Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=backendmariadb, tableId=_airbyte_tmp_sek_MAGENTO_store_group}} created successfully
INFOe[m i.a.i.d.b.BigQueryUtils(getLoadingMethod):284 - Selected loading method is set to: STANDARD
INFOe[m i.a.i.d.b.BigQueryUtils(createPartitionedTable):124 - Partitioned Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=backendmariadb, tableId=_airbyte_tmp_adm_MAGENTO_store_website}} created successfully
INFOe[m i.a.i.b.FailureTrackingAirbyteMessageConsumer(close):65 - Airbyte message consumer: succeeded.
INFOe[m i.a.i.d.b.BigQueryRecordConsumer(close):58 - Started closing all connections
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(close):76 - Field fails during format : 
INFOe[m i.a.i.d.b.f.BigQueryRecordFormatter(printAndCleanFieldFails):70 - No field fails during record format.
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(close):79 - Closing connector:AbstractBigQueryUploader{table=_airbyte_raw_MAGENTO_store, tmpTable=_airbyte_tmp_opg_MAGENTO_store, syncMode=WRITE_TRUNCATE, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter}
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):96 - Uploading data from the tmp table _airbyte_tmp_opg_MAGENTO_store to the source table _airbyte_raw_MAGENTO_store.
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadDataToTableFromTmpTable):121 - Replication finished with no explicit errors. Copying data from tmp tables to permanent
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(copyTable):187 - successfully copied table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=backendmariadb, tableId=_airbyte_tmp_opg_MAGENTO_store}} to table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=backendmariadb, tableId=_airbyte_raw_MAGENTO_store}}
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):98 - Data is successfully loaded to the source table _airbyte_raw_MAGENTO_store!
[32mINFOe[m i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed. 
errors: $: null found, object expected
[1;31mERRORe[m i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$1):70 - Validation failed: null
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):100 - Final state message is accepted.
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(dropTmpTable):112 - Removing tmp tables...
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(dropTmpTable):114 - Finishing destination process...completed
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(close):86 - Closed connector:AbstractBigQueryUploader{table=_airbyte_raw_MAGENTO_store, tmpTable=_airbyte_tmp_opg_MAGENTO_store, syncMode=WRITE_TRUNCATE, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter}
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(close):76 - Field fails during format : 
INFOe[m i.a.i.d.b.f.BigQueryRecordFormatter(printAndCleanFieldFails):70 - No field fails during record format.
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(close):79 - Closing connector:AbstractBigQueryUploader{table=_airbyte_raw_MAGENTO_store_website, tmpTable=_airbyte_tmp_adm_MAGENTO_store_website, syncMode=WRITE_TRUNCATE, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter}
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):96 - Uploading data from the tmp table _airbyte_tmp_adm_MAGENTO_store_website to the source table _airbyte_raw_MAGENTO_store_website.
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadDataToTableFromTmpTable):121 - Replication finished with no explicit errors. Copying data from tmp tables to permanent
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(copyTable):187 - successfully copied table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=backendmariadb, tableId=_airbyte_tmp_adm_MAGENTO_store_website}} to table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=backendmariadb, tableId=_airbyte_raw_MAGENTO_store_website}}
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):98 - Data is successfully loaded to the source table _airbyte_raw_MAGENTO_store_website!
[32mINFOe[m i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed. 
errors: $: null found, object expected
[1;31mERRORe[m i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$1):70 - Validation failed: null
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):100 - Final state message is accepted.
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(dropTmpTable):112 - Removing tmp tables...
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(dropTmpTable):114 - Finishing destination process...completed
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(close):86 - Closed connector:AbstractBigQueryUploader{table=_airbyte_raw_MAGENTO_store_website, tmpTable=_airbyte_tmp_adm_MAGENTO_store_website, syncMode=WRITE_TRUNCATE, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter}
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(close):76 - Field fails during format : 
INFOe[m i.a.i.d.b.f.BigQueryRecordFormatter(printAndCleanFieldFails):70 - No field fails during record format.
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(close):79 - Closing connector:AbstractBigQueryUploader{table=_airbyte_raw_MAGENTO_sales_order, tmpTable=_airbyte_tmp_iya_MAGENTO_sales_order, syncMode=WRITE_TRUNCATE, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter}
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):96 - Uploading data from the tmp table _airbyte_tmp_iya_MAGENTO_sales_order to the source table _airbyte_raw_MAGENTO_sales_order.
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadDataToTableFromTmpTable):121 - Replication finished with no explicit errors. Copying data from tmp tables to permanent
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(copyTable):187 - successfully copied table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=backendmariadb, tableId=_airbyte_tmp_iya_MAGENTO_sales_order}} to table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=backendmariadb, tableId=_airbyte_raw_MAGENTO_sales_order}}
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):98 - Data is successfully loaded to the source table _airbyte_raw_MAGENTO_sales_order!
INFOe[m i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed. 
errors: $: null found, object expected
ERRORe[m i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$1):70 - Validation failed: null
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):100 - Final state message is accepted.
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(dropTmpTable):112 - Removing tmp tables...
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(dropTmpTable):114 - Finishing destination process...completed
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(close):86 - Closed connector:AbstractBigQueryUploader{table=_airbyte_raw_MAGENTO_sales_order, tmpTable=_airbyte_tmp_iya_MAGENTO_sales_order, syncMode=WRITE_TRUNCATE, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter}
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(close):76 - Field fails during format : 
INFOe[m i.a.i.d.b.f.BigQueryRecordFormatter(printAndCleanFieldFails):70 - No field fails during record format.
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(close):79 - Closing connector:AbstractBigQueryUploader{table=_airbyte_raw_MAGENTO_sales_order_item, tmpTable=_airbyte_tmp_pzt_MAGENTO_sales_order_item, syncMode=WRITE_TRUNCATE, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter}
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):96 - Uploading data from the tmp table _airbyte_tmp_pzt_MAGENTO_sales_order_item to the source table _airbyte_raw_MAGENTO_sales_order_item.
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadDataToTableFromTmpTable):121 - Replication finished with no explicit errors. Copying data from tmp tables to permanent
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(copyTable):187 - successfully copied table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=backendmariadb, tableId=_airbyte_tmp_pzt_MAGENTO_sales_order_item}} to table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=backendmariadb, tableId=_airbyte_raw_MAGENTO_sales_order_item}}
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):98 - Data is successfully loaded to the source table _airbyte_raw_MAGENTO_sales_order_item!
INFOe[m i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed. 
errors: $: null found, object expected
1;
ERRORe[m i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$1):70 - Validation failed: null
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):100 - Final state message is accepted.
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(dropTmpTable):112 - Removing tmp tables...
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(dropTmpTable):114 - Finishing destination process...completed
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(close):86 - Closed connector:AbstractBigQueryUploader{table=_airbyte_raw_MAGENTO_sales_order_item, tmpTable=_airbyte_tmp_pzt_MAGENTO_sales_order_item, syncMode=WRITE_TRUNCATE, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter}
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(close):76 - Field fails during format : 
INFOe[m i.a.i.d.b.f.BigQueryRecordFormatter(printAndCleanFieldFails):70 - No field fails during record format.
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(close):79 - Closing connector:AbstractBigQueryUploader{table=_airbyte_raw_MAGENTO_store_group, tmpTable=_airbyte_tmp_sek_MAGENTO_store_group, syncMode=WRITE_TRUNCATE, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter}
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):96 - Uploading data from the tmp table _airbyte_tmp_sek_MAGENTO_store_group to the source table _airbyte_raw_MAGENTO_store_group.
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadDataToTableFromTmpTable):121 - Replication finished with no explicit errors. Copying data from tmp tables to permanent
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(copyTable):187 - successfully copied table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=backendmariadb, tableId=_airbyte_tmp_sek_MAGENTO_store_group}} to table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=backendmariadb, tableId=_airbyte_raw_MAGENTO_store_group}}
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):98 - Data is successfully loaded to the source table _airbyte_raw_MAGENTO_store_group!
INFOe[m i.a.v.j.JsonSchemaValidator(test):56 - JSON schema validation failed. 
errors: $: null found, object expected
ERRORe[m i.a.w.p.a.DefaultAirbyteStreamFactory(lambda$create$1):70 - Validation failed: null
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(uploadData):100 - Final state message is accepted.
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(dropTmpTable):112 - Removing tmp tables...
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(dropTmpTable):114 - Finishing destination process...completed
INFOe[m i.a.i.d.b.u.AbstractBigQueryUploader(close):86 - Closed connector:AbstractBigQueryUploader{table=_airbyte_raw_MAGENTO_store_group, tmpTable=_airbyte_tmp_sek_MAGENTO_store_group, syncMode=WRITE_TRUNCATE, writer=class io.airbyte.integrations.destination.bigquery.writer.BigQueryTableWriter, recordFormatter=class io.airbyte.integrations.destination.bigquery.formatter.DefaultBigQueryRecordFormatter}
INFOe[m i.a.i.b.IntegrationRunner(runInternal):169 - Completed integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination
ERRORe[m i.a.w.DefaultReplicationWorker(run):169 - Sync worker failed.
java.util.concurrent.ExecutionException: io.airbyte.workers.DefaultReplicationWorker$SourceException: Source process exited with non-zero exit code 1
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396) ~[?:?]
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073) ~[?:?]
	at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:162) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:57) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at java.lang.Thread.run(Thread.java:833) [?:?]
	Suppressed: io.airbyte.workers.WorkerException: Source process exit with code 1. This warning is normal if the job was cancelled.
		at io.airbyte.workers.protocols.airbyte.DefaultAirbyteSource.close(DefaultAirbyteSource.java:136) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
		at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:126) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
		at io.airbyte.workers.DefaultReplicationWorker.run(DefaultReplicationWorker.java:57) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
		at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:155) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
		at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: io.airbyte.workers.DefaultReplicationWorker$SourceException: Source process exited with non-zero exit code 1
	at io.airbyte.workers.DefaultReplicationWorker.lambda$getReplicationRunnable$5(DefaultReplicationWorker.java:312) ~[io.airbyte-airbyte-workers-0.35.64-alpha.jar:?]
	at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
	... 1 more
INFOe[m i.a.w.DefaultReplicationWorker(run):228 - sync summary: io.airbyte.config.ReplicationAttemptSummary@6737ba63[status=failed,recordsSynced=0,bytesSynced=0,startTime=1652713163971,endTime=1652713181770,totalStats=io.airbyte.config.SyncStats@427cfdd4[recordsEmitted=0,bytesEmitted=0,stateMessagesEmitted=0,recordsCommitted=0],streamStats=[]]
INFOe[m i.a.w.DefaultReplicationWorker(run):250 - Source did not output any state messages
WARNe[m i.a.w.DefaultReplicationWorker(run):258 - State capture: No new state, falling back on input state: io.airbyte.config.State@28c84a7b[state={}]
INFOe[m i.a.w.t.TemporalAttemptExecution(get):131 - Stopping cancellation check scheduling...

log continued:

INFOe[m i.a.w.t.s.ReplicationActivityImpl(lambda$replicate$1):147 - sync summary: io.airbyte.config.StandardSyncOutput@60dd04ce[standardSyncSummary=io.airbyte.config.StandardSyncSummary@6296b65d[status=failed,recordsSynced=0,bytesSynced=0,startTime=1652713163971,endTime=1652713181770,totalStats=io.airbyte.config.SyncStats@427cfdd4[recordsEmitted=0,bytesEmitted=0,stateMessagesEmitted=0,recordsCommitted=0],streamStats=[]],state=io.airbyte.config.State@28c84a7b[state={}],outputCatalog=io.airbyte.protocol.models.ConfiguredAirbyteCatalog@18aee97f[streams=[io.airbyte.protocol.models.ConfiguredAirbyteStream@3320fa0b[stream=io.airbyte.protocol.models.AirbyteStream@3b5373d5[name=MAGENTO_sales_order,jsonSchema={"type":"object","properties":{"state":{"type":"string"},"tb_id":{"type":"number"},"status":{"type":"string"},"weight":{"type":"number"},"quote_id":{"type":"number"},"store_id":{"type":"number"},"subtotal":{"type":"number"},"entity_id":{"type":"number"},"remote_ip":{"type":"string"},"total_due":{"type":"number"},"created_at":{"type":"string"},"email_sent":{"type":"number"},"is_virtual":{"type":"number"},"send_email":{"type":"number"},"store_name":{"type":"string"},"tax_amount":{"type":"number"},"total_paid":{"type":"number"},"updated_at":{"type":"string"},"coupon_code":{"type":"string"},"customer_id":{"type":"number"},"grand_total":{"type":"number"},"is_exported":{"type":"number"},"tb_order_id":{"type":"string"},"customer_dob":{"type":"string"},"ext_order_id":{"type":"string"},"increment_id":{"type":"string"},"protect_code":{"type":"string"},"tax_canceled":{"type":"number"},"tax_invoiced":{"type":"number"},"tax_refunded":{"type":"number"},"base_subtotal":{"type":"number"},"customer_note":{"type":"string"},"exported_date":{"type":"string"},"base_total_due":{"type":"number"},"customer_email":{"type":"string"},"edit_increment":{"type":"number"},"promo_gift_qty":{"type":"number"},"total_canceled":{"type":"number"},"total_invoiced":{"type":"number"},"total_refunded":{"type":"number"},"base_tax_amount":{"type":"number"},"base_total_paid":{"type":"number"},"customer_gender":{"type":"number"},"customer_prefix":{"type":"string"},"customer_suffix":{"type":"string"},"customer_taxvat":{"type":"string"},"discount_amount":{"type":"number"},"ext_customer_id":{"type":"string"},"gift_message_id":{"type":"number"},"mirakl_order_id":{"type":"string"},"shipping_amount":{"type":"number"},"shipping_method":{"type":"string"},"x_forwarded_for":{"type":"string"},"applied_rule_ids":{"type":"string"},"base_grand_total":{"type":"number"},"coupon_rule_name":{"type":"string"},"promo_gift_count":{"type":"number"},"quote_address_id":{"type":"number"},"total_item_count":{"type":"number"},"base_tax_canceled":{"type":"number"},"base_tax_invoiced":{"type":"number"},"base_tax_refunded":{"type":"number"},"customer_group_id":{"type":"number"},"customer_is_guest":{"type":"number"},"customer_lastname":{"type":"string"},"discount_canceled":{"type":"number"},"discount_invoiced":{"type":"number"},"discount_refunded":{"type":"number"},"hold_before_state":{"type":"string"},"relation_child_id":{"type":"string"},"shipping_canceled":{"type":"number"},"shipping_incl_tax":{"type":"number"},"shipping_invoiced":{"type":"number"},"shipping_refunded":{"type":"number"},"subtotal_canceled":{"type":"number"},"subtotal_incl_tax":{"type":"number"},"subtotal_invoiced":{"type":"number"},"subtotal_refunded":{"type":"number"},"total_qty_ordered":{"type":"number"},"base_currency_code":{"type":"string"},"base_to_order_rate":{"type":"number"},"billing_address_id":{"type":"number"},"can_ship_partially":{"type":"number"},"customer_firstname":{"type":"string"},"hold_before_status":{"type":"string"},"relation_parent_id":{"type":"string"},"store_to_base_rate":{"type":"number"},"adjustment_negative":{"type":"number"},"adjustment_positive":{"type":"number"},"base_to_global_rate":{"type":"number"},"base_total_canceled":{"type":"number"},"base_total_invoiced":{"type":"number"},"base_total_refunded":{"type":"number"},"customer_middlename":{"type":"string"},"order_currency_code":{"type":"string"},"shipping_address_id":{"type":"number"},"shipping_tax_amount":{"type":"number"},"store_currency_code":{"type":"string"},"store_to_order_rate":{"type":"number"},"base_discount_amount":{"type":"number"},"base_shipping_amount":{"type":"number"},"customer_note_notify":{"type":"number"},"discount_description":{"type":"string"},"global_currency_code":{"type":"string"},"mirakl_connection_id":{"type":"number"},"shipping_description":{"type":"string"},"original_increment_id":{"type":"string"},"shipping_tax_refunded":{"type":"number"},"total_online_refunded":{"type":"number"},"base_discount_canceled":{"type":"number"},"base_discount_invoiced":{"type":"number"},"base_discount_refunded":{"type":"number"},"base_shipping_canceled":{"type":"number"},"base_shipping_incl_tax":{"type":"number"},"base_shipping_invoiced":{"type":"number"},"base_shipping_refunded":{"type":"number"},"base_subtotal_canceled":{"type":"number"},"base_subtotal_incl_tax":{"type":"number"},"base_subtotal_invoiced":{"type":"number"},"base_subtotal_refunded":{"type":"number"},"base_total_qty_ordered":{"type":"number"},"relation_child_real_id":{"type":"string"},"total_offline_refunded":{"type":"number"},"can_ship_partially_item":{"type":"number"},"payment_auth_expiration":{"type":"number"},"relation_parent_real_id":{"type":"string"},"base_adjustment_negative":{"type":"number"},"base_adjustment_positive":{"type":"number"},"base_shipping_tax_amount":{"type":"number"},"base_total_invoiced_cost":{"type":"number"},"old_magento_increment_id":{"type":"string"},"shipping_discount_amount":{"type":"number"},"adyen_resulturl_event_code":{"type":"string"},"base_shipping_tax_refunded":{"type":"number"},"base_total_online_refunded":{"type":"number"},"base_total_offline_refunded":{"type":"number"},"trustedshops_mails_accepted":{"type":"number"},"forced_shipment_with_invoice":{"type":"number"},"payment_authorization_amount":{"type":"number"},"paypal_ipn_customer_notified":{"type":"number"},"adyen_notification_event_code":{"type":"string"},"base_shipping_discount_amount":{"type":"number"},"trustedshops_mails_identifier":{"type":"string"},"mp_smtp_email_marketing_synced":{"type":"number"},"discount_tax_compensation_amount":{"type":"number"},"discount_tax_compensation_invoiced":{"type":"number"},"discount_tax_compensation_refunded":{"type":"number"},"adyen_notification_event_code_success":{"type":"boolean"},"base_discount_tax_compensation_amount":{"type":"number"},"mp_smtp_email_marketing_order_created":{"type":"number"},"base_discount_tax_compensation_invoiced":{"type":"number"},"base_discount_tax_compensation_refunded":{"type":"number"},"shipping_discount_tax_compensation_amount":{"type":"number"},"base_shipping_discount_tax_compensation_amnt":{"type":"number"}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[entity_id]],namespace=backendmariadb,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=overwrite,primaryKey=[[entity_id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@6ca01ebc[stream=io.airbyte.protocol.models.AirbyteStream@3aae746[name=MAGENTO_sales_order_item,jsonSchema={"type":"object","properties":{"sku":{"type":"string"},"name":{"type":"string"},"price":{"type":"number"},"tb_id":{"type":"number"},"weight":{"type":"number"},"item_id":{"type":"number"},"order_id":{"type":"number"},"store_id":{"type":"number"},"base_cost":{"type":"number"},"row_total":{"type":"number"},"base_price":{"type":"number"},"created_at":{"type":"string"},"is_virtual":{"type":"number"},"product_id":{"type":"number"},"row_weight":{"type":"number"},"tax_amount":{"type":"number"},"updated_at":{"type":"string"},"description":{"type":"string"},"no_discount":{"type":"number"},"qty_ordered":{"type":"number"},"qty_shipped":{"type":"number"},"tax_percent":{"type":"number"},"product_type":{"type":"string"},"qty_canceled":{"type":"number"},"qty_invoiced":{"type":"number"},"qty_refunded":{"type":"number"},"row_invoiced":{"type":"number"},"tax_canceled":{"type":"number"},"tax_invoiced":{"type":"number"},"tax_refunded":{"type":"number"},"free_shipping":{"type":"number"},"quote_item_id":{"type":"number"},"base_row_total":{"type":"number"},"is_qty_decimal":{"type":"number"},"locked_do_ship":{"type":"number"},"original_price":{"type":"number"},"parent_item_id":{"type":"number"},"price_incl_tax":{"type":"number"},"additional_data":{"type":"string"},"amount_refunded":{"type":"number"},"base_tax_amount":{"type":"number"},"discount_amount":{"type":"number"},"gift_message_id":{"type":"number"},"product_options":{"type":"string"},"qty_backordered":{"type":"number"},"applied_rule_ids":{"type":"string"},"discount_percent":{"type":"number"},"weee_tax_applied":{"type":"string"},"base_row_invoiced":{"type":"number"},"base_tax_invoiced":{"type":"number"},"base_tax_refunded":{"type":"number"},"discount_invoiced":{"type":"number"},"discount_refunded":{"type":"number"},"ext_order_item_id":{"type":"string"},"locked_do_invoice":{"type":"number"},"row_total_incl_tax":{"type":"number"},"base_original_price":{"type":"number"},"base_price_incl_tax":{"type":"number"},"tax_before_discount":{"type":"number"},"base_amount_refunded":{"type":"number"},"base_discount_amount":{"type":"number"},"is_promo_gift_hidden":{"type":"number"},"weee_tax_disposition":{"type":"number"},"base_discount_invoiced":{"type":"number"},"base_discount_refunded":{"type":"number"},"gift_message_available":{"type":"number"},"base_row_total_incl_tax":{"type":"number"},"weee_tax_applied_amount":{"type":"number"},"base_tax_before_discount":{"type":"number"},"weee_tax_row_disposition":{"type":"number"},"base_weee_tax_disposition":{"type":"number"},"weee_tax_applied_row_amount":{"type":"number"},"base_weee_tax_applied_amount":{"type":"number"},"base_weee_tax_row_disposition":{"type":"number"},"base_weee_tax_applied_row_amnt":{"type":"number"},"discount_tax_compensation_amount":{"type":"number"},"discount_tax_compensation_canceled":{"type":"number"},"discount_tax_compensation_invoiced":{"type":"number"},"discount_tax_compensation_refunded":{"type":"number"},"base_discount_tax_compensation_amount":{"type":"number"},"base_discount_tax_compensation_invoiced":{"type":"number"},"base_discount_tax_compensation_refunded":{"type":"number"}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[item_id]],namespace=backendmariadb,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=overwrite,primaryKey=[[item_id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@376ea59a[stream=io.airbyte.protocol.models.AirbyteStream@1783a579[name=MAGENTO_store,jsonSchema={"type":"object","properties":{"code":{"type":"string"},"name":{"type":"string"},"group_id":{"type":"number"},"store_id":{"type":"number"},"is_active":{"type":"number"},"sort_order":{"type":"number"},"website_id":{"type":"number"}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[store_id]],namespace=backendmariadb,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=overwrite,primaryKey=[[store_id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@7bba1ccb[stream=io.airbyte.protocol.models.AirbyteStream@2ffe442d[name=MAGENTO_store_group,jsonSchema={"type":"object","properties":{"code":{"type":"string"},"name":{"type":"string"},"group_id":{"type":"number"},"website_id":{"type":"number"},"default_store_id":{"type":"number"},"root_category_id":{"type":"number"}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[group_id]],namespace=backendmariadb,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=overwrite,primaryKey=[[group_id]],additionalProperties={}], io.airbyte.protocol.models.ConfiguredAirbyteStream@6d7e763d[stream=io.airbyte.protocol.models.AirbyteStream@675cae86[name=MAGENTO_store_website,jsonSchema={"type":"object","properties":{"code":{"type":"string"},"name":{"type":"string"},"is_default":{"type":"number"},"sort_order":{"type":"number"},"website_id":{"type":"number"},"default_group_id":{"type":"number"}}},supportedSyncModes=[full_refresh, incremental],sourceDefinedCursor=<null>,defaultCursorField=[],sourceDefinedPrimaryKey=[[website_id]],namespace=backendmariadb,additionalProperties={}],syncMode=full_refresh,cursorField=[],destinationSyncMode=overwrite,primaryKey=[[website_id]],additionalProperties={}]],additionalProperties={}],failures=[io.airbyte.config.FailureReason@4c2bbae2[failureOrigin=source,failureType=<null>,internalMessage=io.airbyte.workers.DefaultReplicationWorker$SourceException: Source process exited with non-zero exit code 1,externalMessage=Something went wrong within the source connector,metadata=io.airbyte.config.Metadata@11294066[additionalProperties={attemptNumber=1, jobId=20}],stacktrace=java.util.concurrent.CompletionException: io.airbyte.workers.DefaultReplicationWorker$SourceException: Source process exited with non-zero exit code 1
	at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
	at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
	at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1807)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: io.airbyte.workers.DefaultReplicationWorker$SourceException: Source process exited with non-zero exit code 1
	at io.airbyte.workers.DefaultReplicationWorker.lambda$getReplicationRunnable$5(DefaultReplicationWorker.java:312)
	at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
	... 3 more
,retryable=<null>,timestamp=1652713165101]]]
INFOe[m i.a.w.t.TemporalUtils(withBackgroundHeartbeat):235 - Stopping temporal heartbeating...
INFOe[m i.a.c.p.ConfigRepository(updateConnectionState):731 - Updating connection 43cb2e15-3d8d-469b-a7de-fd65a2a318b0 state: io.airbyte.config.State@1cbff3e5[state={}]

Hey @Paulikaite,
You are facing a java.lang.OutOfMemoryError error.
You need to increase your instance memory or change your docker client settings to give more memory to containers.
I also would suggest you upgrade the source-mysql connectors to its latest version (0.5.11), same for Airbyte if you can (we are on 0.38.2)

Hi @alafanechere

Thanks for the swift reply!

I increased the memory limits:

root@h2969161:~# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 1545091
max locked memory       (kbytes, -l) 65536
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) unlimited
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

The error has changed to:

2022-05-19 05:14:49 e[32mINFOe[m i.a.w.w.WorkerRun(call):49 - Executing worker wrapper. Airbyte version: 0.38.4-alpha
2022-05-19 05:14:49 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):108 - Docker volume job log path: /tmp/workspace/32/1/logs.log
2022-05-19 05:14:49 e[32mINFOe[m i.a.w.t.TemporalAttemptExecution(get):113 - Executing worker wrapper. Airbyte version: 0.38.4-alpha
2022-05-19 05:14:49 e[32mINFOe[m i.a.w.DefaultReplicationWorker(run):109 - start sync worker. job id: 32 attempt id: 1
2022-05-19 05:14:49 e[32mINFOe[m i.a.w.DefaultReplicationWorker(run):121 - configured sync modes: {backendmariadb.store=full_refresh - overwrite}
2022-05-19 05:14:49 e[32mINFOe[m i.a.w.p.a.DefaultAirbyteDestination(start):69 - Running destination...
2022-05-19 05:14:49 e[32mINFOe[m i.a.c.i.LineGobbler(voidCall):82 - Checking if airbyte/destination-bigquery:1.1.6 exists...
2022-05-19 05:14:49 e[32mINFOe[m i.a.c.i.LineGobbler(voidCall):82 - airbyte/destination-bigquery:1.1.6 was found locally.
2022-05-19 05:14:49 e[32mINFOe[m i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: 32
2022-05-19 05:14:49 e[32mINFOe[m i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/32/1 --log-driver none --name destination-bigquery-write-32-1-rbjnh --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e WORKER_CONNECTOR_IMAGE=airbyte/destination-bigquery:1.1.6 -e WORKER_JOB_ATTEMPT=1 -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e AIRBYTE_VERSION=0.38.4-alpha -e WORKER_JOB_ID=32 --memory-reservation=4g --memory=4g airbyte/destination-bigquery:1.1.6 write --config destination_config.json --catalog destination_catalog.json
2022-05-19 05:14:49 e[32mINFOe[m i.a.c.i.LineGobbler(voidCall):82 - Checking if airbyte/source-mysql:0.5.11 exists...
2022-05-19 05:14:49 e[32mINFOe[m i.a.c.i.LineGobbler(voidCall):82 - airbyte/source-mysql:0.5.11 was found locally.
2022-05-19 05:14:49 e[32mINFOe[m i.a.w.p.DockerProcessFactory(create):108 - Creating docker job ID: 32
2022-05-19 05:14:49 e[32mINFOe[m i.a.w.p.DockerProcessFactory(create):163 - Preparing command: docker run --rm --init -i -w /data/32/1 --log-driver none --name source-mysql-read-32-1-qkzxy --network host -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -e WORKER_CONNECTOR_IMAGE=airbyte/source-mysql:0.5.11 -e WORKER_JOB_ATTEMPT=1 -e WORKER_ENVIRONMENT=DOCKER -e AIRBYTE_ROLE= -e AIRBYTE_VERSION=0.38.4-alpha -e WORKER_JOB_ID=32 --memory-reservation=4g --memory=4g airbyte/source-mysql:0.5.11 read --config source_config.json --catalog source_catalog.json --state input_state.json
2022-05-19 05:14:49 e[32mINFOe[m i.a.w.DefaultReplicationWorker(run):163 - Waiting for source and destination threads to complete.
2022-05-19 05:14:49 e[32mINFOe[m i.a.w.DefaultReplicationWorker(lambda$getDestinationOutputRunnable$7):382 - Destination output thread started.
2022-05-19 05:14:49 e[32mINFOe[m i.a.w.DefaultReplicationWorker(lambda$getReplicationRunnable$6):292 - Replication thread started.
2022-05-19 05:14:55 e[44msourcee[0m > SLF4J: Class path contains multiple SLF4J bindings.
2022-05-19 05:14:55 e[44msourcee[0m > SLF4J: Found binding in [jar:file:/airbyte/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2022-05-19 05:14:55 e[44msourcee[0m > SLF4J: Found binding in [jar:file:/airbyte/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2022-05-19 05:14:55 e[44msourcee[0m > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2022-05-19 05:14:55 e[44msourcee[0m > SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2022-05-19 05:15:00 e[43mdestinatione[0m > SLF4J: Class path contains multiple SLF4J bindings.
2022-05-19 05:15:00 e[43mdestinatione[0m > SLF4J: Found binding in [jar:file:/airbyte/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2022-05-19 05:15:00 e[43mdestinatione[0m > SLF4J: Found binding in [jar:file:/airbyte/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2022-05-19 05:15:00 e[43mdestinatione[0m > SLF4J: Found binding in [jar:file:/airbyte/lib/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2022-05-19 05:15:00 e[43mdestinatione[0m > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2022-05-19 05:15:00 e[43mdestinatione[0m > SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2022-05-19 05:15:01 e[44msourcee[0m > 2022-05-19 05:15:01 e[32mINFOe[m i.a.i.s.m.MySqlSource(main):210 - starting source: class io.airbyte.integrations.source.mysql.MySqlSource
2022-05-19 05:15:01 e[44msourcee[0m > [11.143s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached.
2022-05-19 05:15:01 e[44msourcee[0m > 2022-05-19 05:15:01,357 main ERROR An exception occurred processing Appender LogSplit org.apache.logging.log4j.core.appender.AppenderLoggingException: java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:165)
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:134)
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:125)
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:89)
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:542)
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:500)
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:483)
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:82)
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.log4j.core.Logger.log(Logger.java:161)
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2205)
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2159)
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2142)
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2034)
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1899)
2022-05-19 05:15:01 e[44msourcee[0m > 	at org.apache.logging.slf4j.Log4jLogger.info(Log4jLogger.java:184)
2022-05-19 05:15:01 e[44msourcee[0m > 	at io.airbyte.integrations.source.mysql.MySqlSource.main(MySqlSource.java:210)
2022-05-19 05:15:01 e[44msourcee[0m > Caused by: java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached

Can it be that:

multiple SLF4J bindings

are causing

main ERROR An exception occurred processing Appender LogSplit org.apache.logging.log4j.core.appender.AppenderLoggingException: java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
?

I increased the memory limits

How did you increase these limits?
I would suggest you increase the memory given to containers in the docker client set up, or upsize your VM instance. You can also increase JOB_MAIN_CONTAINER_MEMORY_REQUEST and JOB_MAIN_CONTAINER_MEMORY_LIMIT env variables value.

Hi, @alafanechere I increased the limits by using ulimit and the same as you suggested

Could you please share the docker ps or docker stats output while you are running the sync?

Couple of schreenshots of docker stats while running the connection:


A colleague find the potential reason: there was thread limit on the server, so we will contact the virtual mashine provider to remove the limit

Nice catch! The output of the docker stats command is not alarming to me, the memory usage is pretty low…