Hello,
I don’t understand why the sync scheduling stopped for a connector while sync is enabled
Do you know why ?
Example with the Salesloft connector below
Hello,
I don’t understand why the sync scheduling stopped for a connector while sync is enabled
Do you know why ?
Example with the Salesloft connector below
Sorry the long delay in replying Yanni. Harshith Mullapudi can you take a look Monday on this issue? Try to identify how the scheduler get the value from the database and launch the next sync please
Sure will take a look. Also is it possible to share Scheduler logs here
What’s is the best way get the scheduler logs ?
I can’t download files from the interface
I will get the docker service logs on the vm otherwise
ok @harshith i found part of the scheduler logs (via docker logs airbyte-scheduler
)
I can see an error
2022-04-22 07:36:36 ERROR i.a.s.a.JobSubmitter(run):81 - Job Submitter Error
org.jooq.exception.DataAccessException: Error getting connection from data source org.apache.commons.dbcp2.BasicDataSource@5c37accd
at org.jooq_3.13.4.POSTGRES.debug(Unknown Source) ~[?:?]
at org.jooq.impl.DataSourceConnectionProvider.acquire(DataSourceConnectionProvider.java:86) ~[jooq-3.13.4.jar:?]
at org.jooq.impl.DefaultExecuteContext.connection(DefaultExecuteContext.java:647) ~[jooq-3.13.4.jar:?]
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:334) ~[jooq-3.13.4.jar:?]
at org.jooq.impl.AbstractResultQuery.fetch(AbstractResultQuery.java:354) ~[jooq-3.13.4.jar:?]
at org.jooq.impl.DefaultDSLContext.fetch(DefaultDSLContext.java:892) ~[jooq-3.13.4.jar:?]
at io.airbyte.scheduler.persistence.DefaultJobPersistence.lambda$getNextJob$21(DefaultJobPersistence.java:410) ~[io.airbyte.airbyte-scheduler-persistence-0.35.51-alpha.jar:?]
at io.airbyte.db.Database.query(Database.java:27) ~[io.airbyte.airbyte-db-lib-0.35.51-alpha.jar:?]
at io.airbyte.db.ExceptionWrappingDatabase.query(ExceptionWrappingDatabase.java:23) ~[io.airbyte.airbyte-db-lib-0.35.51-alpha.jar:?]
at io.airbyte.scheduler.persistence.DefaultJobPersistence.getNextJob(DefaultJobPersistence.java:409) ~[io.airbyte.airbyte-scheduler-persistence-0.35.51-alpha.jar:?]
at io.airbyte.scheduler.app.JobSubmitter.run(JobSubmitter.java:74) ~[io.airbyte.airbyte-scheduler-app-0.35.51-alpha.jar:?]
at io.airbyte.scheduler.app.SchedulerApp.lambda$start$1(SchedulerApp.java:167) ~[io.airbyte.airbyte-scheduler-app-0.35.51-alpha.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: org.postgresql.util.PSQLException: The connection attempt failed.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:313) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:225) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.Driver.makeConnection(Driver.java:466) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.Driver.connect(Driver.java:265) ~[postgresql-42.3.1.jar:42.3.1]
at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:55) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:355) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:889) ~[commons-pool2-2.7.0.jar:2.7.0]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:424) ~[commons-pool2-2.7.0.jar:2.7.0]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:349) ~[commons-pool2-2.7.0.jar:2.7.0]
at org.apache.commons.dbcp2.PoolingDataSource.getConnection(PoolingDataSource.java:134) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:753) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.jooq.impl.DataSourceConnectionProvider.acquire(DataSourceConnectionProvider.java:83) ~[jooq-3.13.4.jar:?]
... 16 more
Caused by: java.net.SocketTimeoutException: Connect timed out
at sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:546) ~[?:?]
at sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597) ~[?:?]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) ~[?:?]
at java.net.Socket.connect(Socket.java:633) ~[?:?]
at org.postgresql.core.PGStream.createSocket(PGStream.java:238) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.core.PGStream.<init>(PGStream.java:98) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:100) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:215) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:225) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.Driver.makeConnection(Driver.java:466) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.Driver.connect(Driver.java:265) ~[postgresql-42.3.1.jar:42.3.1]
at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:55) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:355) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:889) ~[commons-pool2-2.7.0.jar:2.7.0]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:424) ~[commons-pool2-2.7.0.jar:2.7.0]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:349) ~[commons-pool2-2.7.0.jar:2.7.0]
at org.apache.commons.dbcp2.PoolingDataSource.getConnection(PoolingDataSource.java:134) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:753) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.jooq.impl.DataSourceConnectionProvider.acquire(DataSourceConnectionProvider.java:83) ~[jooq-3.13.4.jar:?]
... 16 more
2022-04-22 07:36:49 ERROR i.a.s.a.JobSubmitter(run):81 - Job Submitter Error
org.jooq.exception.DataAccessException: Error getting connection from data source org.apache.commons.dbcp2.BasicDataSource@5c37accd
at org.jooq_3.13.4.POSTGRES.debug(Unknown Source) ~[?:?]
at org.jooq.impl.DataSourceConnectionProvider.acquire(DataSourceConnectionProvider.java:86) ~[jooq-3.13.4.jar:?]
at org.jooq.impl.DefaultExecuteContext.connection(DefaultExecuteContext.java:647) ~[jooq-3.13.4.jar:?]
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:334) ~[jooq-3.13.4.jar:?]
at org.jooq.impl.AbstractResultQuery.fetch(AbstractResultQuery.java:354) ~[jooq-3.13.4.jar:?]
at org.jooq.impl.DefaultDSLContext.fetch(DefaultDSLContext.java:892) ~[jooq-3.13.4.jar:?]
at io.airbyte.scheduler.persistence.DefaultJobPersistence.lambda$getNextJob$21(DefaultJobPersistence.java:410) ~[io.airbyte.airbyte-scheduler-persistence-0.35.51-alpha.jar:?]
at io.airbyte.db.Database.query(Database.java:27) ~[io.airbyte.airbyte-db-lib-0.35.51-alpha.jar:?]
at io.airbyte.db.ExceptionWrappingDatabase.query(ExceptionWrappingDatabase.java:23) ~[io.airbyte.airbyte-db-lib-0.35.51-alpha.jar:?]
at io.airbyte.scheduler.persistence.DefaultJobPersistence.getNextJob(DefaultJobPersistence.java:409) ~[io.airbyte.airbyte-scheduler-persistence-0.35.51-alpha.jar:?]
at io.airbyte.scheduler.app.JobSubmitter.run(JobSubmitter.java:74) ~[io.airbyte.airbyte-scheduler-app-0.35.51-alpha.jar:?]
at io.airbyte.scheduler.app.SchedulerApp.lambda$start$1(SchedulerApp.java:167) ~[io.airbyte.airbyte-scheduler-app-0.35.51-alpha.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: org.postgresql.util.PSQLException: Connection to 35.187.19.193:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:301) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:225) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.Driver.makeConnection(Driver.java:466) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.Driver.connect(Driver.java:265) ~[postgresql-42.3.1.jar:42.3.1]
at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:55) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:355) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:889) ~[commons-pool2-2.7.0.jar:2.7.0]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:424) ~[commons-pool2-2.7.0.jar:2.7.0]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:349) ~[commons-pool2-2.7.0.jar:2.7.0]
at org.apache.commons.dbcp2.PoolingDataSource.getConnection(PoolingDataSource.java:134) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:753) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.jooq.impl.DataSourceConnectionProvider.acquire(DataSourceConnectionProvider.java:83) ~[jooq-3.13.4.jar:?]
... 16 more
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.pollConnect(Native Method) ~[?:?]
at sun.nio.ch.Net.pollConnectNow(Net.java:672) ~[?:?]
at sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:549) ~[?:?]
at sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597) ~[?:?]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) ~[?:?]
at java.net.Socket.connect(Socket.java:633) ~[?:?]
at org.postgresql.core.PGStream.createSocket(PGStream.java:238) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.core.PGStream.<init>(PGStream.java:98) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:100) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:215) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:51) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:225) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.Driver.makeConnection(Driver.java:466) ~[postgresql-42.3.1.jar:42.3.1]
at org.postgresql.Driver.connect(Driver.java:265) ~[postgresql-42.3.1.jar:42.3.1]
at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:55) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:355) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:889) ~[commons-pool2-2.7.0.jar:2.7.0]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:424) ~[commons-pool2-2.7.0.jar:2.7.0]
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:349) ~[commons-pool2-2.7.0.jar:2.7.0]
at org.apache.commons.dbcp2.PoolingDataSource.getConnection(PoolingDataSource.java:134) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:753) ~[commons-dbcp2-2.7.0.jar:2.7.0]
at org.jooq.impl.DataSourceConnectionProvider.acquire(DataSourceConnectionProvider.java:83) ~[jooq-3.13.4.jar:?]
scheduler.log (122.8 KB)
I forgot to mention but i deployed airflow on a compte engine + cloud sql
Machine type: e2-standard-2
CloudSQL : PostgreSQL 13 | vCPUs = 1 | Memory= 1.7 GB | SSD storage = 10 GB
Hey I think it was not able to connect to the database do you mind restarting the Airbyte and also check parallelly on why the database is not reachable
now it’s working now because i click on sync now
but i’m wondering why it stopped completely and don’t retry several times after it got the error
I feel it’s because the scheduler was not able to reach the database it was not having any information
ok fair enough
I was really expecting a retry on this one but ok good to know
Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.