Connecting to Oracle - Finding SID (Oracle System Identifier)

troble · July 18, 2022, 4:30am

Hi, trying to setup Airbyte to replicate data from Oracle to SQL, but getting stuck at connecting. In the logs i see “ORA-12505, TNS:listener does not currently know of SID given in connect descriptor”

Is this your first time deploying Airbyte?: Yes
OS Version / Instance: Ubuntu
Memory / Disk: 8Gb / 1 Tb
Deployment: Docker
Airbyte Version: 0.39.37-alpha
Source name/version: Oracle
Destination name/version: SQL Server
Step: Setting up for the first time, when I click on ‘Set up source’
Description: When i try to connect, i get “ORA-12505, TNS:listener does not currently know of SID given in connect descriptor”. This oracle server isnt ours, we have been given read access from the SAAS provider. We can use Oracle SQL Server to run queries, and googling i have
tried to use the output of SELECT sys_context(‘USERENV’, ‘SID’) FROM DUAL; - but i still get the same error. The strange thing is each time i restart Oracle SQL Developer, the outpt of that query changes (so guessing thats not helpful)

natalyjazzviolin · July 19, 2022, 4:20pm

Hello and welcome to the community, @troble! Looks like there’s an open issue for the error you’re getting here:
https://github.com/airbytehq/airbyte/issues/11348

I’m linking your post to it internally. There has been some activity on it very recently, so somebody should be working on a solution.

troble · July 26, 2022, 12:54pm

I see there has been progress on that issue, i thought i would give it a go.
I did a git clone, then changed to the branch that had the changes. I could now see the difference in the files as per that branch.
I renamed docker-compose.build.yaml to docker-compose.yaml, and tried to do a docker-compose up (and even a docker-compose build), but i get the following error.

WARNING: The GIT_REVISION variable is not set. Defaulting to a blank string.
Building init
Step 1/5 : ARG ALPINE_IMAGE=alpine:3.4
Step 2/5 : FROM ${ALPINE_IMAGE} AS seed
 ---> b7c5ffe56db7
Step 3/5 : WORKDIR /app
 ---> Using cache
 ---> c2a0c04d8818
Step 4/5 : COPY bin/scripts scripts
ERROR: Service 'init' failed to build: COPY failed: stat /mnt/volume_sfo2_12/docker/tmp/docker-builder651947131/bin/scripts: no such file or directory

The folder docker-builderXXXXX folder name changes each time i run it, but the same results.

Any suggestions?

natalyjazzviolin · July 27, 2022, 1:27pm

I found a potential fix, but it is quite old:
https://github.com/airbytehq/airbyte/issues/6850#issuecomment-939702746

Let me know if this helps!

troble · August 4, 2022, 11:54am

Thanks, but that didnt work. Even just compiling doesnt.

Can anyone point me to where i can find the documentation on getting up and compiling?

I Installed java 17, node 16, npm, python 3.9, docker, docker-compose and jq.
git clone https://github.com/airbytehq/airbyte.gi.
cd airbyte
SUB_BUILD=PLATFORM sudo ./gradlew composebuild

That fails, i even try just
SUB_BUILD=PLATFORM sudo ./gradlew build

I keep getting FAILURE: Build failed with an exception.

Thanks

natalyjazzviolin · August 4, 2022, 8:06pm

Sorry to hear that compiling isn’t working! Are you following these docs?
https://docs.airbyte.com/contributing-to-airbyte/developing-locally/

Are you getting any more logs?

I’m working on reproducing your initial issue internally and hope to have some more ideas for you tomorrow.

troble · August 4, 2022, 11:50pm

Hi

I was using the instructions on that link. What i found was it was failing at different points, and those errors never really helped. What i did which improved things was increase the memory for the VM i am using to compile, and that resolved and has now got me to the 1 error i always now get

> Task :airbyte-webapp:npmInstall FAILED
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: react-dom@16.14.0
npm WARN Found: react@17.0.2
npm WARN node_modules/react
npm WARN   react@"^17.0.2" from the root project
npm WARN   74 more (@emotion/react, @fortawesome/react-fontawesome, ...)
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer react@"^16.14.0" from react-dom@16.14.0
npm WARN node_modules/react-lazylog/node_modules/react-dom
npm WARN   peer react-dom@"^15.3.0 || ^16.0.0-alpha" from react-virtualized@9.22.3
npm WARN   node_modules/react-lazylog/node_modules/react-virtualized
npm WARN 
npm WARN Conflicting peer dependency: react@16.14.0
npm WARN node_modules/react
npm WARN   peer react@"^16.14.0" from react-dom@16.14.0
npm WARN   node_modules/react-lazylog/node_modules/react-dom
npm WARN     peer react-dom@"^15.3.0 || ^16.0.0-alpha" from react-virtualized@9.22.3
npm WARN     node_modules/react-lazylog/node_modules/react-virtualized
npm WARN ERESOLVE overriding peer dependency
npm WARN While resolving: react-virtualized@9.22.3
npm WARN Found: react@17.0.2
npm WARN node_modules/react
npm WARN   react@"^17.0.2" from the root project
npm WARN   74 more (@emotion/react, @fortawesome/react-fontawesome, ...)
npm WARN 
npm WARN Could not resolve dependency:
npm WARN peer react@"^15.3.0 || ^16.0.0-alpha" from react-virtualized@9.22.3
npm WARN node_modules/react-lazylog/node_modules/react-virtualized
npm WARN   react-virtualized@"^9.21.0" from react-lazylog@4.5.3
npm WARN   node_modules/react-lazylog
npm WARN 
npm WARN Conflicting peer dependency: react@16.14.0
npm WARN node_modules/react
npm WARN   peer react@"^15.3.0 || ^16.0.0-alpha" from react-virtualized@9.22.3
npm WARN   node_modules/react-lazylog/node_modules/react-virtualized
npm WARN     react-virtualized@"^9.21.0" from react-lazylog@4.5.3
npm WARN     node_modules/react-lazylog
npm ERR! code ENOTEMPTY
npm ERR! syscall rename
npm ERR! path /home/paul/airbyte/airbyte-webapp/node_modules/ajv-formats
npm ERR! dest /home/paul/airbyte/airbyte-webapp/node_modules/.ajv-formats-2GLUqjrf
npm ERR! errno -39
npm ERR! ENOTEMPTY: directory not empty, rename '/home/paul/airbyte/airbyte-webapp/node_modules/ajv-formats' -> '/home/paul/airbyte/airbyte-webapp/node_modules/.ajv-formats-2GLUqjrf'

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2022-08-04T23_20_20_189Z-debug-0.log

I tried to just remove the /home/paul/airbyte/airbyte-webapp/node_modules/ajv-formats folder, but on next run of SUB_BUILD=PLATFORM ./gradlew build (as root) gave me the same error.

troble · August 5, 2022, 8:01am

Hi, I upgraded node to 16.16.0 as i saw something in a log showing the version might of been lowere than required.

When i do a SUB_BUILD=PLATFORM ./gradlew build --info i see at the end this error:

Watching 648 directories to track changes
:buildDockerImage-server (Thread[included builds,5,main]) completed. Took 16.28 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':airbyte-webapp:npm_run_build'.
> Process 'command '/home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/bin/npm'' finished with non-zero exit value 243

And looking up further i can see its related to teh webapp:

> Task :airbyte-webapp:npm_run_build FAILED

Watching 642 directories to track changes
:airbyte-webapp:npm_run_build (Thread[Execution worker for ':',5,main]) completed. Took 2.247 secs.

I tried to just build the airbyte-webapp but its giving me the similar issue

SUB_BUILD=PLATFORM ./gradlew :airbyte-webapp:npm_run_build
Building Airbyte Sub Build: PLATFORM
Type-safe dependency accessors is an incubating feature.

> Task :airbyte-webapp:npm_run_build FAILED

> airbyte-webapp@0.39.41-alpha prebuild
> npm run generate-client



FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':airbyte-webapp:npm_run_build'.
> Process 'command '/home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/bin/npm'' finished with non-zero exit value 243

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3s
13 actionable tasks: 1 executed, 12 up-to-date

running it with --info gives me:

> Task :airbyte-webapp:npmInstall UP-TO-DATE
Watching 24611 directories to track changes
Caching disabled for task ':airbyte-webapp:npmInstall' because:
  Caching has not been enabled for the task
Skipping task ':airbyte-webapp:npmInstall' as it is up-to-date.
:airbyte-webapp:npmInstall (Thread[Execution worker for ':',5,main]) completed. Took 1.301 secs.
:airbyte-webapp:npm_run_build (Thread[Execution worker for ':',5,main]) started.

> Task :airbyte-webapp:npm_run_build FAILED
Caching disabled for task ':airbyte-webapp:npm_run_build' because:
  Caching has not been enabled for the task
Task ':airbyte-webapp:npm_run_build' is not up-to-date because:
  Task has failed previously.
Watching 24610 directories to track changes
Starting process 'command '/home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/bin/npm''. Working directory: /home/paul/airbyte2/airbyte/airbyte-webapp Command: /home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/bin/npm run build
Successfully started process 'command '/home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/bin/npm''

> airbyte-webapp@0.39.41-alpha prebuild
> npm run generate-client


Watching 24611 directories to track changes
:airbyte-webapp:npm_run_build (Thread[Execution worker for ':',5,main]) completed. Took 0.604 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':airbyte-webapp:npm_run_build'.
> Process 'command '/home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/bin/npm'' finished with non-zero exit value 243

When i run it wth debug, these lines are the ones that could be of intrest

2022-08-05T15:02:08.962+1000 [LIFECYCLE] [class org.gradle.internal.buildevents.TaskExecutionLogger] > Task :airbyte-webapp:npm_run_build
2022-08-05T15:02:08.962+1000 [DEBUG] [org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter] Putting task artifact state for task ':airbyte-webapp:npm_run_build' into context took 0.0 secs.
2022-08-05T15:02:08.962+1000 [DEBUG] [org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep] Implementation for task ':airbyte-webapp:npm_run_build': com.github.gradle.node.npm.task.NpmTask_Decorated@5c357d326515d97e8a3596a1c29c6ac7
2022-08-05T15:02:08.962+1000 [DEBUG] [org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep] Additional implementations for task ':airbyte-webapp:npm_run_build': [com.github.gradle.node.npm.task.NpmTask_Decorated@5c357d326515d97e8a3596a1c29c6ac7]
2022-08-05T15:02:08.963+1000 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Snapshot inputs and outputs before executing task ':airbyte-webapp:npm_run_build'' completed
2022-08-05T15:02:08.964+1000 [INFO] [org.gradle.internal.execution.steps.ResolveCachingStateStep] Caching disabled for task ':airbyte-webapp:npm_run_build' because:
  Caching has not been enabled for the task
2022-08-05T15:02:08.964+1000 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Snapshot task inputs for :airbyte-webapp:npm_run_build' completed
2022-08-05T15:02:08.964+1000 [DEBUG] [org.gradle.internal.execution.steps.SkipUpToDateStep] Determining if task ':airbyte-webapp:npm_run_build' is up-to-date
2022-08-05T15:02:08.964+1000 [INFO] [org.gradle.internal.execution.steps.SkipUpToDateStep] Task ':airbyte-webapp:npm_run_build' is not up-to-date because:
  Task has failed previously.
2022-08-05T15:02:08.964+1000 [DEBUG] [org.gradle.internal.vfs.impl.AbstractVirtualFileSystem] Invalidating VFS paths: [/home/paul/airbyte2/airbyte/airbyte-webapp/build/app]
2022-08-05T15:02:08.964+1000 [INFO] [org.gradle.internal.watch.registry.impl.NonHierarchicalFileWatcherUpdater] Watching 24345 directories to track changes
2022-08-05T15:02:08.964+1000 [DEBUG] [org.gradle.internal.execution.steps.CreateOutputsStep] Ensuring directory exists for property $1 at /home/paul/airbyte2/airbyte/airbyte-webapp/build/app
2022-08-05T15:02:08.965+1000 [DEBUG] [org.gradle.api.internal.tasks.execution.TaskExecution] Executing actions for task ':airbyte-webapp:npm_run_build'.
2022-08-05T15:02:08.965+1000 [INFO] [org.gradle.process.internal.DefaultExecHandle] Starting process 'command '/home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/bin/npm''. Working directory: /home/paul/airbyte2/airbyte/airbyte-webapp Command: /home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/bin/npm run build
2022-08-05T15:02:08.965+1000 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Changing state to: STARTING
2022-08-05T15:02:08.966+1000 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Waiting until process started: command '/home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/bin/npm'.
2022-08-05T15:02:08.968+1000 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Changing state to: STARTED
2022-08-05T15:02:08.968+1000 [DEBUG] [org.gradle.process.internal.ExecHandleRunner] waiting until streams are handled...
2022-08-05T15:02:08.969+1000 [INFO] [org.gradle.process.internal.DefaultExecHandle] Successfully started process 'command '/home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/bin/npm''
2022-08-05T15:02:09.338+1000 [QUIET] [system.out] 
2022-08-05T15:02:09.338+1000 [QUIET] [system.out] > airbyte-webapp@0.39.41-alpha prebuild
2022-08-05T15:02:09.338+1000 [QUIET] [system.out] > npm run generate-client
2022-08-05T15:02:09.338+1000 [QUIET] [system.out] 
2022-08-05T15:02:09.672+1000 [ERROR] [system.err] 
2022-08-05T15:02:09.684+1000 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Changing state to: FAILED
2022-08-05T15:02:09.684+1000 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Process 'command '/home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/bin/npm'' finished with exit value 243 (state: FAILED)
2022-08-05T15:02:09.684+1000 [LIFECYCLE] [org.gradle.internal.operations.DefaultBuildOperationRunner] 
2022-08-05T15:02:09.684+1000 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Completing Build operation 'Execute exec for :airbyte-webapp:npm_run_build'
2022-08-05T15:02:09.684+1000 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Completing Build operation 'Executing task ':airbyte-webapp:npm_run_build''
2022-08-05T15:02:09.685+1000 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Build operation 'Snapshot outputs after executing task ':airbyte-webapp:npm_run_build'' started
2022-08-05T15:02:09.685+1000 [DEBUG] [org.gradle.internal.operations.DefaultBuildOperationRunner] Completing Build operation 'Snapshot outputs after executing task ':airbyte-webapp:npm_run_build''
2022-08-05T15:02:09.685+1000 [DEBUG] [org.gradle.cache.internal.btree.BTreePersistentIndexedCache] Opening cache outputFiles.bin (/home/paul/airbyte2/airbyte/.gradle/buildOutputCleanup/outputFiles.bin)
2022-08-05T15:02:11.284+1000 [LIFECYCLE] [class org.gradle.internal.buildevents.TaskExecutionLogger] 
2022-08-05T15:02:11.284+1000 [LIFECYCLE] [class org.gradle.internal.buildevents.TaskExecutionLogger] > Task :airbyte-webapp:npm_run_build FAILED

If i go into /home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/bin and run npm run build, i just get this output:

> airbyte-webapp@0.39.41-alpha prebuild
> npm run generate-client

Looking at the npm log file i see this:

0 verbose cli /home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/bin/node /home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/bin/npm
1 info using npm@8.11.0
2 info using node@v16.15.1
3 timing npm:load:whichnode Completed in 0ms
4 timing config:load:defaults Completed in 1ms
5 timing config:load:file:/home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/lib/node_modules/npm/npmrc Completed in 1ms
6 timing config:load:builtin Completed in 1ms
7 timing config:load:cli Completed in 1ms
8 timing config:load:env Completed in 1ms
9 timing config:load:file:/home/paul/airbyte2/airbyte/airbyte-webapp/.npmrc Completed in 1ms
10 timing config:load:project Completed in 4ms
11 timing config:load:file:/root/.npmrc Completed in 1ms
12 timing config:load:user Completed in 1ms
13 timing config:load:file:/home/paul/airbyte2/airbyte/airbyte-webapp/.gradle/nodejs/node-v16.15.1-linux-x64/etc/npmrc Completed in 0ms
14 timing config:load:global Completed in 0ms
15 timing config:load:validate Completed in 1ms
16 timing config:load:credentials Completed in 0ms
17 timing config:load:setEnvs Completed in 1ms
18 timing config:load Completed in 11ms
19 timing npm:load:configload Completed in 11ms
20 timing npm:load:mkdirpcache Completed in 1ms
21 timing npm:load:mkdirplogs Completed in 1ms
22 verbose title npm run build
23 verbose argv "run" "build"
24 timing npm:load:setTitle Completed in 1ms
25 timing config:load:flatten Completed in 2ms
26 timing npm:load:display Completed in 2ms
27 verbose logfile logs-max:10 dir:/root/.npm/_logs
28 verbose logfile /root/.npm/_logs/2022-08-05T04_43_44_012Z-debug-0.log
29 timing npm:load:logFile Completed in 5ms
30 timing npm:load:timers Completed in 0ms
31 timing npm:load:configScope Completed in 0ms
32 timing npm:load Completed in 22ms
33 silly logfile start cleaning logs, removing 1 files
34 silly logfile done cleaning log files
35 timing command:run Completed in 291ms
36 verbose exit 243
37 timing npm Completed in 318ms
38 verbose code 243

I also tried to upgrade node th 16.15.1, no changes.

I am lost as to what else i can try. My VM is Ubuntu 22.04 LTS

natalyjazzviolin · August 5, 2022, 1:51pm

Re: increasing VM memory - this is a common issue, I’m glad you were able to figure it out!

And re:airbyte-webapp:npm_run_build FAILED - I can’t find any related issues or fixes, while I look into that - could you try a fresh install of Airbyte? We typically run npm scripts in the /airbyte/airbyte-webapp folder, not in the bin folder

Topic		Replies	Views
Using service name to set up Oracle Db as destination Connector Development destination-oracle	4	752	December 13, 2022
Could not connect with provided configuration. ORA-00972: identifier is too long Connector Development destination-oracle , data-loading , connectors , deploy	3	396	December 14, 2022
Non-root ORACLE connector Connector Questions & Issues source-oracle-db , connectors	4	219	August 22, 2022
Sync fails on Oracle Destination connector Connector Questions & Issues connectors	15	802	July 12, 2022
MSSQL - Failed to fetch schema Connector Questions & Issues source-microsoft-sql-server-mssql	2	474	December 20, 2022

Connecting to Oracle - Finding SID (Oracle System Identifier)

Related topics