Source MSSQL- Not recognizing data types

[davison]

• Is this your first time deploying Airbyte?: No
• OS Version / Instance: Ubuntu
• Memory / Disk: 16Gb / 100 Gb
• Deployment: Kubernetes deployment?
• Airbyte Version: 0.40.18?
• Source name/version: 0.4.26
• Destination name/version:
• Step: The issue is happening during creating the source connection
• Description: I configured the SQL Server source connector for the CDC process, the CDC is working normally but some numerical columns of the tables do not appear mapped by the SQL Server source connector, in this case the “QT_EQUIP” column. There are other numerical columns with the same problem, I would like some help if it is possible to edit and add the column that the connector didn’t map?
  Thanks

[a0de5f4e_02c4_44e7_9c2b_32668a1d315f_logs_6889_txt.txt|attachment](upload://4JIfVvoVsDqRxrFSFgzMG9sX3jy.txt) (189.6 KB)

[marcosmarxm]

Hello there! You are receiving this message because none of your fellow community members has stepped in to respond to your topic post. (If you are a community member and you are reading this response, feel free to jump in if you have the answer!) As a result, the Community Assistance Team has been made aware of this topic and will be investigating and responding as quickly as possible.
Some important considerations that will help your to get your issue solved faster:

• It is best to use our topic creation template; if you haven’t yet, we recommend posting a followup with the requested information. With that information the team will be able to more quickly search for similar issues with connectors and the platform and troubleshoot more quickly your specific question or problem.
• Make sure to upload the complete log file; a common investigation roadblock is that sometimes the error for the issue happens well before the problem is surfaced to the user, and so having the tail of the log is less useful than having the whole log to scan through.
• Be as descriptive and specific as possible; when investigating it is extremely valuable to know what steps were taken to encounter the issue, what version of connector / platform / Java / Python / docker / k8s was used, etc. The more context supplied, the quicker the investigation can start on your topic and the faster we can drive towards an answer.
• We in the Community Assistance Team are glad you’ve made yourself part of our community, and we’ll do our best to answer your questions and resolve the problems as quickly as possible. Expect to hear from a specific team member as soon as possible.

Thank you for your time and attention.
Best,
The Community Assistance Team

[sajarin]

Hey @davidson,

Thanks for the question and welcome to the community. Could you describe what the expected data type mapping should be for the QT_EQUIP column and what Airbyte maps it as? How many other columns are showcasing the same behavior and could you describe the expected mappings for those as well?

Can you also post the logs once more, it seems like they didn’t get uploaded.

Thanks for your patience.

[davison]

Thanks for helping @sajarin. In the source the field that has a different numerical typing that does not carry the column “QT_EQUIP” in the SQL Server source connector, this typing works in the source database but in the sQL connector in Airbyte this column does not appear.
View source table at Dbeaver

Here does not appear
View source table connector in Airbyte

[sajarin]

Hello rebechi, it’s been a while without an update from us. Are you still having problems or did you find a solution?

[philippeboyd]

@sajarin I’m having the same issue with Microsoft SQL Server 2019.

I have the following in the SQL table

COLUMN_NAME                  DATA_TYPE  
---------------------------- -----------
BnkId                        varchar    
Name                         varchar    
AcctNbr                      varchar    
RouteNbr                     varchar    
Addr1                        varchar    
Addr2                        varchar    
City                         varchar    
State                        varchar    
Post                         varchar    
Phone1                       varchar    
Phone2                       varchar    
Fax                          varchar    
Contact                      varchar    
Email                        varchar    
DepTypId                     varchar    
MemTypId                     varchar    
CheckAuto                    int        
CheckPrefix                  varchar    
CheckLast                    int        
TS                           timestamp  

And only TS gets mapped.

I have the same behaviour on 247 other tables. Would it be a timeout from Airbyte that it’s skipping the mapping?

[philippeboyd]

@sajarin

I think I found the issue as to why Airbyte is not mapping my columns. They are of type Unknown to the JDBC connector

According to the code in public JsonSchemaType getAirbyteType(final JDBCType jdbcType), shouldn’t they be mapped to VARCHAR?