Jira connector cannot import custom fields for stream 'issues' when list of custom fields is too long

Connector Questions & Issues
,
1
  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Ubuntu
  • Memory / Disk: 4Gb
  • Deployment: Kubernetes
  • Airbyte Version: 0.35.49
  • Source name/version: Jira / 0.2.21
  • Destination name/version: PostgreSQL
  • Step: sync of source
  • Description: Custom fields are only included in the stream ‘issues’ if they are explicitly listed in the source configuration. However, the query to the issue search API raises an exception when this list is too long

400 Client Error: Bad Request for url: https://company.atlassian.net/rest/api/3/search?fields=customfield_10145&fields=customfield_10193&fields=customfield_10110&fields=customfield_10204&fields=customfield_10053&fields=customfield_10028&fields=customfield_10099&fields=customfield_10065&fields=customfield_10054&fields=customfield_10184&fields=customfield_10047&fields=customfield_10095&fields=customfield_10025&fields=customfield_10022&fields=customfield_10086&fields=customfield_10091&fields=status&fields=customfield_10225&fields=customfield_10161&fields=customfield_10104&fields=customfield_10038&fields=customfield_10185&fields=customfield_10018&fields=customfield_10157&fields=customfield_10093&fields=security&fields=customfield_10085&fields=customfield_10037&fields=customfield_10138&fields=customfield_10219&fields=customfield_10068&fields=customfield_10094&fields=customfield_10173&fields=customfield_10155&fields=customfield_10209&fields=customfield_10005&fields=customfield_10153&fields=customfield_10070&fields=customfield_10130&fields=customfield_10004&fields=customfield_10156&fields=customfield_10230&fields=customfield_10207&fields=customfield_10181&fields=customfield_10120&fields=customfield_10119&fields=customfield_10097&fields=customfield_10083&fields=customfield_10020&fields=customfield_10030&fields=customfield_10218&fields=customfield_10075&fields=customfield_10052&fields=customfield_10133&fields=customfield_10229&fields=customfield_10078&fields=customfield_10090&fields=customfield_10171&fields=customfield_10146&fields=customfield_10062&fields=customfield_10122&fields=customfield_10046&fields=customfield_10163&fields=customfield_10164&fields=customfield_10129&fields=customfield_10222&fields=customfield_10149&fields=components&fields=customfield_10040&fields=customfield_10011&fields=customfield_10180&fields=created&fields=customfield_10162&fields=creator&fields=customfield_10051&fields=customfield_10044&fields=customfield_10016&fields=customfield_10071&fields=customfield_10008&fields=customfield_10076&fields=customfield_10002&fields=customfield_10069&fields=customfield_10201&fields=customfield_10144&fields=customfield_10174&fields=customfield_10023&fields=customfield_10178&fields=customfield_10007&fields=customfield_10079&fields=customfield_10041&fields=assignee&fields=customfield_10147&fields=customfield_10034&fields=customfield_10009&fields=customfield_10128&fields=customfield_10108&fields=customfield_10057&fields=customfield_10035&fields=customfield_10059&fields=customfield_10003&fields=customfield_10158&fields=customfield_10172&fields=customfield_10077&fields=customfield_10191&fields=customfield_10063&fields=subtasks&fields=customfield_10039&fields=customfield_10212&fields=updated&fields=customfield_10045&fields=customfield_10217&fields=customfield_10127&fields=customfield_10208&fields=customfield_10199&fields=customfield_10176&fields=customfield_10140&fields=parent&fields=customfield_10012&fields=customfield_10067&fields=customfield_10033&fields=customfield_10186&fields=customfield_10151&fields=customfield_10100&fields=customfield_10088&fields=customfield_10136&fields=customfield_10159&fields=customfield_10137&fields=customfield_10160&fields=description&fields=customfield_10206&fields=customfield_10139&fields=customfield_10098&fields=customfield_10048&fields=customfield_10032&fields=customfield_10055&fields=customfield_10126&fields=customfield_10154&fields=customfield_10220&fields=customfield_10215&fields=customfield_10132&fields=attachment&fields=customfield_10121&fields=customfield_10006&fields=customfield_10135&fields=customfield_10213&fields=customfield_10194&fields=customfield_10170&fields=customfield_10087&fields=customfield_10150&fields=customfield_10214&fields=customfield_10202&fields=customfield_10134&fields=customfield_10042&fields=resolutiondate&fields=customfield_10166&fields=customfield_10082&fields=customfield_10096&fields=customfield_10107&fields=customfield_10125&fields=customfield_10010&fields=customfield_10105&fields=customfield_10141&fields=customfield_10183&fields=customfield_10001&fields=project&fields=labels&fields=customfield_10195&fields=customfield_10031&fields=customfield_10019&fields=customfield_10021&fields=customfield_10177&fields=customfield_10064&fields=customfield_10084&fields=customfield_10056&fields=customfield_10196&fields=customfield_10221&fields=customfield_10192&fields=customfield_10000&fields=summary&fields=customfield_10203&fields=customfield_10050&fields=issuetype&fields=customfield_10226&fields=customfield_10182&fields=customfield_10015&fields=customfield_10049&fields=customfield_10058&fields=customfield_10061&fields=customfield_10043&fields=priority&fields=customfield_10224&fields=customfield_10175&fields=customfield_10148&fields=customfield_10143&fields=customfield_10152&fields=customfield_10089&fields=customfield_10036&fields=customfield_10179&fields=customfield_10200&fields=customfield_10029&fields=customfield_10211&fields=issuelinks&fields=customfield_10216&fields=customfield_10014&fields=customfield_10024&fields=customfield_10197&fields=customfield_10205&fields=customfield_10103&fields=customfield_10210&fields=customfield_10111&fields=customfield_10228&jql=project+%3D+‘10007’+and+updated+>+‘2022%2F01%2F01+00%3A00’
HTTP Status 400 – Bad Requestbody {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}

HTTP Status 400 – Bad Request

Type Exception Report

Message Request header is too large

Description The server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing).

Exception

java.lang.IllegalArgumentException: Request header is too large
2

Hey could you open up an issue in github with the logs of the sync so that team can follow up on this

4

Thank you for your quick reply.
I created the issue → Jira connector cannot import custom fields for stream ‘issues’ when list of custom fields is too long · Issue #16065 · airbytehq/airbyte · GitHub

5

Thanks a lot. Do let me know if you need some help over the PR