Is it possible to retrieve all projects from the JIRA source connector withut explicitly listing them? If I don’t list any project keys the source test fails, I have 100+ projects to potentially import.
Basic normalization also fails for this connector - multiple streams fail with this error
Unsupported nested cursor field {’.’.join(self.cursor_field)} for stream {self.stream_name}\
Hey could you share the logs of the job so that we have better understanding on what is failing
2022-06-02 15:18:49 e[42mnormalizatione[0m > Generating airbyte_ctes/jira_airbyte/application_roles_ab1.sql from application_roles
2022-06-02 15:18:49 e[42mnormalizatione[0m > Generating airbyte_ctes/jira_airbyte/application_roles_ab2.sql from application_roles
2022-06-02 15:18:49 e[42mnormalizatione[0m > Generating airbyte_ctes/jira_airbyte/application_roles_ab3.sql from application_roles
2022-06-02 15:18:49 e[42mnormalizatione[0m > Generating airbyte_tables/jira_airbyte/application_roles.sql from application_roles
2022-06-02 15:18:49 e[42mnormalizatione[0m > Generating airbyte_ctes/jira_airbyte/avatars_ab1.sql from avatars
2022-06-02 15:18:49 e[42mnormalizatione[0m > Generating airbyte_ctes/jira_airbyte/avatars_ab2.sql from avatars
2022-06-02 15:18:49 e[42mnormalizatione[0m > Generating airbyte_ctes/jira_airbyte/avatars_ab3.sql from avatars
2022-06-02 15:18:49 e[42mnormalizatione[0m > Generating airbyte_tables/jira_airbyte/avatars.sql from avatars
2022-06-02 15:18:49 e[42mnormalizatione[0m > Generating airbyte_ctes/jira_airbyte/board_issues_ab1.sql from board_issues
2022-06-02 15:18:49 e[42mnormalizatione[0m > Generating airbyte_ctes/jira_airbyte/board_issues_ab2.sql from board_issues
2022-06-02 15:18:49 e[42mnormalizatione[0m > Generating airbyte_views/jira_airbyte/board_issues_stg.sql from board_issues
2022-06-02 15:18:49 e[42mnormalizatione[0m > {"type": "LOG", "log": {"level": "FATAL", "message": "Unsupported nested cursor field fields.updated for stream board_issues\nTraceback (most recent call last):\n File \"/usr/local/bin/transform-catalog\", line 8, in <module>\n sys.exit(main())\n File \"/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py\", line 104, in main\n TransformCatalog().run(args)\n File \"/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py\", line 36, in run\n self.process_catalog()\n File \"/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py\", line 64, in process_catalog\n processor.process(catalog_file=catalog_file, json_column_name=json_col, default_schema=schema)\n File \"/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/catalog_processor.py\", line 76, in process\n nested_processors = stream_processor.process()\n File \"/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/stream_processor.py\", line 294, in process\n self.generate_scd_type_2_model(from_table, column_names),\n File \"/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/stream_processor.py\", line 699, in generate_scd_type_2_model\n cursor_field = self.get_cursor_field(column_names)\n File \"/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/stream_processor.py\", line 984, in get_cursor_field\n raise ValueError(f\"Unsupported nested cursor field {'.'.join(self.cursor_field)} for stream {self.stream_name}\")\nValueError: Unsupported nested cursor field fields.updated for stream board_issues"}}
2022-06-02 15:18:49 e[42mnormalizatione[0m > {"type": "TRACE", "trace": {"type": "ERROR", "emitted_at": 1654183129210.775, "error": {"message": "Something went wrong in the connector. See the logs for more details.", "internal_message": "Unsupported nested cursor field fields.updated for stream board_issues", "stack_trace": "Traceback (most recent call last):\n File \"/usr/local/bin/transform-catalog\", line 8, in <module>\n sys.exit(main())\n File \"/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py\", line 104, in main\n TransformCatalog().run(args)\n File \"/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py\", line 36, in run\n self.process_catalog()\n File \"/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/transform.py\", line 64, in process_catalog\n processor.process(catalog_file=catalog_file, json_column_name=json_col, default_schema=schema)\n File \"/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/catalog_processor.py\", line 76, in process\n nested_processors = stream_processor.process()\n File \"/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/stream_processor.py\", line 294, in process\n self.generate_scd_type_2_model(from_table, column_names),\n File \"/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/stream_processor.py\", line 699, in generate_scd_type_2_model\n cursor_field = self.get_cursor_field(column_names)\n File \"/usr/local/lib/python3.9/site-packages/normalization/transform_catalog/stream_processor.py\", line 984, in get_cursor_field\n raise ValueError(f\"Unsupported nested cursor field {'.'.join(self.cursor_field)} for stream {self.stream_name}\")\nValueError: Unsupported nested cursor field fields.updated for stream board_issues\n", "failure_type": "system_error"}}}
2022-06-02 15:18:49 e[42mnormalizatione[0m >
Is retrieving all projects possible or each key needs to be specified?
Hey I think the issue is same as https://github.com/airbytehq/airbyte/issues/8473. Do you mind taking this over the github issue
I don’t care about the normalization error, will use my own. More concerned with the project keys, do they need to be specified or is there a wildcard I can use to retrieve them all?
I don’t think we have a wildcard option right now
Unless it goes in a loop. If there is a problem do you mind creating a new topic with logs of that sync
The issue is with the issue_properties stream. It takes more than 2 days to execute vs 45 minutes for all other streams
Got it. Kindly create a new ticket/issue with the information so that team can look into
Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.
