Using dpath for record selector in Airbyte connector


The user is trying to extract a full list of ‘id’ values from a nested JSON object using dpath for the record selector in an Airbyte connector.


According to*slvvfm*_gcl_au*MTUxNDQ2OTA2Mi4xNzEyMDY1MzQx|this I can use “dpath” for record selector So I wrote some python test test out getting all the “values” I want to pass to a “sub-stream” and it spits out all the id I need in the next stream.

import dpath.util

# Load JSON data from a file
with open('input.json', 'r') as file:
    data = json.load(file)

# Traverse the JSON data and find all values associated with 'id'
ids = list(dpath.util.values(data, '**/id'))

# Print the values found for 'id'
for id_value in ids:
But when I try and put that in the `**` in the record selector I just get an error. Basically I need to get a "list" of all `id` that exist in a json graph.

How can I get the full list of `id` from a json object (it is nested data) so `blah, *`  does not get me there.



This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here.
[Join the conversation on Slack](

[Click here]( if you want to access the original thread.

`["dpath", "record-selector", "json-object", "nested-data", "id-values"]`

This appears to be true:;cid=C01AHCD885S

Yeah I don’t think the dpath docs explain this very well. In the Builder, you’ll need to provide the full path to the field you want to extract as the record, and each * means it should extract all values from the array as records at its current level