Removing additional meta fields in Airbyte output

Slack Knowledge Base Q&A Platform Questions
, , , ,
1

Summary

Airbyte automatically adds meta fields to the output data. The user wants to know if there is a way to remove these additional fields.


Question

Hey community, I’m testing airbyte and noticing I’m getting additional meta fields in my data that airbyte automatically adds, is there a way to remove them?
my input:

{"id": "AAAAA"}
{"id": "BBBBB"}```
The airbyte output:
```{"_airbyte_raw_id":"f2d826c9-b97a-4de9-8385-7d8c881dec10","_airbyte_extracted_at":1726404844929,"_airbyte_generation_id":1,"_airbyte_meta":{"changes":[],"sync_id":18059475},"_airbyte_data":{"id":"12345","_ab_source_file_last_modified":"2024-09-15T12:44:00.000000Z","_ab_source_file_url":"p2/e1/sample.jsonl"}}
{"_airbyte_raw_id":"ecba6554-9b7e-455b-a0de-8c571c22956e","_airbyte_extracted_at":1726404844929,"_airbyte_generation_id":1,"_airbyte_meta":{"changes":[],"sync_id":18059475},"_airbyte_data":{"id":"AAAAA","_ab_source_file_last_modified":"2024-09-15T12:44:00.000000Z","_ab_source_file_url":"p2/e1/sample.jsonl"}}
{"_airbyte_raw_id":"153b70ab-031a-4b45-bf6d-d734d1d358b4","_airbyte_extracted_at":1726404844930,"_airbyte_generation_id":1,"_airbyte_meta":{"changes":[],"sync_id":18059475},"_airbyte_data":{"id":"BBBBB","_ab_source_file_last_modified":"2024-09-15T12:44:00.000000Z","_ab_source_file_url":"p2/e1/sample.jsonl"}}```

<br>

---

This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. [Click here](https://airbytehq.slack.com/archives/C021JANJ6TY/p1726406659927129) if you want 
to access the original thread.

[Join the conversation on Slack](https://slack.airbyte.com)

<sub>
["airbyte-platform", "remove-meta-fields", "data-output"]
</sub>
2

Hello! I hope https://discuss.airbyte.io/t/hide-airbyte-columns-created-during-sync/2868/2#:~:text=Oct%202022-,No%20this%20is%20not%20possible%20to%20hide%20because%20Airbyte%20uses%20internally%20to%20manage%20the%20syncs.,-Solution|this is still relevant. Perhaps if possible it would be good to use a view that use only the columns that you need?

3

View isn’t a good solution, as we’re using airbyte to deliver data to our costumers, and can’t control the tables in their end.