Summary
Check if Airbyte supports Reddit as a data source
Question
Hi all, I have a light use case for the Reddit API that will stay within their limits. Does Airbyte have Reddit as a source? I couldn’t find anything online.
This topic has been created from a Slack thread to give it more visibility.
It will be on Read-Only mode here. Click here if you want
to access the original thread.
Join the conversation on Slack
["reddit-api", "airbyte-source", "connector"]
At the moment no it doesn’t have one . If I’m not wrong Reddit doesn’t have a API allows you to retrieve posts in a subreddit using REST or deterministic, correct? Basically you can retrieve only the news and hot discussions with some period limitation.
I’ve been running some python scripts on cron that grab posts and comments. something like this:
headers = {'User-Agent': 'Mozilla/5.0'}
response = requests.get(url, headers=headers)
for post in response.json()['data']['children']:
xyz..```
the limitation is 100 request every 10 minutes, and if you use oAuth you can do 1000 every 10 mins.
I’d love to have a low-code Reddit connector, do you want to give a try?
I could but no promises on when it’ll be done. My day job is marketing so I code on the side
but Being a contributor to a great project like Airbyte sounds like a fun challenge! I could likely get started next week.
You can try building using the Connector Builder, in couple of weeks we’re going to make simpler to contribute connectors made in the UI
Yeah, should be a really easy use case for Builder . . . probably a great first connector.
(IIRC there’s something funny with Reddit’s oAuth implementation, but it may not matter in this use case—I’d give it a try and don’t be shy to post back if you run into errors!)