Resource requirements are not applied when a reset is triggered, but are applied when a regular sync runs

Platform, Deploy & Infra Issues
,
1
  • Is this your first time deploying Airbyte?: No
  • OS Version / Instance: Kubernetes AWS AMI
  • Memory / Disk: you can use something like 4Gb / 1 Tb
  • Deployment: Are you using Docker or Kubernetes deployment?
  • Airbyte Version: 0.39.2-alpha
  • Source name/version:
  • Destination name/version: airbyte/destination-redshift:0.3.38
  • Step: reset
  • Description: In our audit logs the main container does not have resources specified, which causes a failure since our cluster has an OPA policy around resource requirements.

Message: admission webhook “validating-webhook.openpolicyagent.org” denied the request: Missing resource limit or request. Must have defined cpu and memory limits + requests for all containers!. Received status: Status(apiVersion=v1, code=400, details=null, kind=Status, message=admission webhook “validating-webhook.openpolicyagent.org” denied the request: Missing resource limit or request. Must have defined cpu and memory limits + requests for all containers!, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Missing resource limit or request. Must have defined cpu and memory limits + requests for all containers!, status=Failure, additionalProperties={}).

2

resolved this by compiling a version of airbyte where resource requirements were consolidated. I will make a PR for this when i get a chance and am sure my solution makes sense.

3

Hey @gfeldman,
Thank you for spotting this and considering contributing. Do you mind opening an issue on our repo, sharing the link here and linking your future PR to this issue? I’m asking these to make our engineering team aware of the issue and make sure your work does not overlap with theirs.

4

Hi there from the Community Assistance team.
We’re letting you know about an issue we discovered with the back-end process we use to handle topics and responses on the forum. If you experienced a situation where you posted the last message in a topic that did not receive any further replies, please open a new topic to continue the discussion. In addition, if you’re having a problem and find a closed topic on the subject, go ahead and open a new topic on it and we’ll follow up with you. We apologize for the inconvenience, and appreciate your willingness to work with us to provide a supportive community.