-
Notifications
You must be signed in to change notification settings - Fork 304
feat(webhook): add rate limiting to webhook endpoint #1210
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Signed-off-by: Christopher Coco <[email protected]>
5046ebe
to
b46b8e9
Compare
…fests to include them Signed-off-by: Christopher Coco <[email protected]> fix manifests Signed-off-by: Christopher Coco <[email protected]>
b46b8e9
to
a1fdd60
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1210 +/- ##
==========================================
+ Coverage 63.05% 63.62% +0.56%
==========================================
Files 23 24 +1
Lines 3140 3222 +82
==========================================
+ Hits 1980 2050 +70
- Misses 1050 1062 +12
Partials 110 110 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Signed-off-by: Christopher Coco <[email protected]>
…ment Signed-off-by: Christopher Coco <[email protected]>
Signed-off-by: Christopher Coco <[email protected]>
Signed-off-by: Christopher Coco <[email protected]>
Resolved the things mentioned in the review. |
docs/configuration/webhook.md
Outdated
@@ -168,6 +191,10 @@ environment variables. Below is the list of which variables correspond to which | |||
|`GHCR_WEBHOOK_SECRET` |`--gchr-webhook-secret`| | |||
|`HARBOR_WEBHOOK_SECRET` |`--harbor-webhook-secret`| | |||
|`QUAY_WEBHOOK_SECRET` |`--quay-webhook-secret`| | |||
|`ENABLE_WEBHOOK_RATELIMIT`|`--enable-webhook-ratelimit`| | |||
|`WEBHOOK_RATELIMIT_ALLOWED_`|`--webhook-ratelimit-allowed`| |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
extra _ at the end of the env variable name.
Signed-off-by: Christopher Coco <[email protected]>
Sorry about that, tried to get that done quick before leaving the office fixed again. |
I was wondering why you would roll up your own rate limiter, especially given the fact that we already use the rate limiter from uber within image updater |
I did not realize this and haven't really touched the registry scanner. I'll refactor this PR tomorrow thanks for the advice I should definitely look more at the packages used next time 🤦♂️ |
This PR aims to add rate limiting to the webhook endpoint so it can not be spammed with requests. This is accomplished with a sliding window approach where timestamps of requests are stored and then when you make a request it is checked how many requests were made in the specified window from the time of your request. If the window was fixed you could burst the requests at the end of the window and then at the start of the window which could cause overloading problems. The sliding window approach makes it so this is not a problem. The downside to this approach however, is that it requires more memory to store each individual timestamp. Because container registries could have rotating IP addresses this can be a problem due to information of an IP that won't make any more requests will still be stored. So to fix this I added a clean up interval that can be set that will clean any clients that have not been seen in a while.
I looked into some packages that were available and found the rate and uber/ratelimit package but after doing research into what strategies are used for rate limiting it seemed like it wasn't the most difficult to implement.
I will add the documentation for this in a separate PR.