mimas: block alibaba and tencent from scraping hydra #811
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mweinelt
changed the title
mweinelt
force-pushed
the
nft-asblock
branch
5 times, most recently
from
ee196d0 to
f8c0912
Compare
jfly
reviewed
Aug 26, 2025
There was a problem hiding this comment.
Took a look through this and your new Python repo. Overall LGTM.
Some comments:
- I don't understand how old stuff get cleaned up when it should no longer exist. I see
nft add elementbeing called here. I honestly don't know exactly what it does, but it sounds additive. - Sure would appreciate a comment here: https://github.com/mweinelt/nft-prefix-import/blob/6ef9ed3cc2bfd0d2498c03907b0ad87a8366d793/src/nft_prefix_import/__init__.py#L79.
- This looks kind of silly with the
check=Trueright above it: https://github.com/mweinelt/nft-prefix-import/blob/6ef9ed3cc2bfd0d2498c03907b0ad87a8366d793/src/nft_prefix_import/__init__.py#L62. If the intent is to be explicit, I think acheck=Falsewould do that. Either way, could use a comment explaining why we're ok with failures. Is our intent to just accomplish as much as we can? If so, should we be tracking that a failure occurred and then go on to crash the process at the end, so our systemd unit alerting can tell us to take a look?
|
|
I dropped the timeout completely, since adding the same item again does not seem to reset the timeout. The list now resets at boot and will only be appended to for the uptime of the box.
It prints the error to stdout/stderr and moves on. Not silly at all. This is the effect we see in 90 minutes: |
That's not because you're setting demo.py#!/usr/bin/env python
import subprocess
noisy_error = (
"echo 'some info' >/dev/stdout && echo 'this is an error' >/dev/stderr && exit 42"
)
print("=== Experiment 1: Running `noisy_error` with `check=False`")
subprocess.run(noisy_error, shell=True, check=False)
print("=" * 30)
print(
"=== Experiment 2: Running `noisy_error` with `check=True`, and catching and tossing the Python exception"
)
try:
subprocess.run(noisy_error, shell=True, check=True)
except subprocess.SubprocessError:
pass
print("=" * 30)Note how both Experiment 1 and 2 output the exact same thing: $ python demo.py
=== Experiment 1: Running `noisy_error` with `check=False`
some info
this is an error
==============================
=== Experiment 2: Running `noisy_error` with `check=True`, and catching and tossing the Python exception
some info
this is an error
==============================Not that there's anything wrong with the code you've written, it just looks odd to me. My other questions are more important. |
These networks keep scraping hydra.nixos.org with a high request rate across a wide range of IP addresses with bogus user-agents. They did it, they made me develop a tool to lookup prefixes for an AS and block them from accessing tcp/443 using an nftable set match.
mweinelt
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Testing right now on mimas. Will probably pull the python package out into a dedicated repo.