feat: Configure codejail and run safety check at startup #10
Conversation
timmc-edx
force-pushed
the
timmc/safe-startup
branch
from
edd7062 to
6479eca
Compare
timmc-edx
force-pushed
the
timmc/safe-startup
branch
2 times, most recently
from
2123d53 to
e1ae885
Compare
| (responses(math=Exception("Divide by zero")), False), | ||
| ) | ||
| @ddt.unpack | ||
| @patch('codejail_service.startup_check.STARTUP_SAFETY_CHECK_OK', None) |
There was a problem hiding this comment.
I thought this ends up as a parameter on the test? I'm confused about how this works.
There was a problem hiding this comment.
That surprised me too. But if you set the new parameter here, you don't get an additional argument to the decorated function.
If patch() is used as a decorator and new is omitted, the created mock is passed in as an extra argument to the decorated function.
There was a problem hiding this comment.
Which is the new parameter? The None? Maybe I'm just used to patched functions, and that is what is throwing me off.
There was a problem hiding this comment.
Yeah, this sets codejail_service.startup_check.STARTUP_SAFETY_CHECK_OK to None for the duration of the test and then sets it back to the original value afterwards.
There was a problem hiding this comment.
I wonder if I should just use the constant "uninitialized" instead of None to make the code and tests clearer. (This wouldn't affect any of the startup check functions, as anything other than True should make it be considered unhealthy.)
- Initialize codejail at startup - Run safety checks at startup, locking out the API if the checks fail If codejail isn't properly configured, it defaults to running code unsafely. To prevent this from affecting the service, we run a smoke test at startup to check if there's anything just *drastically* wrong. If this check does not pass, two things happen: - The healthcheck endpoint will never return a 200 OK - The code-exec endpoint will refuse with a 500 error Supporting changes: - Define an explicit AppConfig for the api subpackage so that we can hook into the `ready()` mechanism - Wrap `safe_exec` to prevent codejail eagerly setting `UNSAFE=True` at module load time. (Not clear why this doesn't affect edx-platform; maybe something to do with app vs. middleware load order.) Filed openedx/codejail#225 for possibly fixing this. - `safe_exec` wrapper also performs a deepcopy to allow callers to reason about the globals dict more easily. Other changes: - Clean up healthcheck docstring (mostly just trim it down) - Lint cleanup Part of edx/edx-arch-experiments#927
timmc-edx
force-pushed
the
timmc/safe-startup
branch
from
2d9fbc9 to
d659920
Compare
|
@timmc-edx, I won't perform any actual review, but I'm assuming you are ready to merge right? |
|
Yes, ready to merge -- thank you! |
If codejail isn't properly configured, it defaults to running code unsafely. To prevent this from affecting the service, we run a smoke test at startup to check if there's anything just drastically wrong.
If this check does not pass, two things happen:
Supporting changes:
ready()mechanismsafe_execto prevent codejail eagerly settingUNSAFE=Trueat module load time. (Not clear why this doesn't affect edx-platform; maybe something to do with app vs. middleware load order.) Issue for fixing this: Import order affects whether code is sandboxed. codejail#16safe_execwrapper also performs a deepcopy to allow callers to reason about the globals dict more easily.Other changes:
Part of edx/edx-arch-experiments#927
Manual testing performed with changes to the Dockerfile and to devstack (PRs pending), and mostly entailed calling the healthcheck endpoint.
When passing, the startup logs look like this:
When codejail is misconfigured:
Merge checklist:
Check off if complete or not applicable: