[RAPTOR-14353] Run flask server using CLI (gunicorn with gevent support) #1633

s-gavrenkov · 2025-08-20T06:05:39Z

Added support for running Flask from the command line. This is required for launching the Flask app using the Gunicorn server with Gevent.
To be integrated with the following PR:
https://github.com/datarobot/datarobot-custom-templates/pull/419

…eration to help prevent logic diverging (#1615) * Fix Drum Inline Runner and streamline DRUM options generation * Cleanup * Refactor setup to new file * Fix mocks * Add unit tests * Change tracer

Co-authored-by: GenAI Git Bot <[email protected]>

Bumps [keras](https://github.com/keras-team/keras) from 3.10.0 to 3.11.0. - [Release notes](https://github.com/keras-team/keras/releases) - [Commits](keras-team/keras@v3.10.0...v3.11.0) --- updated-dependencies: - dependency-name: keras dependency-version: 3.11.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [RAPTOR-14453] Regen requirements.txt to fix CVE-2025-8747 This regenerates the requirements.txt file from requirements.in, to pull in the latest keras 3.11.2, upgrading from 3.10.0. Upgrading past 3.11.0 fixes CVE-2025-8747. * Reconcile dependencies, updated IDs, tags --------- Co-authored-by: svc-harness-git2 <[email protected]>

… chat` kwargs (#1621) * Update DRUM version * Update CHANGELOG.md

…onment (#1627) * Bump requirements in GenAI Agents environment * Update * Reconcile dependencies, updated IDs, tags --------- Co-authored-by: svc-harness-git2 <[email protected]>

…for credential cache daemon (#1630) * [CFX-3334] Update to latest drgithelper and properly set permissions for credential cache daemon * Reconcile dependencies, updated IDs, tags --------- Co-authored-by: svc-harness-git2 <[email protected]>

* [RAPTOR-13851] pytorch: rebuild requirements to pull in updates This rebuilds requirements.txt to pull in updates for pytorch to resolve CVE-2025-3730. * Reconcile dependencies, updated IDs, tags --------- Co-authored-by: svc-harness-git2 <[email protected]>

Co-authored-by: “Nickolai <“[email protected]”>

* addd timeouts * fiz black * fix tests * fixed black * add timeout * fixed * replaced with RuntimeParameters * removed unused import * replaced with static methods

* add watchdog * 5 attempts * watchdog_additional * fixed version * add env USE_NIM_WATCHDOG * add env USE_NIM_WATCHDOG * fixed lint * fixed version * rever docker changes * add changes * removed changelogs * max_attempts = 3 * replaced with os.kill(pid, signal.SIGTERM) * renamed * fixed comments * add NIM_WATCHDOG_REQUEST_TIMEOUT and NIM_WATCHDOG_MAX_ATTEMPTS * add logs * fix lint

# Conflicts: # custom_model_runner/datarobot_drum/drum/root_predictors/prediction_server.py # custom_model_runner/datarobot_drum/drum/runtime.py # public_dropin_environments/python311_genai_agents/env_info.json

…renkov/POC-gunicorn

engprod-2 · 2025-09-01T07:08:03Z

The Needs Review labels were added based on the following file changes.

Team @datarobot/core-modeling (#predictive-ai) was assigned because of changes in files:

custom_model_runner/datarobot_drum/drum/drum.py
custom_model_runner/datarobot_drum/drum/main.py
custom_model_runner/datarobot_drum/drum/root_predictors/prediction_server.py
custom_model_runner/datarobot_drum/drum/runtime.py
custom_model_runner/datarobot_drum/drum/server.py

Team @datarobot/genai-systems (#genai-systems) was assigned because of changes in files:

custom_model_runner/datarobot_drum/drum/drum.py
custom_model_runner/datarobot_drum/drum/main.py
custom_model_runner/datarobot_drum/drum/root_predictors/prediction_server.py
custom_model_runner/datarobot_drum/drum/runtime.py
custom_model_runner/datarobot_drum/drum/server.py

If you think that there are some issues with ownership, please discuss with C&A domain at #sdtk slack channel and create PR to update DRCODEOWNERS\CODEOWNERS file.

peterzdeb · 2025-09-01T13:41:28Z

custom_model_runner/datarobot_drum/drum/main.py

+    signal interruptions, and starts the CMRunner for executing user-defined models.
+
+    Args:
+        flask_app: Optional[Flask] Flask application instance, used when running using command line.


..., used when running using command line.

It's always a command line, when running locally or when running DRUM in as a sidecar container.
Is the plan to setup Flask App when running DRUM as a sidecar?

the old logic remains. It was made to run drum using command gunicorn -c gunicorn.conf.py app:app

if [ "$SERVER_TYPE" == "gunicorn" ]; then echo "Starting gunicorn server..." exec gunicorn -c gunicorn.conf.py app:app else echo "Starting werkzeug (dev Flask) server..." exec drum server --sidecar --gpu-predictor=nim --logging-level=info "$@" fi

peterzdeb · 2025-09-01T13:43:34Z

custom_model_runner/datarobot_drum/drum/root_predictors/prediction_server.py

+                # when running application via the command line (e.g., gunicorn worker)
+                pass
+            else:
+                if RuntimeParameters.has("USE_NIM_WATCHDOG") and str(


this check for has(<attribute>) is redundant as you can check for RuntimeParameters.get("USE_NIM_WATCHDOG", "no") with default False value.

ah, I see it was defined in the original version so not critical for applying in current PR, just a suggestion

user should explicitly set this value.

peterzdeb · 2025-09-01T13:45:21Z

custom_model_runner/datarobot_drum/drum/main.py

+    signal interruptions, and starts the CMRunner for executing user-defined models.
+
+    Args:
+        flask_app: Optional[Flask] Flask application instance, used when running using command line.


flast_app is not optional for DrumRuntime so it can't be optional for main()

if flast_app is None it is created app = create_flask_app() in def get_flask_app(api_blueprint):

https://github.com/datarobot/datarobot-user-models/pull/1633/files#diff-1d592639344a14e290f1bd489098ef8322773e22f75ab54e7cd5280177bbfd43R34-R39

amarmudrankit · 2025-09-03T17:40:46Z

custom_model_runner/datarobot_drum/drum/drum.py

+                if self.worker_ctx:
+                    # Add cleanup when running via the command line (gunicorn worker)
+                    self.worker_ctx.defer_cleanup(
+                        lambda: stats_collector.disable(), desc="stats_collector.disable()"


Probably lets add short but genuine descriptions, than simply the method name?

amarmudrankit

LGTM you can address this before merge #1633 (review)

mbromanowsky

LGTM for Predictive AI

s-gavrenkov and others added 30 commits August 13, 2025 18:40

working

9ba4e58

fixed

209c4f9

removed rr

1251e1a

Reconcile dependencies, updated IDs, tags

7a2b0bc

working

e75d3e4

working

52541cd

Reconcile dependencies, updated IDs, tags

039611f

[BUZZOK-27100] Fix Drum Inline Runner and streamline DRUM options gen…

7e45b62

…eration to help prevent logic diverging (#1615) * Fix Drum Inline Runner and streamline DRUM options generation * Cleanup * Refactor setup to new file * Fix mocks * Add unit tests * Change tracer

[-] (Auto) Bump env_info versions (#1622)

ae5245c

Co-authored-by: GenAI Git Bot <[email protected]>

Configure OTel metrics by default in drum. (#1620)

cfa570e

[BUZZOK-27241] Update DRUM version to 1.16.22 to add support for `def…

db6ce4c

… chat` kwargs (#1621) * Update DRUM version * Update CHANGELOG.md

[BUZZOK-27241] [BUZZOK-27421] Bump requirements in GenAI Agents envir…

f8adbb2

…onment (#1627) * Bump requirements in GenAI Agents environment * Update * Reconcile dependencies, updated IDs, tags --------- Co-authored-by: svc-harness-git2 <[email protected]>

Add OTEL logging configuration, refactor traces and metrics. (#1626)

7b847f6

Bump DRUM version. (#1631)

df74c73

Avoid infinite recursion in logs. (#1638)

bd34a1e

Update version for new release. (#1639)

f17efc8

Co-authored-by: “Nickolai <“[email protected]”>

[RAPTOR-14353] add client and NIM timeouts (#1640)

f5e64fd

* addd timeouts * fiz black * fix tests * fixed black * add timeout * fixed * replaced with RuntimeParameters * removed unused import * replaced with static methods

updated version (#1643)

eef913e

Reconcile dependencies, updated IDs, tags

b0bd728

working

55c8763

Merge remote-tracking branch 'origin/master' into gavrenkov/POC-gunicorn

0971bc0

# Conflicts: # custom_model_runner/datarobot_drum/drum/root_predictors/prediction_server.py # custom_model_runner/datarobot_drum/drum/runtime.py # public_dropin_environments/python311_genai_agents/env_info.json

Reconcile dependencies, updated IDs, tags

a690532

refactoring

170a04c

Merge remote-tracking branch 'origin/gavrenkov/POC-gunicorn' into gav…

f180994

…renkov/POC-gunicorn

revert nim sidecar

3b1967c

s-gavrenkov added 3 commits September 1, 2025 00:52

delint

4ef3541

revert

77f6ad2

added comments

81eeb91

s-gavrenkov marked this pull request as ready for review September 1, 2025 07:07

s-gavrenkov added the 00 - Ready for Review label Sep 1, 2025

engprod-2 bot added Needs Review: Core Modeling Needs Review: GenAI Systems labels Sep 1, 2025

s-gavrenkov changed the title ~~Gavrenkov/poc gunicorn~~ [RAPTOR-14353] Run flask server using CLI (gunicorn with gevent) Sep 1, 2025

s-gavrenkov changed the title ~~[RAPTOR-14353] Run flask server using CLI (gunicorn with gevent)~~ [RAPTOR-14353] Run flask server using CLI (gunicorn with gevent support) Sep 1, 2025

svc-engprod-git1 requested a review from peterzdeb September 1, 2025 09:16

engprod-2 bot removed the Needs Review: GenAI Systems label Sep 1, 2025

peterzdeb reviewed Sep 1, 2025

View reviewed changes

s-gavrenkov requested review from alexromanyuk, klichukb and yakov-g September 3, 2025 14:29

amarmudrankit reviewed Sep 3, 2025

View reviewed changes

amarmudrankit approved these changes Sep 3, 2025

View reviewed changes

svc-engprod-git1 requested a review from mbromanowsky September 3, 2025 19:03

devexp-slackbot bot removed the Needs Review: Core Modeling label Sep 3, 2025

mbromanowsky approved these changes Sep 3, 2025

View reviewed changes

s-gavrenkov added 4 commits September 3, 2025 16:39

added is_client_request_timeout_enabled

b7c58fa

added comments

8d1c1ae

added comments

c2b1ef7

fix signal termination

b976d6b

s-gavrenkov merged commit 62d1c92 into master Sep 4, 2025
49 checks passed

svc-engprod-git1 removed the 00 - Ready for Review label Sep 4, 2025

svc-engprod-git1 deleted the gavrenkov/POC-gunicorn branch September 4, 2025 04:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[RAPTOR-14353] Run flask server using CLI (gunicorn with gevent support) #1633

[RAPTOR-14353] Run flask server using CLI (gunicorn with gevent support) #1633

Uh oh!

s-gavrenkov commented Aug 20, 2025 •

edited

Loading

Uh oh!

engprod-2 bot commented Sep 1, 2025

Uh oh!

peterzdeb Sep 1, 2025

Uh oh!

s-gavrenkov Sep 1, 2025 •

edited

Loading

Uh oh!

peterzdeb Sep 1, 2025

Uh oh!

peterzdeb Sep 1, 2025

Uh oh!

s-gavrenkov Sep 1, 2025

Uh oh!

peterzdeb Sep 1, 2025

Uh oh!

s-gavrenkov Sep 1, 2025

Uh oh!

amarmudrankit Sep 3, 2025

Uh oh!

s-gavrenkov Sep 3, 2025

Uh oh!

amarmudrankit left a comment

Uh oh!

mbromanowsky left a comment

Uh oh!

Uh oh!

Uh oh!

[RAPTOR-14353] Run flask server using CLI (gunicorn with gevent support) #1633

[RAPTOR-14353] Run flask server using CLI (gunicorn with gevent support) #1633

Uh oh!

Conversation

s-gavrenkov commented Aug 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

engprod-2 bot commented Sep 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

s-gavrenkov Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amarmudrankit left a comment

Choose a reason for hiding this comment

Uh oh!

mbromanowsky left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

s-gavrenkov commented Aug 20, 2025 •

edited

Loading

s-gavrenkov Sep 1, 2025 •

edited

Loading