Add unified job status endpoint GET /api/v3_0/jobs/<uuid>

[Copilot]

Single endpoint to poll the status of any background job (scheduling, forecasting, etc.) by UUID — without needing to know which queue or resource the job belongs to.

Changes

• flexmeasures/api/v3_0/jobs.py — new JobAPI view with GET /<uuid>:

  • Requires authentication (@auth_required()); no resource-level permission check needed since the UUID is only known to the submitter
  • Fetches the job from the shared Redis connection (covers all queues)
  • Returns a rich response on success; HTTP 400 if UUID not found
  • Datetime fields (enqueued_at, started_at, ended_at) are serialised as ISO-8601 strings or null

• flexmeasures/api/v3_0/__init__.py — registers JobAPI at /api/v3_0

• flexmeasures/api/v3_0/tests/test_jobs_api.py — covers: unknown UUID → 400, queued → QUEUED (with metadata assertions), finished → FINISHED (with timing assertions), unauthenticated → 401

• documentation/api/change_log.rst / documentation/changelog.rst — changelog entries

Example response

GET /api/v3_0/jobs/b3d26a8a-7a43-4a9f-93e1-fc2a869ea97b

200 OK
{
  "status": "FINISHED",
  "message": "Scheduling job has finished.",
  "result": null,
  "func_name": "flexmeasures.data.services.scheduling.create_schedule",
  "origin": "scheduling",
  "enqueued_at": "2026-04-28T10:00:00+00:00",
  "started_at": "2026-04-28T10:00:01+00:00",
  "ended_at": "2026-04-28T10:00:05+00:00"
}

GET /api/v3_0/jobs/b3d26a8a-7a43-4a9f-93e1-fc2a869ea97b

200 (while still queued)
{
  "status": "QUEUED",
  "message": "Scheduling job waiting to be processed.",
  "result": null,
  "func_name": "flexmeasures.data.services.scheduling.create_schedule",
  "origin": "scheduling",
  "enqueued_at": "2026-04-28T10:00:00+00:00",
  "started_at": null,
  "ended_at": null
}

[read-the-docs-community]

Documentation build overview

📚 flexmeasures | 🛠️ Build #32467670 | 📁 Comparing 41a9d26 against latest (3f95e86)

  🔍 Preview build  

8 files changed · + 1 added · ± 7 modified

+ Added

• _autosummary/flexmeasures.api.v3_0.jobs.html

± Modified

• changelog.html
• genindex.html
• http-routingtable.html
• py-modindex.html
• _autosummary/flexmeasures.api.v3_0.html
• api/change_log.html
• api/v3_0.html

[BelhsanHmida]

@copilot please update the JSON response returned by the jobs endpoint.

Add the missing result field for job.result.

Also include these documented RQ job attributes as top-level response fields:

• func_name
• origin
• enqueued_at
• started_at
• ended_at

So the response should look roughly like:

{
  "status": "FINISHED",
  "message": "Job has finished.",
  "result": ...,
  "func_name": "...",
  "origin": "...",
  "enqueued_at": "...",
  "started_at": "...",
  "ended_at": "..."
}

Add/update tests to check these fields are present for queued/started/finished jobs where applicable.

Make sure datetime fields are returned in a JSON-safe way, e.g. serialized as strings or null when unavailable.

[Copilot]

@copilot please update the JSON response returned by the jobs endpoint.

Add the missing result field for job.result.
...

Done in commit 454b0fc. The response now includes all the requested fields:

{
  "status": "FINISHED",
  "message": "Scheduling job has finished.",
  "result": null,
  "func_name": "flexmeasures.data.services.scheduling.create_schedule",
  "origin": "scheduling",
  "enqueued_at": "2026-04-28T10:00:00+00:00",
  "started_at": "2026-04-28T10:00:01+00:00",
  "ended_at": "2026-04-28T10:00:05+00:00"
}

Datetime fields are serialised as ISO-8601 strings or null when not yet set. The result field uses job.return_value() (the non-deprecated RQ API). Tests for both the queued and finished cases now assert the presence and expected values of all new fields.

[BelhsanHmida]

@copilot The tests still feel a bit incomplete for the response contract we discussed. Can you add coverage for a STARTED job as well, and also explicitly assert the result field in the finished-job test so queued/started/finished are all properly covered with the expected metadata fields?

[BelhsanHmida]

@nhoening right now GET /api/v3_0/jobs/<uuid> seems to let any authenticated user fetch any job as long as they know the UUID. I don’t think “only the submitter knows the UUID” is a strong enough permission boundary. Do you agree we should enforce a proper ownership/access check here, or otherwise tie the lookup to a resource or organisation the requester is actually allowed to access?

[Copilot]

@copilot The tests still feel a bit incomplete for the response contract we discussed. Can you add coverage for a STARTED job as well, and also explicitly assert the result field in the finished-...

Done in commit ab0d6f3.

• Added test_get_job_status_started: enqueues a job then calls job.set_status(JobStatus.STARTED) directly on Redis to simulate a mid-execution job. Asserts status == "STARTED", "in progress" in the message, enqueued_at is set, ended_at and result are None.
• Updated test_get_job_status_finished to explicitly assert data["result"] is not None — the scheduling function returns True on success, which now shows up in the response.

All 5 tests pass.

[nhoening]

@nhoening right now GET /api/v3_0/jobs/<uuid> seems to let any authenticated user fetch any job as long as they know the UUID. I don’t think “only the submitter knows the UUID” is a strong enough permission boundary. Do you agree we should enforce a proper ownership/access check here, or otherwise tie the lookup to a resource or organisation the requester is actually allowed to access?

Good point.
We might give hints in the meta data about which sensor or asset is involved.
And on that we could check_access("read").

I'm not sure all jobs have access to this in a similar way, though. Can you check for scheduling and forecasting jobs?

[BelhsanHmida]

@nhoening right now GET /api/v3_0/jobs/<uuid> seems to let any authenticated user fetch any job as long as they know the UUID. I don’t think “only the submitter knows the UUID” is a strong enough permission boundary. Do you agree we should enforce a proper ownership/access check here, or otherwise tie the lookup to a resource or organisation the requester is actually allowed to access?

Good point. We might give hints in the meta data about which sensor or asset is involved. And on that we could check_access("read").

I'm not sure all jobs have access to this in a similar way, though. Can you check for scheduling and forecasting jobs?

I checked most of the jobs we create locally.

For forecasting jobs, yes: we already have enough context. The jobs carry sensor_id in their kwargs and also in job.meta["forecast_kwargs"], so we can resolve the sensor and do check_access(sensor, "read").

For scheduling jobs, mostly yes as well: create_scheduling_job already stores job.meta["asset_or_sensor"] with the related asset or sensor id/class, so normal scheduling jobs can be checked the same way.

One edge case I found is the wrap-up job from sequential scheduling: that one is cached against the asset, but it does not seem to store the related asset/sensor ref in job.meta itself. So if we want this to work uniformly for all scheduling jobs, we’d probably need to add that metadata there too.

[nhoening]

So if we want this to work uniformly for all scheduling jobs, we’d probably need to add that metadata there too.

Yes, let's do that as a principle from now on. Jobs need to provide a means to show what they are about.

[nhoening]

Scope update:

I am currently onboarding a partner and a big value-add would be if the returned information had exception information if something went wrong.

If job.is_failed, there might be valuable information in job.exc_info.

It would be great to test this:

• add a assert 1 == 2 statement into code to see the result.
• ask for an infeasible schedule (soc-target impossible to reach) and check the response of the endpoint talking about that.

Maybe the latter can be turned into an automated test, ask copilot about that.

[BelhsanHmida]

Scope update:

I am currently onboarding a partner and a big value-add would be if the returned information had exception information if something went wrong.

If job.is_failed, there might be valuable information in job.exc_info.

It would be great to test this:

• add a assert 1 == 2 statement into code to see the result.
• ask for an infeasible schedule (soc-target impossible to reach) and check the response of the endpoint talking about that.

Maybe the latter can be turned into an automated test, ask copilot about that.

Implemented this now.

Failed jobs on GET /api/v3_0/jobs/<uuid> now include traceback information via exc_info, so the response is more helpful when something goes wrong.

I also checked the two cases you suggested:

• a forced failure (assert 1 == 2)
• an infeasible schedule with impossible soc-targets

In both cases the endpoint now returns the failed status, the human-readable message, and the traceback context.

• Successful job:

{
  "ended_at": "2026-04-30T14:09:48.281063+00:00",
  "enqueued_at": "2026-04-30T14:09:46.032891+00:00",
  "exc_info": null,
  "func_name": "flexmeasures.data.services.scheduling.make_schedule",
  "message": "Scheduling job has finished.",
  "origin": "scheduling",
  "result": true,
  "started_at": "2026-04-30T14:09:46.047736+00:00",
  "status": "FINISHED"
}

• Forced failure (assert 1 == 2):

{
  "ended_at": "2026-04-30T14:09:48.354535+00:00",
  "enqueued_at": "2026-04-30T14:09:48.327122+00:00",
  "exc_info": "Traceback (most recent call last): ... AssertionError: assert 1 == 2\n",
  "func_name": "flexmeasures.data.services.scheduling.make_schedule",
  "message": "Scheduling job failed with AssertionError: assert 1 == 2.",
  "origin": "scheduling",
  "result": null,
  "started_at": "2026-04-30T14:09:48.336711+00:00",
  "status": "FAILED"
}

• Infeasible schedule:

{
  "ended_at": "2026-04-30T14:09:48.638125+00:00",
  "enqueued_at": "2026-04-30T14:09:48.408118+00:00",
  "exc_info": "Traceback (most recent call last): ... ValueError: The input data yields an infeasible problem. Constraint validation has found the following issues: ...\n",
  "func_name": "flexmeasures.data.services.scheduling.make_schedule",
  "message": "Scheduling job failed with ValueError: The input data yields an infeasible problem. Constraint validation has found the following issues: ...",
  "origin": "scheduling",
  "result": null,
  "started_at": "2026-04-30T14:09:48.419319+00:00",
  "status": "FAILED"
}

I also turned the failure scenarios into automated coverage in the jobs endpoint tests, including the infeasible schedule case.

[nhoening]

Great - on the infeasible example, is the actual message about contraint validation truncated?

And what happens if a mis-configuration happens, e.g. "soc-min": "abc" ?

[BelhsanHmida]

Great - on the infeasible example, is the actual message about contraint validation truncated?

And what happens if a mis-configuration happens, e.g. "soc-min": "abc" ?

• The infeasible-schedule case, the actual GET /api/v3_0/jobs/<uuid> response is not truncated. I had only shortened it in my earlier comment for example. In the real response, the message includes the full constraint-validation details, and the same details also show up in exc_info.

• For a misconfiguration like "soc-min": "abc", it does not become a failed background job. It is rejected immediately by POST /api/v3_0/sensors/<id>/schedules/trigger, so no job UUID is created and there is no /jobs/<uuid> response afterwards.

The actual response I got was:

{
  "message": {
    "json": {
      "soc-min": [
        "Cannot convert value 'abc' to a valid quantity. 'abc' is not defined in the unit registry"
      ]
    }
  },
  "result": "Rejected",
  "status": "UNPROCESSABLE_ENTITY"
}

[nhoening]

It works.

Two requests about error states, and although the job's meta information now contains asset/sensor ID, no use of check_access(asset, "read") or check_access(sensor, "read") is in this PR. For that, I'd also request an extra test (to check that a user who is allowed to read information gets it, but a user who doesn't have the permission cannot.

[BelhsanHmida]

It works.

Two requests about error states, and although the job's meta information now contains asset/sensor ID, no use of check_access(asset, "read") or check_access(sensor, "read") is in this PR. For that, I'd also request an extra test (to check that a user who is allowed to read information gets it, but a user who doesn't have the permission cannot.

Implemented this now.

The jobs endpoint now resolves the related asset/sensor from the job metadata and applies check_access(..., "read") before returning job information.

I also added coverage for that, so a user who can read the related resource gets the job status, while a user without read access gets a 403.

In addition, I handled the Redis / queue-unavailable case as well: the endpoint now returns a 503 with the generic message Job queues are currently unavailable., and I added coverage for that outage case too.

[nhoening]

Thanks.

Just one question: I ran a forecasting job and got this:

{
  "ended_at": "2026-05-04T15:01:03.146562+00:00",
  "enqueued_at": "2026-05-04T14:59:24.156219+00:00",
  "exc_info": null,
  "func_name": "run_cycle",
  "message": "Forecasting job has finished.",
  "origin": "forecasting",
  "result": 14.210693120956421,
  "started_at": "2026-05-04T15:00:48.893739+00:00",
  "status": "FINISHED"
}

Why is result 14.21 ?

[nhoening]

I assumed you wanted me to review. If I was correct, I ask you to be more explicit with that going forward. Re-request the review.

[BelhsanHmida]

I assumed you wanted me to review. If I was correct, I ask you to be more explicit with that going forward. Re-request the review.

yes i meant for a re review.

[BelhsanHmida]

I assumed you wanted me to review. If I was correct, I ask you to be more explicit with that going forward. Re-request the review.

That result is currently the raw return value of the forecasting job function.

For forecasting cycle jobs, run_cycle(...) returns the total runtime in seconds (train_runtime + predict_runtime), and the jobs endpoint exposes that via job.return_value(). So in this case 14.21 means the forecasting cycle took about 14.21 seconds to run, not that it produced a forecast value of 14.21.

I agree this is not very explicit in the unified jobs endpoint, especially since the actual forecast values are available through the forecasting endpoint. I think the better follow-up would be to make that more explicit there, for example by returning structured result data like {"runtime_seconds": 14.21} or by moving runtime to a dedicated metadata field instead of exposing it as a bare numeric result.

I’ll open an issue for this.

[BelhsanHmida]

If this pr is approved @nhoening please merge, i can't.