feat: add backwards compatible history to v3/datasets response

[omkar-ethz]

Description

Add history array to v3/datasets response, with the same format as backend-v3. The main difference between GenericHistory and HistoryClass format is that in the former, changed fields are grouped into before and after. Whereas in the latter, each field is present at the top level with field: {previousValue: ..., currentValue: ...}.

Another difference is with Delta strategy, the current history implementation only records fields that are acutally changed. Whereas in old backend, for Dataset.datasetlifecycle, all fields from previous value of datasetlifecyle were recorded in previousValue.

Motivation

Archive/retrieve workflows at PSI depend on backend-v3 representation of History

Fixes

• Include history in /api/v3/datasets #2248
• Update return type in dataset.service from DatasetClass to DatasetDocument

Changes:

• Created adapter from GenericHistory to HistoryClass
• Created adapter from (GenericHistory[], Dataset) -> HistoryClass[] that preserves older datasetlifecycle in previousValue

Tests included

• Included for each change/fix?
• Passing?

Documentation

• swagger documentation updated (required for API changes) - auto updated OutputDatasetObsoleteDto to include the history[] array.
• official documentation updated

Sample output

GET /datasets/{pid}

{
...
  "principalInvestigator": "[email protected]",
  "history": [
    {
      "updatedAt": "2025-10-14T08:05:10.202Z",
      "updatedBy": "ingestor",
      "_id": "",
      "keywords": {
        "previousValue": [
          "new",
          "keyword"
        ],
        "currentValue": [
          "new",
          "keyword",
          "trois"
        ]
      },
      "description": {
        "previousValue": "This is a test description just for change",
        "currentValue": "one last test description"
      }
    },
    {
      "updatedAt": "2025-10-14T07:37:04.757Z",
      "updatedBy": "ingestor",
      "_id": "",
      "isPublished": {
        "previousValue": true,
        "currentValue": false
      }
    },
    {
      "updatedAt": "2025-10-14T07:23:28.580Z",
      "updatedBy": "ingestor",
      "_id": "",
      "keywords": {
        "previousValue": [],
        "currentValue": [
          "new",
          "keyword"
        ]
      },
      "description": {
        "previousValue": "This is a test description what noooo",
        "currentValue": "This is a test description just for change"
      }
    },
    {
      "updatedAt": "2025-10-14T07:15:34.684Z",
      "updatedBy": "ingestor",
      "_id": "",
      "description": {
        "previousValue": "This is a test description back on v3-history",
        "currentValue": "This is a test description what noooo"
      }
    },
    {
      "updatedAt": "2025-10-14T07:15:24.809Z",
      "updatedBy": "ingestor",
      "_id": ""
    },
    {
      "updatedAt": "2025-10-14T07:00:24.262Z",
      "updatedBy": "ingestor",
      "_id": "",
      "keywords": {
        "previousValue": [
          "newtag"
        ],
        "currentValue": []
      },
      "description": {
        "previousValue": "This is a test description hello HEAD^2",
        "currentValue": "This is a test description back on v3-history"
      }
    },
    {
      "updatedAt": "2025-10-14T06:57:00.237Z",
      "updatedBy": "ingestor",
      "_id": "",
      "keywords": {
        "previousValue": [],
        "currentValue": [
          "newtag"
        ]
      }
    },
    {
      "updatedAt": "2025-10-14T06:56:44.914Z",
      "updatedBy": "ingestor",
      "_id": "",
      "description": {
        "previousValue": "This is a test description hello",
        "currentValue": "This is a test description hello HEAD^2"
      }
    },
    {
      "updatedAt": "2025-10-13T13:59:47.634Z",
      "updatedBy": "ingestor",
      "_id": ""
    },
    {
      "updatedAt": "2025-10-13T13:59:17.508Z",
      "updatedBy": "ingestor",
      "_id": "",
      "description": {
        "previousValue": "This is a test description hola",
        "currentValue": "This is a test description hello"
      }
    },
    {
      "updatedAt": "2025-10-13T13:57:50.173Z",
      "updatedBy": "ingestor",
      "_id": "",
      "description": {
        "previousValue": "This is a test description hello",
        "currentValue": "This is a test description hola"
      }
    }
  ]
}

[minottic]

thanks! minor comments, LGTM otherwise.

I think a class transformer would have been better suited by I see that we are already using these convertors throughout the code

[minottic]

is this used anywhere?

[omkar-ethz]

This will be used in the up migration script: afbb56d, creating a PR soon

[omkar-ethz]

Migration script PR: #2286

[minottic]

could you please move it in the migration PR then? I am not sure I agree too much with the migration one, but we could merge this one in the meantime if this was part of the other PR

[minottic]

could you please move it in the migration PR then? I am not sure I agree too much with the migration one, but we could merge this one in the meantime if this was part of the other PR

[omkar-ethz]

@minottic sure, moved the obsolete-to-generic function in migration PR #2286

[minottic]

LGTM, thanks!