Problem migration tool

[sjoqvist]

The code converts parts of problem.yaml between format versions.

It's not complete. Feel free to continue working, or provide feedback on the structure or feasibility of this attempt.

[mpsijm]

Nice! 😄

Just linking RagnarGrootKoerkamp/BAPCtools#432 to mention parallel developments. Both should exist, because BAPCtools has some extensions that Problemtools doesn't, and the migration script in Problemtools should probably follow the spec as closely as possible (as in, without extensions). Probably good to keep an eye on each other's implementations to keep them as consistent as possible 😄

[gkreitz]

Just as a heads-up, we're working on making problemtools.verifyproblem support the new format. I'm very strongly inclined towards an approach where verifyproblem (or, probably, a submodule) will basically:

1. Read problem.yaml and parse it according to the problem version
2. Convert the config to an internal format for verifyproblem (so that beyond this point, code reading the config doesn't need to care about format version).

The internal format would be very close to the 2023-07-draft version, except for some added fields (there are a few things in the legacy format I don't think we can represent in the 2023-07 format, e.g., grading.objective: min). If that route is successful (I'm prototyping now), that would also give us a tool to update problem.yaml from legacy to 2023-07 basically for free. I'm not sure if this PR does more than update problem.yaml, but if it doesn't, I'd recommend pausing this effort for a bit, as I hope to make such updates trivial.

[gkreitz]

I added #300 which implements the design I mentioned above. I also did a very quick prototype of something that updates problem.yaml. I think it's the right approach, but we'll probably need to fight PyYAML a bit, as it looks very wonky in how it dumps stuff by default. That should be fixable by massaging the data (which is just a dict) between model_dump and yaml.dump, if nothing else. There one can also filter out superfluous keys (or, alternatively, not exlude anything and get a maximal example).

For an actual update tool, we should also make it easy to get the problem names from statements (here, I just hard coded).

update.py:

import yaml
import uuid
from problemtools import formatversion, metadata

data = yaml.safe_load(open('examples/hello/problem.yaml'))
name_data = {'en': 'Hello World!', 'sv': 'Hej Världen!'}
m = metadata.parse_metadata(formatversion.get_format_data('examples/hello'), data, name_data)
m.problem_format_version = '2023-07-draft'
m.uuid = uuid.uuid4()
print(yaml.dump(m.model_dump(exclude_defaults=True, exclude_none=True)))

gkreitz@norn:~/problemtools$ venv/bin/python update.py 
allow_file_writing: true
constants: {}
credits: {}
keywords: []
languages: all
license: !!python/object/apply:problemtools.metadata.License
- public domain
limits:
  code: 128
  compilation_memory: 1024
  compilation_time: 60
  memory: 512
  output: 8
  time_multipliers:
    ac_to_time_limit: 5.0
    time_limit_to_tle: 2.0
  validation_memory: 1024
  validation_output: 8
  validation_time: 60
name:
  en: Hello World!
  sv: "Hej V\xE4rlden!"
problem_format_version: 2023-07-draft
source:
- name: Kattis
type:
- pass-fail
uuid: !!python/object:uuid.UUID
  int: 80988162101442689093179833518653111665

The obvious problems in what PyYAML does are:

• Spits out license in a weird way.
• Spits out uuid in a crazy way.
• Encodes swedish characters using ISO8859-1