Is there a recommended delimiter for an xml variable containing a list? Should there be?

[billsacks]

I'm introducing a new xml variable in CMEPS, WATER_TRACERS, that will contain a user-settable list of water tracers to include in the simulation. I was looking around for what other xml variables use for a delimiter when they contain a list. It seems like the two dominant choices are comma (e.g., COMP_CLASSES, CASE_SUPPORT_LIBRARIES) and colon (e.g., CDEPS's DOCN_IMPORT_FIELDS).

I wanted to use comma, based on its intuitiveness and also because that's what case.get_values assumes. However, it turns out that using a comma as separator doesn't work with xmlchange, since xmlchange interprets a comma as the start of a new key-value pair:

$ ./xmlchange WATER_TRACERS=MyRegularTracer,HDO
ERROR: Expecting a key value pair in the form of key=value. Got ['HDO']

Is there a recommended delimiter for an xml variable containing a list? Should there be?

I tentatively propose either choosing comma (and somehow modifying xmlchange to allow that), or colon (and possibly changing existing lists to use colon for consistency, at least long-term). Thoughts?

[jedwards4b]

You just need to quote it: ./xmlchange WATER_TRACERS="MyRegularTracer,HDO"

[jedwards4b]

Well I grant you - that's a bit confusing. I think I've figured out how to do this a number of times, but everytime I do it I need to figure out how again.

[billsacks]

I also started down the path of looking at what it would take to modify xmlchange to respect quotes, but even in the best case (where quotes are preserved by the shell) I think this would add some error-prone complexity, and when I did some additional testing I realized that the quotes are being stripped by the shell, so inside the python ./xmlchange WATER_TRACERS="MyRegularTracer,HDO" gives an argument that no longer contains quotes. You can escape the quotes, or do something like ./xmlchange WATER_TRACERS="'MyRegularTracer,HDO'", but this again gets messy from a user perspective.

I feel like the ideal solution, if we were comfortable breaking users' scripts/workflows, would be to change the default delimiter in xmlchange to something like ;. But I suspect that will break a LOT.

So my inclination for now is just to use : for my new variable. I'm happy to hear any additional thoughts / dissenting views on that, though.

[billsacks]

(And thank you, @jedwards4b - I very much appreciate your iterating with me on this!)

[billsacks]

Also: at least in this case, : seems like an intuitive delimiter because it matches the delimiter that I plan to use for the corresponding variables in nuopc.runconfig: : seems like a typical delimiter in that context.

[jedwards4b]

I think that I chose comma for many of this is because that lends itself naturally to treating the variable as a python list.

[bartgol]

Would it be reasonable to change xmlchange/xmlquery to use spaces instead of commas to separate requests? I assume there may be a reason why that is not already the case thought...

[billsacks]

Oh, I just saw that there is documentation of how to xmlchange a variable where the value has commas (added by @jgfouca in #4657):

   To set a variable in which the value has commas, you'll need to use
   the alternative syntax:
      ./xmlchange --id VARNAME --val "one,two"

@bartgol - I like the idea of using spaces to delimit requests; the issue I see is that this will probably break a lot of people's scripts / workflows. I believe that the ability to set multiple key-value pairs separated by commas goes back a very long time in xmlchange.

@mvertens chimed in over at ESCOMP/CMEPS#601 (comment) to strongly support the use of colon rather than comma as a delimiter in the case I'm working on now. So I'm inclined to go that direction in this case, but I'll still keep this open to invite any further thoughts because I don't love that we're getting different delimiters in different situations. (And I'm happy to revisit my choice for water tracers... it won't become a fully operational feature across CESM for at least a year, so we should have time to revisit this.)