Storybook-Theming: Evaluation of TAL-Theming

[drfho]

It is intended to create a basic set of TAL files that can be utilized in Storybook for constructing frontend designs.

Ref: https://github.com/idasm-unibe-ch/unibe-cms/issues/971

• Definition of the features these TAL files need for integration into Storybook

[drfho]

Content Schema: Where to define/store ?

Maybe we put the schema of the content object as a separate yml/json file or into a TAL attribute schema ?

Or can we derive it from the html-structure?

[drfho]

@zmsdev :
if the document-schema is defined with data-schema-attributes in the HTML code, it is feasable to create a schema-definition file, I uses JSON for the evaluation like:

ZMS/Products/zms/skins/storybook/schema/infobox2.json

Lines 1 to 10 in cd8e3c3

	{
	"title": {
	"id": "title",
	"type": "string",
	"name": "Title",
	"mandatory": "0",
	"multilang": "1",
	"repetitive": "0",
	"default": ""
	},

[drfho]

ZMS-Schema Extraction from HTML/TAL-Design-Templates

Goal: 'Single source of Truth'

Web-design is generated with/by the content model. So, to allow the web-designer to define/maintain the design-data AND the connected model (which again creates GUI) a logical bridge is needed for transferring the HTML-implicit model into the CMS.

Basic Idea: data-schema-* attributes

The idea is simple: Designer creates an HTML template (in the file system) for each content object-class as before (with storybook, figma etc.). A ZMS-compliant data schema is extracted from this HTML data using a Py shell script. Currently the result is a JSON file.
HTML attrs of the type “data-schema-*” are used to specify the schema in the design HTML.

Alternatives to data-attributes

HTML has the following “standards” for schema definition:

1. JSON-LD
2. microdata
3. RDFa

JSON-LD is the most precise and separates the content model into the script tag; however, this would result in redundant data maintenance.
Micodata and RDFa have their own attribute syntax with property and itemprop respectively, which has a content focus.
There is no further namespace for data attributes, i.e. full flexibility

Declaring the CMS-content-model by data-schema-* attributes

All relevant ZMS model attributes are declared by data-attributes of an HTML-element. The Python script create_schema.py extracts ZMS-like metaobj-model from the HTML/TAL code.

1. data-schema-id
2. data-schema-type
3. data-schema-name
4. data-schema-mandatory
5. data-schema-multilang
6. data-schema-repetitive
7. data-schema-default
8. data-schema-sort_id

If the content-model shall create HTML-attributes there may occur one or many of them for each HTML-element, so the naming convention for the data-attribute declaration has to be more specific:

1. data-schema-attr-$name-id (e.g. data-schema-attr-href-id)
2. data-schema-attr-$name-type
3. data-schema-attr-$name-name
4. data-schema-attr-$name-mandatory
5. data-schema-attr-$name-multilang
6. data-schema-attr-$name-repetitive
7. data-schema-attr-$name-default
8. data-schema-attr-$name-sort_id

Screenshot: Automatic schema creation from HMTL-template

The Py script create_schema.py makes a guess, so that a single attribute “data-schema-id” is often sufficient to declare the CMS data field required for content capture.

• We need to consider whether/how we can make the known model definition file init.py and the init.json automatically extracted from the HTML template compatible. It is clear that we cannot synchronize complex attribute values such as Py script, selects etc. bidirectionally with the HTML templates.

[cmeier76]

@drfho Maybe we should think in terms of a StorybookConnector for ZMS.

Such a Python module should handle the shown extraction of data-schema-* attributes here.

Additionally it should provide these attributes (subset of allowed data types) to the existing ZMSMetaobjManager.

The the well established bi-directional syncing from filesystem into ZMS and vice versa using ZMSRepositoryManager should be untouched.

-> IMHO the existing development and deployment workflows should remain untouched as much as possible

-> Maybe the data-schema-* attributes in the HTML templates should also consist of a identifier like "zms" to make usage explicit - and scraping more easier...

[drfho]

@cmeier76 Thank you very much for the inspiration.

[1] fully agree not touching the well established repo-sync/deploy. The StorybookConnector might be a kind of preproessor, but does not directly interfere with repomanager.
Actually there are only 2, maybe 3 files created from storybook for importing into ZMS: 1. metaobj-standard_html.zpt, 2. init.json/py/yml, 3. css/scss. Usually a cms-schema is more complex, containing more code-elements.
This allows us to rehink generally about the schema-format (py vs. json/yml):
advantage of py is containing more complex data-types. json/yml is more user-friendly, but - for now - it seems not being able to cover the schema complexity well. Maybe further evaluation is needed, to check whether it is possible to create a py-compatible schema file in json or yml !?

[2] Because the data-schema-* vocabulary is invented for ZMS, name-collisions are not expected, but may not be excluded ;-)
So, my conclusion is to set the idenifying key of the data-attribute as a global variable datakey to allow a sematics of choice, like data-zmsschema-, data-unibeschema or data-xyz*

ZMS/Products/zms/skins/storybook/infobox.zpt

Lines 26 to 35 in 995abb5

	<a href="#" class="readmore"
	data-unibeschema-id="linktext"
	data-unibeschema-type="string"
	data-unibeschema-name="Linktext"
	data-unibeschema-default="Read more..."
	data-unibeschema-attr-href-id="url"
	tal:attributes="href python:zmscontext.attr('url') or '#';"
	tal:content="python:zmscontext.attr('linktext') or default">
	Weiter lesen ...
	</a>

[drfho]

Creating YAML-based Schema-Data

@zmsdev: as a prototype I added a function for creating the metaobj-schema as yaml-format

ZMS/Products/zms/skins/storybook/create_schema.py

Lines 121 to 134 in 3761f09

	def save_schema_as_yaml(schema, output_file_path):
	"""
	Save the schema dictionary to a YAML file.
	:param schema: The schema dictionary to be saved.
	:param output_file_path: The path where the YAML file will be saved.
	"""
	yaml_file = os.path.join(output_file_path, 'init.yaml')
	with open(yaml_file, 'w', encoding='utf-8') as output_file:
	class CustomDumper(yaml.Dumper):
	pass
	# Preserve the order of elements in the dictionary when dumping to YAML
	CustomDumper.add_representer(dict, lambda dumper, data: dumper.represent_dict(data.items()))
	yaml.dump(schema, output_file, allow_unicode=True, default_flow_style=False, Dumper=CustomDumper)
	return True

There might be substancial advantage of the YAML-format using the Literal Block Scalar |. The code-block are much better readable/editable compared to py or json:

ZMS/Products/zms/skins/storybook/schema/infobox/init.yaml

Lines 51 to 56 in 3761f09

	# This is a test attribute to demonstrate the use of multiline Python code in attributes.
	# | (Literal Block Scalar) preserves newlines exactly as they appear in the YAML file.
	default: |
	def test():
	return "This is a test attribute that executes Python code."
	return test()

To utilize py (__init__.py) or yaml (__init__.yaml) for the class definition the following function needs to deal with both formats ...

ZMS/Products/zms/repositoryutil.py

Lines 67 to 75 in 60e4a67

	def get_class(py):
	id = re.findall('class (.*?):', py)[0]
	if sys.version_info >= (3, 13):
	py = py + "\nglobal c\nc = " + id
	exec(py, globals=globals(), locals=locals())
	return eval("c", globals=globals(), locals=locals())
	else:
	exec(py)
	return eval(id)

... e.g. like this:

def get_class(py):
  if re.search(r'^\s*class:', py, re.M):
    # YAML-style class definition
    import yaml
    d = yaml.safe_load(py)
    return d['class']
  else:
    id = re.findall('class (.*?):', py)[0]
    if sys.version_info >= (3, 13):
      py = py + "\nglobal c\nc = " + id
      exec(py, globals=globals(), locals=locals())
      return eval("c", globals=globals(), locals=locals())
    else:
      exec(py)
      return eval(id)

The functions readRepository and remoteFiles shall differ the data-type ...

ZMS/Products/zms/repositoryutil.py

Lines 100 to 104 in 60e4a67

	try:
	c = get_class(py)
	d = c.__dict__
	except:
	d['revision'] = standard.writeError(self,"[remoteFiles.traverse]: can't analyze filepath=%s"%filepath)

... e.g. like this:

try:
    c = get_class(py)
    if hasattr(c, '__dict__'):
      # MappingProxyType from Python-style class definition
      d = c.__dict__
    else:
      # Dict from YAML-style class definition
      d = c
except:
    d['revision'] = standard.writeError(self,"[remoteFiles.traverse]: can't analyze filepath=%s"%filepath)

[cmeier76]

[1] fully agree not touching the well established repo-sync/deploy.

@drfho To achieve this, IMHO sync between existing __init__.py and the proposed new schema.yaml generated from HTML templates in the Storybook folders should be ensured manual at filesystem-level.

There may be a helper user interface in zmi of the ZMSMetaobjManager:

• highlight changed attributes in either model or schema, which existing in both but have different settings -> to be resolved manually
• propose adding of new schema-attributes, which are missed in model so far -> to be inserted in ZMSMetaobjManger by pre-filled input fields with the values set in schema
• additional model-only attributes will be intended and/or technical -> if there are new model attribute(s) also needed in schema, they have to be added in the template(s) manually

[drfho]

Python-based Schema Code

Ref: #407 (comment)
okay, the we may stick with the traditional py schema. Function save_schema_as_python() creates the __init__.py-file

ZMS/Products/zms/skins/storybook/create_schema.py

Lines 133 to 158 in 68c4666

	# #####################################################################################
	# Function: Save the Schema as a Python File
	# #####################################################################################
	def save_schema_as_python(schema, output_file_path):
	"""
	Save the schema dictionary to a Python file.
	:param schema: The schema dictionary to be saved.
	:param output_file_path: The path where the Python file will be saved.
	"""
	python_file = os.path.join(output_file_path, '__init__.py')
	with open(python_file, 'w', encoding='utf-8') as output_file:
	output_file.write('class %s:\n'% schema['class']['id'])
	output_file.write(f'\t"""\n\tpython-representation of {schema['class']['id']}\n\t"""\n\n')
	# Sort keys alphabetically
	for key, value in sorted(schema['class'].items()):
	# Skip the 'Attrs' dictionary, as it will be handled separately
	if key != 'Attrs':
	value = standard.str_json(value, encoding="utf-8", formatted=True, level=2, allow_booleans=False)
	output_file.write(f'\t# {key.capitalize()}\n')
	output_file.write(f'\t{key} = {value}\n\n')
	# Write the 'Attr' dictionary
	output_file.write('\tclass Attrs:\n')
	for key, value in schema['class']['Attrs'].items():
	value = standard.str_json(value, encoding="utf-8", formatted=True, level=3, allow_booleans=False)
	output_file.write(f'\t\t{key} = {value}\n\n')
	return True

Result:

ZMS/Products/zms/skins/storybook/schema/infobox/__init__.py

Lines 1 to 65 in 68c4666

	class infobox:
	"""
	python-representation of infobox
	"""
	# Access
	access = {"delete_custom":""
	,"delete_deny":[]
	,"insert_custom":"{$}"
	,"insert_deny":[]}
	# Enabled
	enabled = 1
	# Id
	id = "infobox"
	# Name
	name = "Infobox"
	# Package
	package = "com.zms.infobox"
	# Revision
	revision = "0.0.1"
	# Type
	type = "ZMSObject"
	class Attrs:
	title = {"default":""
	,"id":"title"
	,"keys":[]
	,"mandatory":0
	,"multilang":1
	,"name":"Title"
	,"repetitive":0
	,"type":"string"}
	attr_dc_description = {"default":""
	,"id":"attr_dc_description"
	,"keys":[]
	,"mandatory":0
	,"multilang":1
	,"name":"Description"
	,"repetitive":0
	,"type":"text"}
	url = {"default":""
	,"id":"url"
	,"keys":[]
	,"mandatory":0
	,"multilang":1
	,"name":"Url"
	,"repetitive":0
	,"type":"url"}
	linktext = {"default":"Read more..."
	,"id":"linktext"
	,"keys":[]
	,"mandatory":0
	,"multilang":1
	,"name":"Linktext"
	,"repetitive":0
	,"type":"string"}