Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Promote "Metadata in Table Schema" recipe to the specs. #961

Merged
merged 38 commits into from
Jan 27, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
ebfc90f
Schema Metadata spec proposal
pierrecamilleri Apr 25, 2024
cdbd1c9
🔵 wording and typos
pierrecamilleri Apr 25, 2024
2631462
taking into account review comments
pierrecamilleri Jun 11, 2024
83396b7
Remove `path` property
pierrecamilleri Jun 24, 2024
d9221c6
Remove `updated` property
pierrecamilleri Jun 24, 2024
ded47a6
inherits "if distribted in a data package descriptor"
pierrecamilleri Jun 24, 2024
2161fb0
resource -> example resource
pierrecamilleri Jun 24, 2024
f7ffacf
`name` "as for Data Package
pierrecamilleri Jun 24, 2024
e59b7d9
Remove sources and build profile
pierrecamilleri Jun 24, 2024
213ef87
`examples` are `Data Resource`s
pierrecamilleri Jun 26, 2024
5fdd322
Attempt at removing "schema" property from table schema example
pierrecamilleri Jun 26, 2024
9a32b79
Revert "Attempt at removing "schema" property from table schema example"
pierrecamilleri Jun 26, 2024
1088c70
Revert "`examples` are `Data Resource`s"
pierrecamilleri Jun 26, 2024
229bbf6
Clear-up examples documentation (ref to data files)
pierrecamilleri Jun 26, 2024
eddb54c
fix: created stores only creation date
pierrecamilleri Jun 26, 2024
063ba61
changelog
pierrecamilleri Jul 5, 2024
02c8482
fix: link
pierrecamilleri Jun 26, 2024
fac7c7f
fix: unintended change
pierrecamilleri Jul 5, 2024
e640063
fix: link
pierrecamilleri Jul 5, 2024
c5ced5b
Update doc with Peters suggestions + fix links
pierrecamilleri Jul 12, 2024
800593f
"partial Data Resource" ?
pierrecamilleri Jul 12, 2024
3f80e97
Use title case for Data Package
peterdesmet Jul 15, 2024
5773c6b
Correct typo + align Data Package
peterdesmet Jul 15, 2024
935cbf5
Require name for examples
peterdesmet Jul 15, 2024
ad023bf
Update profiles/source/dictionary/common.yaml
pierrecamilleri Jul 26, 2024
eb1117e
Update profiles/source/dictionary/schema.yaml
pierrecamilleri Jul 26, 2024
dbd4ebd
Update profiles/source/dictionary/schema.yaml
pierrecamilleri Jul 26, 2024
79d915e
Update profiles/source/dictionary/schema.yaml
pierrecamilleri Jul 26, 2024
615a635
Update : example title to example name
pierrecamilleri Jul 26, 2024
a3233f6
Rephrase #961 changes
peterdesmet Sep 19, 2024
8288aa7
Change order cf. Data Package
peterdesmet Sep 19, 2024
8884e5d
Merge remote-tracking branch 'upstream/next' into main
pierrecamilleri Nov 18, 2024
94ace89
Changelog includes changes for v2.1
pierrecamilleri Nov 18, 2024
de00124
Linting
pierrecamilleri Nov 18, 2024
205a949
Apply suggestions from code review
pierrecamilleri Dec 20, 2024
09a285f
Update table-schema.mdx
pierrecamilleri Dec 20, 2024
dd9d791
npm run generate
pierrecamilleri Jan 8, 2025
e3975fc
Remove changes on v2.0 spec
pierrecamilleri Jan 27, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions content/docs/overview/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,42 @@ This document includes all meaningful changes made to the **Data Package standar

[fieldsMatch](/standard/table-schema/#fieldsMatch) has been corrected from array to string to match its definition ([#965](https://github.com/frictionlessdata/datapackage/issues/965)).

##### `schema.name` (new)

[`name`](/standard/table-schema/#name) allows to specify a name for a schema ([#961](https://github.com/frictionlessdata/datapackage/pull/961)).

##### `schema.title` (new)

[`title`](/standard/table-schema/#title) allows to specify a title for a schema ([#961](https://github.com/frictionlessdata/datapackage/pull/961)).

##### `schema.description` (new)

[`description`](/standard/table-schema/#description) allows to specify a description for a schema ([#961](https://github.com/frictionlessdata/datapackage/pull/961)).

##### `schema.homepage` (new)

[`homepage`](/standard/table-schema/#homepage) allows to specify a homepage for a schema ([#961](https://github.com/frictionlessdata/datapackage/pull/961)).

##### `schema.version` (new)

[`version`](/standard/table-schema/#version) allows to specify a version for a schema ([#961](https://github.com/frictionlessdata/datapackage/pull/961)).

##### `schema.created` (new)

[`created`](/standard/table-schema/#created) allows to specify when a schema was created ([#961](https://github.com/frictionlessdata/datapackage/pull/961)).

##### `schema.keywords` (new)

[`keywords`](/standard/table-schema/#keywords) allows to specify keywords for a schema ([#961](https://github.com/frictionlessdata/datapackage/pull/961)).

##### `schema.contributors` (new)

[`contributors`](/standard/table-schema/#contributors) allows to specify contributors for a schema ([#961](https://github.com/frictionlessdata/datapackage/pull/961)).

##### `schema.examples` (new)

[`examples`](/standard/table-schema/#examples) allows to specify a list of illustrative data resources that use a schema ([#961](https://github.com/frictionlessdata/datapackage/pull/961)).

## v2.0

This release includes a rich set of specification improvements to make Data Package a finished product (see [announcement](https://frictionlessdata.io/blog/2023/11/15/frictionless-specs-update/)). All changes were reviewed and accepted by the Data Package Working Group.
Expand Down
2 changes: 1 addition & 1 deletion content/docs/standard/data-package.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ An Array of string keywords to assist users searching for the package in catalog

### `contributors`

The people or organizations who contributed to this Data Package. It `MUST` be an array. Each entry is a Contributor and `MUST` be an `object`. A Contributor `MUST` have at least one property. A Contributor is `RECOMMENDED` to have `title` property and `MAY` contain `givenName`, `familyName`, `path`, `email`, `roles`, and `organization` properties:
The people or organizations that contributed to this Data Package. It `MUST` be an array. Each entry is a Contributor and `MUST` be an `object`. A Contributor `MUST` have at least one property. A Contributor is `RECOMMENDED` to have `title` property and `MAY` contain `givenName`, `familyName`, `path`, `email`, `roles`, and `organization` properties:

- `title`: A string containing a name of the contributor.
- `givenName`: A string containing the name a person has been given, if the contributor is a person.
Expand Down
42 changes: 40 additions & 2 deletions content/docs/standard/table-schema.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -202,14 +202,14 @@ In contrast with `field.constraints.unique`, `uniqueKeys` allows to define uniqu

#### `foreignKeys` {#foreignKeys}

A foreign key is a reference where values in a field (or fields) on the table ('resource' in data package terminology) described by this Table Schema connect to values a field (or fields) on this or a separate table (resource). They are directly modelled on the concept of foreign keys in SQL.
A foreign key is a reference where values in a field (or fields) of the table ('resource' in Data Package terminology) described by this Table Schema connect to values a field (or fields) of this or a separate table (resource). This concept is directly modelled on the concept of foreign keys in SQL.

The `foreignKeys` property, if present, `MUST` be an Array. Each entry in the array `MUST` be a `foreignKey`. A `foreignKey` `MUST` be a `object` and `MUST` have the following properties:

- `fields` - `fields` is an array of strings specifying the field or fields on this resource that form the source part of the foreign key. The structure of the array is as per `primaryKey` above.
- `reference` - `reference` `MUST` be a `object`. The `object`
- `MUST` have a property `fields` which is an array of strings of the same length as the outer `fields`, describing the field (or fields) references on the destination resource. The structure of the array is as per `primaryKey` above.
- `MAY` have a property `resource` which is the name of the resource within the current data package, i.e. the data package within which this Table Schema is located. For referencing another data resource the `resource` property `MUST` be provided. For self-referencing, i.e. references between fields in this Table Schema, the `resource` property `MUST` be omitted.
- `MAY` have a property `resource` which is the name of the resource within the current Data Package, i.e. the Data Package within which this Table Schema is located. For referencing another Data Resource the `resource` property `MUST` be provided. For self-referencing, i.e. references between fields in this Table Schema, the `resource` property `MUST` be omitted.

Here's an example:

Expand Down Expand Up @@ -277,6 +277,44 @@ If the value of the `foreignKey.reference.resource` property is an empty string
Data consumer MUST support the `foreignKey.fields` and `foreignKey.reference.fields` properties in a form of a single string e.g. `"fields": "a"` which was a part of the `v1.0` of the specification.
:::

#### `name`

A simple name or identifier for the schema (cf. [Data Package](https://datapackage.org/standard/data-package/#name)).

#### `title`

A string providing a title or one sentence description for the schema.

#### `description`

A description of the schema (cf. [Data Package](https://datapackage.org/standard/data-package/#description)).

#### `homepage`

A URL for the web page associated with the schema.

#### `version`

A version string identifying the version of the schema (cf. [Data Package](https://datapackage.org/standard/data-package/#version)). If not specified, the schema inherits from the Data Package if distributed in a Data Package descriptor.

#### `created`

The datetime on which the schema was created (cf. [Data Package](https://datapackage.org/standard/data-package/#created)).

#### `keywords`

An array of string keywords to assist users searching for the schema in catalogs.

#### `contributors`

The people or organizations that contributed to the schema (cf. [Data Package](https://datapackage.org/standard/data-package/#contributors)). If not specified the schema inherits from the Data Package if distributed in a Data Package descriptor.

#### `examples`

A list of Data Resources that use and illustrate the schema.

If present, it `MUST` be a non-empty array of objects. Each object is a [Data Resource](https://datapackage.org/standard/data-resource/) that `MUST` at least have the `name` and `path` property. The `path` must be a URL.

### Field

A field descriptor `MUST` be a JSON `object` that describes a single field. The descriptor provides additional human-readable documentation for a field, as well as additional information that can be used to validate the field or create a user interface for data entry.
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions profiles/dictionary/common.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ example:
}
homepage:
title: Home Page
description: The home on the web that is related to this data package.
description: The home on the web that is related to this descriptor.
type: string
format: uri
examples:
Expand Down Expand Up @@ -149,7 +149,7 @@ created:
}
keywords:
title: Keywords
description: A list of keywords that describe this package.
description: A list of keywords that describe this descriptor.
type: array
minItems: 1
items:
Expand Down
45 changes: 45 additions & 0 deletions profiles/dictionary/schema.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,24 @@ tableSchema:
}
missingValues:
"$ref": "#/definitions/tableSchemaMissingValues"
name:
"$ref": "#/definitions/name"
title:
"$ref": "#/definitions/title"
description:
"$ref": "#/definitions/description"
homepage:
"$ref": "#/definitions/homepage"
created:
"$ref": "#/definitions/created"
version:
"$ref": "#/definitions/version"
keywords:
"$ref": "#/definitions/keywords"
contributors:
"$ref": "#/definitions/contributors"
examples:
"$ref": "#/definitions/tableSchemaExamples"
examples:
- |
{
Expand Down Expand Up @@ -261,6 +279,33 @@ tableSchemaMissingValues:
{
"missingValues": []
}
tableSchemaExamples:
title: Examples
description: A list of Data Resources that use and illustrate the schema.
type: array
minItems: 0
items:
"$ref": "#/definitions/tableSchemaExample"
examples:
- |
{
"examples": [
{
"name": "valid-data",
"path": "http://example.com/valid-data.csv"
}
]
}
tableSchemaExample:
title: Example
description: A Data Resource that uses and illustrates the schema.
type: object
properties:
name:
"$ref": "#/definitions/name"
path:
"$ref": "#/definitions/path"
required: ["name", "path"]
tableSchemaFieldString:
type: object
title: String Field
Expand Down
Loading