Merge pull request #575 from NCATSTranslator/metakg-updates

Add MetaEdge changes to 2.0 migration guide

Author: tokebe

ImplementationGuidance/MigrationGuides/MigrationAndImplementationGuide2-0.md

@@ -2,7 +2,7 @@
 
 This guide lays out the format and functionality changes for queries and responses in TRAPI 2.0.0 (compared to 1.6.0-beta).
 
-TRAPI 2.0 includes many breaking changes, new/reintroduced functionality, and format changes designed to slim down TRAPI messages. This guide provides before-after examples to illustrate the more complex changes and a list for the other important changes (mainly formatting).  
+TRAPI 2.0 includes many breaking changes, new/reintroduced functionality, and format changes designed to slim down TRAPI messages. This guide provides before-after examples to illustrate the more complex changes and a list for the other important changes (mainly formatting).
 
 
 ## Changes
@@ -12,7 +12,7 @@ TRAPI 2.0 includes many breaking changes, new/reintroduced functionality, and fo
 
 #### BEFORE
 
-In 1.6.0-beta, you could include `attribute_constraints` and `qualifier_constraints` on QEdges. If you wanted to only include or exclude specific knowledge_level/agent_type (KL/AT) values, you'd use `attribute_constraints` because KL/AT are stored in Edge `attributes`. If you wanted to only include or exclude specific sources (infores), you may have used `attribute_constraints`. BUT this format no longer makes sense after we moved source info out of Edge `attributes` into its own top-level property `sources` several versions ago (1.4.0-beta). 
+In 1.6.0-beta, you could include `attribute_constraints` and `qualifier_constraints` on QEdges. If you wanted to only include or exclude specific knowledge_level/agent_type (KL/AT) values, you'd use `attribute_constraints` because KL/AT are stored in Edge `attributes`. If you wanted to only include or exclude specific sources (infores), you may have used `attribute_constraints`. BUT this format no longer makes sense after we moved source info out of Edge `attributes` into its own top-level property `sources` several versions ago (1.4.0-beta).
 
 <details><summary>A QEdge in 1.6.0-beta with all of these constraints would look like this (click to expand)
 </summary>
@@ -57,22 +57,22 @@ In 1.6.0-beta, you could include `attribute_constraints` and `qualifier_constrai
 ```
 
 </p>
-</details> 
+</details>
 
 #### AFTER
 
 In 2.0, there is instead one property on a QEdge, `constraints`, that holds all the types of constraints, organized by key. There are 5 keys currently specified:
 
-* `knowledge_level`: we moved KL/AT out of Edge `attributes` and into its own top-level properties on an Edge (see #4), so they need corresponding separate constraints. This constraint is an object with two keys: `behavior` (`ALLOW` or `DENY`) and `values` (an array of strings). 
-   * `ALLOW` means "ANY (at least 1) of the `values` MUST be in the matched Edge's corresponding property". 
-   * `DENY` means "ALL of the `values` MUST NOT be in the matched Edge's corresponding property". 
+* `knowledge_level`: we moved KL/AT out of Edge `attributes` and into its own top-level properties on an Edge (see #4), so they need corresponding separate constraints. This constraint is an object with two keys: `behavior` (`ALLOW` or `DENY`) and `values` (an array of strings).
+   * `ALLOW` means "ANY (at least 1) of the `values` MUST be in the matched Edge's corresponding property".
+   * `DENY` means "ALL of the `values` MUST NOT be in the matched Edge's corresponding property".
 * `agent_type`: see above (KL)
    * FYI: if a specified value has descendants (ex: `automated_agent`), the tool MUST treat those descendants (ex: `text_mining_agent`, etc.) as if they were included in the `values` array (i.e. "hierarchy expansion").
-* `sources`: this constrains the Edge `sources`. It has the same keys as KL/AT (`behavior`, `values`) plus the optional `primary_only` (if true, the constraint ONLY applies to the `primary_knowledge_source`). 
+* `sources`: this constrains the Edge `sources`. It has the same keys as KL/AT (`behavior`, `values`) plus the optional `primary_only` (if true, the constraint ONLY applies to the `primary_knowledge_source`).
 * `attributes`: minItems 1 and `name` no longer required, otherwise the same as previous `attribute_constraints`
-* `qualifiers`: simplified format to an array of objects but preserved previous behavior. Each object represents a qualifier-set, and multiple objects/sets have an `OR` relationship. Within an object, the keys are the "qualifier-type-ids" and their values are the "qualifier values". Multiple key/value pairs in one object/set have an `AND` relationship. 
+* `qualifiers`: simplified format to an array of objects but preserved previous behavior. Each object represents a qualifier-set, and multiple objects/sets have an `OR` relationship. Within an object, the keys are the "qualifier-type-ids" and their values are the "qualifier values". Multiple key/value pairs in one object/set have an `AND` relationship.
 
-**NOTE on SUBCLASSING**: we haven't decided yet how QEdge constraints work with subclassing. For now, QEdge constraints should only apply to the bound Edges (constructed in subclassing cases), NOT to the bound Edge's support_graph edges. QNode constraints also has this ambiguity/confusion with subclassing. 
+**NOTE on SUBCLASSING**: we haven't decided yet how QEdge constraints work with subclassing. For now, QEdge constraints should only apply to the bound Edges (constructed in subclassing cases), NOT to the bound Edge's support_graph edges. QNode constraints also has this ambiguity/confusion with subclassing.
 
 <details><summary>The same QEdge in 2.0 would look like this (click to expand)</summary>
 <p>
@@ -108,7 +108,7 @@ In 2.0, there is instead one property on a QEdge, `constraints`, that holds all
 ```
 
 </p>
-</details> 
+</details>
 
 <br>
 
@@ -152,16 +152,16 @@ In 2.0, there is instead one property on a QEdge, `constraints`, that holds all
 ```
 
 </p>
-</details> 
+</details>
 
 
 ### 2. New Query/Response Parameters
 
 #### BEFORE
 
-In 1.6.0-beta, `log_level` and `bypass_cache` were top-level properties in `Query` and `AsyncQuery`. 
+In 1.6.0-beta, `log_level` and `bypass_cache` were top-level properties in `Query` and `AsyncQuery`.
 
-A query with them would look like this: 
+A query with them would look like this:
 
 ```json
 {
@@ -175,7 +175,7 @@ A query with them would look like this:
 
 In 2.0, these are moved under a new top-level property `parameters` (their behavior is otherwise kept the same). `parameters` also includes a new parameter/property `timeout`, so a client can state how long they will wait for a response.
 
-Tools can also use the new `parameters` property to hold undefined query-time parameters that affect overall behavior of the server in query execution, like specifying data-tier in the Translator ecosystem. 
+Tools can also use the new `parameters` property to hold undefined query-time parameters that affect overall behavior of the server in query execution, like specifying data-tier in the Translator ecosystem.
 
 `parameters` has also been added to `Response`; the server receiving a Query/AsyncQuery with `parameters` MUST echo them in its Response. If there is a conflict between the `parameters` and the server's capabilities, the server SHOULD return HTTP `409`.
 
@@ -196,9 +196,9 @@ A query with the same parameters (plus timeout and custom data-tier) would look
 
 ### 3. Binding Structure Changes (Node/Edge/Path)
 
-#### BEFORE 
+#### BEFORE
 
-In 1.6.0-beta, the 3 kinds of bindings have this format: 
+In 1.6.0-beta, the 3 kinds of bindings have this format:
 
 ```json
 ...
@@ -259,7 +259,7 @@ Details:
 ```
 
 </p>
-</details> 
+</details>
 
 <details><summary>Example result with node/path bindings</summary>
 <p>
@@ -296,7 +296,7 @@ Details:
 ```
 
 </p>
-</details> 
+</details>
 
 #### AFTER
 
@@ -316,7 +316,7 @@ In 2.0, the format is simplified:
 ```
 
 Changes:
-* `<node/edge/path>_bindings` are now `minProperties: 1` (i.e. when these fields are present, they MUST contain data) 
+* `<node/edge/path>_bindings` are now `minProperties: 1` (i.e. when these fields are present, they MUST contain data)
 * `ids` arrays are `minItems: 1` (this property is still required)
 * `attributes` were removed from NodeBinding/EdgeBinding (were never used, empty arrays bloated responses)
 * `query_id` was removed from NodeBinding (obsolete with the current subclassing behavior)
@@ -380,9 +380,9 @@ Changes:
 </details>
 
 
-### 4. KL/AT turned into top-level Edge properties, now required 
+### 4. KL/AT turned into top-level Edge properties, now required
 
-In 1.6.0-beta, `knowledge_level` and `agent_type` are stored in `Edge.attributes`, which is not a required property. However, for several years the Translator Consortium has actually required KL/AT on Edges. 
+In 1.6.0-beta, `knowledge_level` and `agent_type` are stored in `Edge.attributes`, which is not a required property. However, for several years the Translator Consortium has actually required KL/AT on Edges.
 
 In 2.0, they are now top-level properties that are required on an `Edge`. The way to constrain them in queries has also changed (own keys under `QEdge.constraints`) - see #1 for details.
 
@@ -402,7 +402,7 @@ Example snippet of an `Edge` in 2.0, showing the top-level KL/AT:
 
 ### 5. Add `COLLATE` option to `QNode.set_interpretation`
 
-In 2.0, `COLLATE` is an option that is only allowed on QNodes with no `ids` set and indicates that multiple matching nodes MUST be collated into a single Result, rather than put into separate Results. This restores some of the `QNode.is_set` behavior that was removed in 1.5.0 (don't confuse with **Node**.is_set!). 
+In 2.0, `COLLATE` is an option that is only allowed on QNodes with no `ids` set and indicates that multiple matching nodes MUST be collated into a single Result, rather than put into separate Results. This restores some of the `QNode.is_set` behavior that was removed in 1.5.0 (don't confuse with **Node**.is_set!).
 
 When `COLLATE` is set, `QNode.member_ids` must not be used.
 
@@ -421,12 +421,12 @@ Drug A -interacts_with→ Gene B -causes→ Diabetes
        ↘interacts_with→ Gene C -causes↗
 ```
 
-The `node_bindings.[Gene QNode].ids` would include Genes A, B, and C. The edge_bindings would be collated accordingly. 
+The `node_bindings.[Gene QNode].ids` would include Genes A, B, and C. The edge_bindings would be collated accordingly.
 
 
 ### 6. `null` is no longer a valid value in queries and responses
 
-To convey "no data", instead omit the field or use an empty array/object if the schema allows (doesn't set `minProperties`/`minItems`). However, unless the field's description explicitly states that the empty array/object MUST be used, we strongly encourage omitting fields to reduce needless bloat. 
+To convey "no data", instead omit the field or use an empty array/object if the schema allows (doesn't set `minProperties`/`minItems`). However, unless the field's description explicitly states that the empty array/object MUST be used, we strongly encourage omitting fields to reduce needless bloat.
 
 Examples:
 * `Message.knowledge_graph` MUST be omitted, not be set to `null`, when there is no data (ex: a query, or a response with no data found).
@@ -440,7 +440,7 @@ For many array and object properties, `minItems`/`minProperties` was set to 1. T
 
 ### 8. `AuxiliaryGraph.attributes` was removed
 
-It was previously required, but never used and its empty arrays bloated responses. Additional undefined properties are still allowed in `AuxiliaryGraph` objects. 
+It was previously required, but never used and its empty arrays bloated responses. Additional undefined properties are still allowed in `AuxiliaryGraph` objects.
 
 
 ### 9. Properties changed to not-required:
@@ -457,14 +457,23 @@ These properties were changed to not required:
 
 Analysis allows edge_bindings and path_bindings to be present together (for experimental use only, small change introduced when simplifying schema classes).
 
+### 11. MetaEdges can now advertise KL/AT and sources
+
+Three properties were added to `MetaEdge`:
+* `knowledge_levels`: Should contain all relevant knowledge_level values that the meta edge covers
+* `agent_types`: Should contain all relevant agent_type values that the meta edge covers
+* `sources`: Should contain all the source inforeses that contribute to this meta edge, regardless of role.
+  May omit aggregator resource_ids created by the service producing the meta knowledge graph.
+
+These properties allow a service to advertise filterable values for the new `QEdge` KL/AT and source filtering.
 
 ## Full Examples
 
 This is an example of a 1.6.0-beta Response "transformed" into 2.0 (includes query_graph/parameters). Note that the 2.0 query/response has 2 functional differences:
 * COLLATE was set on the intermediate QNode
 * new parameters were added (timeout, custom tiers)
 
-These examples show the main changes 1-5 (#1: all types of constraints, #5: COLLATE in 2.0 only) and the other changes 1,2, and 4. 
+These examples show the main changes 1-5 (#1: all types of constraints, #5: COLLATE in 2.0 only) and the other changes 1,2, and 4.
 
 [1.6.0-beta Response](../DataExamples/1-6_example_response.json)