Add Meadow.Data.Schemas.AgentPlan schema and Meadow.Data.Planner context

[bmquinn]

Summary

First, make sure to run the migrations:
mix ecto.migrate

alias Meadow.Data.Planner

# Create a work first
{:ok, work} = Works.create_work(%{accession_number: "1234", descriptive_metadata: %{
  title: "Test Work Title", 
  subject: [
    %{
      role: %{id: "TOPICAL", scheme: "subject_role"},
      term: %{id: "http://id.loc.gov/authorities/subjects/sh85141086"}
    }
  ]
}})

# Index it
Meadow.Data.Indexer.reindex_all

# Define your query
query = ~s({"query": {"match": {"_id": "#{work.id}"}}})
# note: you can use string queries like: "id:(#{work.id})" too

# Creating a plan automatically creates PlanChange entries for each matching query result:
{:ok, plan} = Planner.create_plan(%{prompt: "Add date_created EDTF fields", query: query})
plan = Planner.get_plan(plan.id) |> Meadow.Repo.preload(:plan_changes)

# Updating the automatically created PlanChange with an `add`
change_a = Planner.get_plan_changes_by_work(plan, work) |> hd()
{:ok, updated_change_a} =
  Meadow.Data.Planner.update_plan_change(change_a, %{
    add: %{descriptive_metadata: %{date_created: ["1896-11-10"]}}
  })

# Manually creating a separate PlanChange with a `delete`
{:ok, change_b} = Planner.create_plan_change(%{
  plan_id: plan.id,
  work_id: work.id,
  user: "bmq449",
  delete: %{
    descriptive_metadata: %{
      subject: [
        %{
          role: %{id: "TOPICAL", scheme: "subject_role"},
          term: %{id: "http://id.loc.gov/authorities/subjects/sh85141086"}
        }
      ]
    }
  }
})

# Approve the PlanChanges
Planner.approve_plan_change(updated_change_a, "bmq449")  
Planner.approve_plan_change(change_b, "bmq449")

# Approve the Plan
{:ok, plan} = Planner.approve_plan(plan, "bmq449")

# Execute the plan
Planner.execute_plan(plan)

# Check the work
work = Works.get_work(work.id)
work.descriptive_metadata.date_created
[%{edtf: "1896-11-10", humanized: "November 10, 1896"}]
work.descriptive_metdata.subject
[]

Specific Changes in this PR

• list changes
• list changes

Version bump required by the PR

See Semantic Versioning 2.0.0 for help discerning which is required.

• Patch
• Minor
• Major

Steps to Test

Please let end users know what they need to do to test on staging or production

Also please let developers know if there are any special instructions to test this in the development environment.

🚀 Deployment Notes

Note - if you check any of these boxes go to the (always open) main <- staging PR and add detailed notes and instructions to help out others who may end up deploying your changes to production

• Backward compatible API changes
  • Database Schema changes
  • GraphQL API
  • Elasticsearch API
  • Ingest Sheet
  • CSV metadata export/update API
  • Shared Links export API
• Backwards-incompatible API changes
  • Database Schema changes
  • GraphQL API
  • Elasticsearch API
  • Ingest Sheet
  • CSV metadata export/update API
  • Shared Links export API
• Requires data migration
• Requires database triggers disabled during deployment/migration
• Requires reindex
• Terraform changes
  • Adds/requires new or changed Terraform variables
• Pipeline configuration changes (requires mix meadow.pipeline.setup run)
• Requires new variable added to miscellany
• Specific deployment synchronization instructions with other apps/API's
• Other specific instructions/tasks

Tested/Verified

• End users/stakeholders

[mbklein]

My understanding of the plan is that it's going to target individual works with specific updates, e.g., a single plan could:

• Add 2 subjects to one work
• Add the same 2 subjects and a note to another work
• Add 4 different subjects to a third work

So my questions given this implementation are:

1. Is each change to a work a different plan? I know we looked at the UI that will allow accepting or rejecting individual changes, but is each of those changes its own plan, or is the plan a collection of those changes?
2. If each plan is atomic (i.e., one specific set of related changes to a single work), do we need the plan to have a query or just a work_id?

This looks like an alternate implementation of the regular batch update, but with the changes submitted by an agent instead of a user. But I got the impression that we were trying to make the agent's approach much more granular.

[kdid]

@bmquinn - I think the PR needs to go into deploy/meadow-ai

[kdid]

I'm not sure if I have the same questions as @mbklein or not so apologies if this is a duplicate question.

So, from our discussions what I envisioned was basically one row for each work with a plan_id column (so actually two tables). So that if the front end needed to retrieve changes for an individual work in the plan or to edit or remove an individual work in the plan they would be editing/retrieving one row. And to get the whole plan you would lookup by plan id.

So, my questions are, what is the id in the table here? Is that the plan's id? Is it one entry for the entire plan and then the changeset column contains all the works? And then where do you associate the specific changes in the changeset with the appropriate work id?

[mbklein]

I think our questions are different on the specifics but the same in terms of conceptualizing exactly what a “plan” is.

[bmquinn]

Honestly my approach was to keep things as simple as possible so we could have this discussion, and coalesce around what a plan really is 😄. It was pretty vague when I started working on the issue but I'm going to think about the questions and probably push up another commit that attempts to address them together. Thanks @mbklein and @kdid . Also, rebased on the deploy/meadow-ai branch thanks for the heads up.

[mbklein]

Looking closer, I think the schema looks great except:

• I'd call what you have something like a PlanChange (or PlannedChange or ProposedChange) instead of a Plan (see below)
• I'd replace query with work_id so the agent can submit one or more (non-conflicting) changesets per work and have them individually accepted or rejected by the user
• I'd add a Plan schema that aggregates a bunch of changes into a single executable plan.
• I think user can probably move from the change level to the plan level.

[mbklein]

I like the overall shape of this one. I think it's worth merging into the meadow-ai branch and working with it for a bit.

[kdid]

Looks 👍