[Proposal] Change Component to be an instance, not schema

[hongchaodeng]

Problem description

Problem 1: Upgrading Application requires writing another Component

Currently the container image is hard-coded in ComponentSchematic. Each time a new version of the application needs to rollout, we need to write a new ComponentSchematic and update ApplicationConfiguration simultaneously. This combination of work is complex. We heard complains from our users often, and has raised issue #265. We want to let developers define only one component and update image in one place.

Problem 2: Mapping existing Application definition to OAM is not intuitive

Many existing projects (both open source and internally) have defined Application schema and done deployment with instances of Application. For examples:

• Crossplane defines KubernetesApplication CRD and let users deploys KubernetesApplication CR.
• Alibaba: It's similar to what Pinterest did in this blog. Our projects define AlibabaService CRD. Alibaba's internal projects define AlibabaService CRD. Developers write AlibabaService CR yaml and put it in Git, which then triggers CI/CD pipeline.

The current model having Component as another schema on top of Workload as a schema doesn't fit well for these real world cases.

Proposal

There are two issues in OAM that we want to improve:

1. Change Component to be an instance instead of schema, and
2. Change workload settings to schema, and move all of workload related information here, e.g. container definition, from Component.

Walkthrough

Let me give a concrete example. Let's say I have defined a WorkloadType AlibabaService:

apiVersion: oam.dev/v1alpha1
kind: WorkloadType
metadata:
  name: alibaba-service
spec:
  group: alibaba.io/v1
  names: [AlibabaService]
  settings: |
    {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "type": "object",
      "required": [
        "image"
      ],
      "properties": {
        "image": {
          "type": "string",
          "description": "The container image to deploy"
        }
      }
    }

Then deploy a Component:

apiVersion: oam.dev/v1alpha1
kind: Component
metadata:
  name: my-nginx
spec:
  workloadType: alibaba.io/v1.AlibabaService
  workloadSettings:
    image: nginx/v1

While it might look the same up until now, the key change happens when we write ApplicationConfiguration:

kind: ApplicationConfiguration
spec:
  Components:
  - componentRef: my-nginx
    traits:
    - name: ingress
    - name: monitoring

As described above, Component becomes an instance itself, and ApplicationConfiguration references instances of Components.

What's changed here is that previously we asked developers how to describe the app, but now we ask them what to deploy. This brings us the following benefits:

• Describing how to run the app is now entirely encapsulated in Workload provided by the infra, and dev will just use the capabilities to deploy the app.
• Dev will now know exactly what the deployment of the app looks like because they write it, reducing the complexities of passing information between dev and ops.
• Ops will focus on attaching operational traits to components.

This better resembles the use cases we have been encountering and solves those problems that users asked.

How it solves above problems?

Problem 1: Since component is an instance now, updating image would trigger correlated deployments to be updated as well.

Problem 2: Application CRD could be modeled as WorkloadType and Application CR could be modeled as Component. This would intuitively map existing use cases to OAM.

[hongchaodeng]

cc @prasek @resouer @wonderflow @suhuruli @technosophos @vturecek

[wonderflow]

I totally agree with this proposal and this is just what I want described in #272.

I think I could close #272 and modify the PR #277 for this issue.

[negz]

Hi folks,

I'm the tech lead for Crossplane core. I'm currently thinking about how Crossplane might function as an OAM runtime, and what changes to the OAM spec (if any) might help enable that. I'm wondering whether this proposal would change how runtimes render and process OAM configuration.

My Rust is terrible, but my understanding is that in Rudr we build the Kubernetes configuration necessary to deploy an application by:

1. Watching for resources of kind ApplicationConfig
2. Iterating over each ComponentSchematic referenced by the ApplicationConfig
3. Using the ApplicationConfig's component parameters to render the ComponentSchematic
4. Iterating over each Trait referenced by the ApplicationConfig component
5. Using the Trait properties to modify or extend the config rendered by the ComponentSchematic

Or put otherwise, nothing happens if I only create a ComponentSchematic. No controller watches for resources of this kind - it is just an input to the processing of an ApplicationConfig.

Do I understand correctly that the problem this proposal describes is that because no controller is watching for ComponentSchematic, editing a ComponentSchematic and changing its image would not trigger the ApplicationConfig to be re-processed and thus would not trigger a deployment of the newly referenced image?

Is the idea that by reframing ComponentSchematic as an "instance" of Component, a controller would watch for Component, and thus updating the image a Component specifies would trigger a deployment of that image? If this is the case, would creating a Component alone (without ever referencing it from an ApplicationConfig) trigger a deployment?

[hongchaodeng]

Hi @negz . Great to hear from you!
What you describe looks correct to me.
Let me reply to some key questions you asked:

editing a ComponentSchematic and changing its image would not trigger the ApplicationConfig to be re-processed and thus would not trigger a deployment of the newly referenced image?

In fact, we don't edit ComponentSchematic, but create a new ComponentSchematic.
See Rudr upgrade section.

Is the idea that by reframing ComponentSchematic as an "instance" of Component

No. But Component as an instance of Workload.

a controller would watch for Component, and thus updating the image a Component specifies would trigger a deployment of that image?

Yes.

If this is the case, would creating a Component alone (without ever referencing it from an ApplicationConfig) trigger a deployment?

No. Component is a subsidiary resource of ApplicationConfig. The runtime controller needs to watch ApplicationConfig to understand the whole picture and plan for the work (e.g. canary rollout, traffic split).

[resouer]

Glad to hear from you @negz!

This PR is needed because currently OAM ComponentSchematic could not be mapped to either K8s CRD (schema) or CR (instance) directly: it's a combination of schema and instance. This makes it very hard for existing k8s based app delivery projects like crossplane.io to say:

"Hey, I already have a KubernetesApplication, can I leverage OAM Component to define a similar but more generic CrossplaneApplication without big change in my current logic?"

This may not be so explicit, so I've already tried to draft an illustrated version of proposal and will share it to you tomorrow.

[negz]

Thanks for the fast reply @hongchaodeng!

See Rudr upgrade section.

Ah, that's useful thank you.

The runtime controller needs to watch ApplicationConfig to understand the whole picture and plan for the work (e.g. canary rollout, traffic split).

This seems sensible to me, but it also raises more questions about this proposal. :)

It seems that a Component alone is insufficient to build the information needed to deploy an application - we need the ApplicationConfig and any Traits it references as well. Given this, it seems like we don't really want a controller watching for Component and reconciling purely the information stored in each Component that changes, but instead a controller that watches for Component and queues a reconcile of the ApplicationConfig to which that Component belongs. So not "reconcile this Component if it changes", but "reconcile this ApplicationConfig if it or any of its referenced Components change". Would this require a reference back from each Component to the ApplicationConfig to which it belongs?

One thing I like about the current ComponentSchematic is that it can be created by an application application developer to publish a bunch of "sane default" settings, which the application operator can then override (by supplying parameter values). It sounds like, per #265, the model is a little flawed because it assumes the application developer will publish a new (likely almost identical) ComponentSchematic each time they publish a new build of their software.

I wonder if there is a solution that retains the ability for the application developer to publish a sane set of defaults, and allows the application owner to override them (including the image version) in a fashion that is more CRD-compatible. Perhaps something like this:

apiVersion: core.oam.dev/v1alpha1
kind: ComponentSchematic
metadata:
  name: frontend
  annotations:
    description: >
      Sample schematic that describes the administrative interface for our Twitter bot.
spec:
  workloadType: core.oam.dev/v1alpha1.Server
  osType: linux
  containers:
  - name: my-twitter-bot-frontend
    image:
      name: example/my-twitter-bot-frontend:latest
    resources:
      cpu:
        required: 1.0
      memory:
        required: 100MB
    ports:
    - name: http
      value: 8080
    env:
    - name: USERNAME
    - name: PASSWORD
    - name: BACKEND_ADDRESS
    livenessProbe:
      httpGet:
        port: 8080
        path: /healthz
    readinessProbe:
      httpGet:
        port: 8080
        path: /healthz

apiVersion: core.oam.dev/v1alpha1
kind: Component
metadata:
  name: cool-frontend
  annotations:
    version: v1.0.0
spec:
  schematicRef:
    name: frontend
  containers:
  - name: my-twitter-bot-frontend
    image:
      name: example/my-twitter-bot-frontend:v1.0.0
    env:
    - name: USERNAME
       value: cooladmin
    - name: PASSWORD
       value: verysecret
    - name: BACKEND_ADDRESS
       value: https://example.org

kind: ApplicationConfiguration
metadata:
  name: coolest-app
spec:
  components:
  - componentRef:
       name: cool-frontend
    traits:
    - name: ingress
    - name: monitoring

In this scenario:

1. The application developer publishes their component schematic. Schematics no longer publish or accept parameters.
2. The application operator publishes their component. This component references the schematic published by the application developer, and thus inherits all of its settings. Any setting published by the component either replaces or extends the settings of its schematic; it's a little like a Kustomize patch overlay.
3. The application operator publishes the application config, referencing components (not schematics).

Under this model all schema definition is left to CRDs - CRs only specify values - but the application developer still gets to specify a set of default values for their component, and the application operator still gets to override those default values.

[hongchaodeng]

Hi @negz . That's a very thoughtful discussion.

Would this require a reference back from each Component to the ApplicationConfig to which it belongs?

I think this could be an implementation detail (e.g. reverse index).

Regarding the frontend ComponentSchematic shown above, we want to move all the schema settings (e.g. container definition) into Workload. Once we model the frontend ComponentSchematic as a Workload, then we have converge on the same workflow basically.

an application application developer to publish a bunch of "sane default" settings, which the application operator can then override (by supplying parameter values).

This is the problem of managing and passing parameters. And we can resolve it with tools like Helm template, Kustomize, etc. By asking developers to write Component instance, we defer such responsibility to external tools. For example, dev writes a Component:

apiVersion: core.oam.dev/v1alpha1
kind: Component
metadata:
  name: cool-frontend
  annotations:
    version: v1.0.0
spec:
  containers:
  - name: my-twitter-bot-frontend
    image:
      name: example/my-twitter-bot-frontend:v1.0.0
    env:
    - name: USERNAME
       value: {{ .Values.Username | default "a" }}
    - name: PASSWORD
       value: {{ .Values.Password | default "b" }}

Under this model CRD -> Workload, CR -> Component. Application developers and operators convey parameters and default values via external tools like Helm or Kustomize.

What you described above looks right to me, but at this stage we could reduce schema layers (combine ComponentSchematic into Workload) and give users a minimal spec to setup.

[negz]

Under this model CRD -> Workload, CR -> Component

Are you saying that in a Kubernetes based implementation kind: Workload would not exist, and instead the workload OAM concept would be implemented as the CRD that defines Component, or are you saying kind: Workload would define the schema for kind: Component?

[hongchaodeng]

kind: Workload would define the schema for kind: Component

This one

[wonderflow]

I think this issue could be closed now

[hongchaodeng]

Yeah. This is already done in alpha2.