Divide and Conquer Approach

[EneaGore]

Motivation and Context

A classic approach. Split up the grading instructions into criteria and create specific prompts for each criterion. Asynchronously invoke the LLM for each criterion (practically 0 added latency) (output token remains more or less the same, but elevated input token costs). The responses are combined into one final assessment.

Some preconditions:

1. The exercise must have structured grading criterions.
2. The usage counts must be all the same within one criterion. They can differ between criterions.
3. The usage counts must be well defined, If you use the default 0, this approach takes it as if it can be applied as many times as possible.

Evaluation

Evaluation on Exercise: System Design Review (SS21) with 9 credits. and 3 Criteria.

Steps for Testing

Testserver States

Note

These badges show the state of the test servers.
Green = Currently available, Red = Currently locked
Click on the badges to get to the test servers.

Screenshots