Deployed 72e4090 with MkDocs version: 1.6.1

Author: celine-lee

agent/index.html

@@ -432,7 +432,7 @@ <h2 id="running">Running</h2>
 running AI agents to assist with code development and testing.
 In this example we use <a href="https://aider.chat/">Aider</a> as the
 baseline code completion agent</p>
-<div class="highlight"><pre><span></span><code>pip<span class="w"> </span>install<span class="w"> </span>aider
+<div class="highlight"><pre><span></span><code>pip<span class="w"> </span>install<span class="w"> </span>aider-chat
 </code></pre></div>
 <p>First we assume there is an underlying <code>commit0</code>
 project that is configured. To create a new project,
@@ -446,15 +446,16 @@ <h2 id="running">Running</h2>
 agent<span class="w"> </span>config<span class="w"> </span>aider
 </code></pre></div>
 <p>Finally we run the underlying agent. This will create a display
-that shows the current progress of the agent.</p>
-<div class="highlight"><pre><span></span><code>agent<span class="w"> </span>run
+that shows the current progress of the agent. Specify the branch
+you want to commit changes on.</p>
+<div class="highlight"><pre><span></span><code>agent<span class="w"> </span>run<span class="w"> </span>BRANCH
 </code></pre></div>
 <h3 id="extending">Extending</h3>
 <p>Refer to <code>class Agents</code> in <code>agent/agents.py</code>. You can design your own agent by inheriting <code>Agents</code> class and implement the <code>run</code> method.</p>
 <h2 id="notes">Notes</h2>
 <ul>
 <li>Aider automatically retries certain API errors. For details, see <a href="https://github.com/paul-gauthier/aider/blob/75e1d519da9b328b0eca8a73ee27278f1289eadb/aider/sendchat.py#L17">here</a>.</li>
-<li>When increasing --max-parallel-repos, be mindful of aider's <a href="https://github.com/paul-gauthier/aider/blob/75e1d519da9b328b0eca8a73ee27278f1289eadb/aider/sendchat.py#L39">60-second retry timeout</a>. Set this value according to your API tier to avoid RateLimitErrors stopping processes.</li>
+<li>When increasing <code>--max-parallel-repos</code>, be mindful of aider's <a href="https://github.com/paul-gauthier/aider/blob/75e1d519da9b328b0eca8a73ee27278f1289eadb/aider/sendchat.py#L39">60-second retry timeout</a>. Set this value according to your API tier to avoid RateLimitErrors stopping processes.</li>
 <li>Currently, agent will skip file with more than 1500 lines. See <code>agent/agent_utils.py#L199</code> for details.</li>
 <li>Running a full <code>all</code> commit0 split costs approximately $100 with Claude Sonnet 3.5.</li>
 </ul>

analysis/index.html

@@ -409,21 +409,21 @@ <h2 id="leaderboard-lite">Leaderboard (lite)</h2>
 </thead>
 <tbody>
 <tr>
-<td>Reference (Gold)</td>
-<td style="text-align: center;">10</td>
-<td style="text-align: center;">20.94</td>
-<td style="text-align: center;">NA</td>
-<td><a href="/analysis_gold_reference">Analysis</a></td>
-<td><a href="https://github.com/commit-0">Github</a></td>
-</tr>
-<tr>
 <td>Claude Sonnet 3.5 - Base</td>
-<td style="text-align: center;">5</td>
-<td style="text-align: center;">39.56</td>
+<td style="text-align: center;">0</td>
+<td style="text-align: center;">14.84</td>
 <td style="text-align: center;">09/25/2024</td>
 <td><a href="/analysis_test-save-commit0_baseline">Analysis</a></td>
 <td><a href="https://github.com/test-save-commit0">Github</a></td>
 </tr>
+<tr>
+<td>Reference (Gold)</td>
+<td style="text-align: center;">10</td>
+<td style="text-align: center;">27.83</td>
+<td style="text-align: center;">NA</td>
+<td><a href="/analysis_commit0_reference">Analysis</a></td>
+<td><a href="https://github.com/commit-0">Github</a></td>
+</tr>
 </tbody>
 </table>
 <h2 id="leaderboard-all">Leaderboard (all)</h2>

analysis_gold_reference/index.html renamed to analysis_commit0_reference/index.html

@@ -16,7 +16,7 @@
 
 
 
-        <title>Analysis gold reference - Commit-0</title>
+        <title>Analysis commit0 reference - Commit-0</title>
 
 
 
@@ -95,7 +95,7 @@
         <div class="md-header__topic" data-md-component="header-topic">
           <span class="md-ellipsis">
 
-              Analysis gold reference
+              Analysis commit0 reference
 
           </span>
         </div>
@@ -331,80 +331,80 @@ <h1 id="submission-name-reference-gold-split-lite">Submission Name: <strong>Refe
 <td>simpy</td>
 <td>Yes</td>
 <td style="text-align: center;">140 / 150</td>
-<td style="text-align: center;">2.01</td>
-<td><a href="/analysis_gold_reference_simpy">Analysis</a></td>
+<td style="text-align: center;">2.20</td>
+<td><a href="/analysis_commit0_reference_simpy">Analysis</a></td>
 <td><a href="https://github.com/commit-0/simpy">Github</a></td>
 </tr>
 <tr>
 <td>tinydb</td>
 <td>Yes</td>
 <td style="text-align: center;">201 / 201</td>
-<td style="text-align: center;">1.19</td>
-<td><a href="/analysis_gold_reference_tinydb">Analysis</a></td>
+<td style="text-align: center;">2.67</td>
+<td><a href="/analysis_commit0_reference_tinydb">Analysis</a></td>
 <td><a href="https://github.com/commit-0/tinydb">Github</a></td>
 </tr>
 <tr>
 <td>marshmallow</td>
 <td>Yes</td>
 <td style="text-align: center;">1229 / 1229</td>
-<td style="text-align: center;">2.57</td>
-<td><a href="/analysis_gold_reference_marshmallow">Analysis</a></td>
+<td style="text-align: center;">6.77</td>
+<td><a href="/analysis_commit0_reference_marshmallow">Analysis</a></td>
 <td><a href="https://github.com/commit-0/marshmallow">Github</a></td>
 </tr>
 <tr>
 <td>wcwidth</td>
 <td>Yes</td>
 <td style="text-align: center;">38 / 39</td>
-<td style="text-align: center;">1.00</td>
-<td><a href="/analysis_gold_reference_wcwidth">Analysis</a></td>
+<td style="text-align: center;">1.04</td>
+<td><a href="/analysis_commit0_reference_wcwidth">Analysis</a></td>
 <td><a href="https://github.com/commit-0/wcwidth">Github</a></td>
 </tr>
 <tr>
 <td>imapclient</td>
 <td>Yes</td>
 <td style="text-align: center;">267 / 267</td>
-<td style="text-align: center;">1.45</td>
-<td><a href="/analysis_gold_reference_imapclient">Analysis</a></td>
+<td style="text-align: center;">0.50</td>
+<td><a href="/analysis_commit0_reference_imapclient">Analysis</a></td>
 <td><a href="https://github.com/commit-0/imapclient">Github</a></td>
 </tr>
 <tr>
 <td>voluptuous</td>
 <td>Yes</td>
 <td style="text-align: center;">149 / 149</td>
-<td style="text-align: center;">0.37</td>
-<td><a href="/analysis_gold_reference_voluptuous">Analysis</a></td>
+<td style="text-align: center;">0.42</td>
+<td><a href="/analysis_commit0_reference_voluptuous">Analysis</a></td>
 <td><a href="https://github.com/commit-0/voluptuous">Github</a></td>
 </tr>
 <tr>
 <td>jinja</td>
 <td>Yes</td>
 <td style="text-align: center;">851 / 851</td>
-<td style="text-align: center;">4.14</td>
-<td><a href="/analysis_gold_reference_jinja">Analysis</a></td>
+<td style="text-align: center;">6.16</td>
+<td><a href="/analysis_commit0_reference_jinja">Analysis</a></td>
 <td><a href="https://github.com/commit-0/jinja">Github</a></td>
 </tr>
 <tr>
 <td>deprecated</td>
 <td>Yes</td>
 <td style="text-align: center;">171 / 171</td>
-<td style="text-align: center;">1.15</td>
-<td><a href="/analysis_gold_reference_deprecated">Analysis</a></td>
+<td style="text-align: center;">0.46</td>
+<td><a href="/analysis_commit0_reference_deprecated">Analysis</a></td>
 <td><a href="https://github.com/commit-0/deprecated">Github</a></td>
 </tr>
 <tr>
 <td>cookiecutter</td>
 <td>Yes</td>
 <td style="text-align: center;">367 / 371</td>
-<td style="text-align: center;">6.72</td>
-<td><a href="/analysis_gold_reference_cookiecutter">Analysis</a></td>
+<td style="text-align: center;">7.27</td>
+<td><a href="/analysis_commit0_reference_cookiecutter">Analysis</a></td>
 <td><a href="https://github.com/commit-0/cookiecutter">Github</a></td>
 </tr>
 <tr>
 <td>cachetools</td>
 <td>Yes</td>
 <td style="text-align: center;">215 / 215</td>
-<td style="text-align: center;">0.35</td>
-<td><a href="/analysis_gold_reference_cachetools">Analysis</a></td>
+<td style="text-align: center;">0.34</td>
+<td><a href="/analysis_commit0_reference_cachetools">Analysis</a></td>
 <td><a href="https://github.com/commit-0/cachetools">Github</a></td>
 </tr>
 </tbody>