Skip to content
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
[submodule "ai-core/subprojects/llama.cpp"]
path = ai-core/subprojects/llama.cpp
[submodule "ai-agent-local/subprojects/llama.cpp"]
path = ai-agent-local/subprojects/llama.cpp
url = https://github.com/appdevforall/llama.cpp.git
branch = androidide-custom
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,9 @@ See the official [plugin documentation](https://www.appdevforall.org/codeonthego
| [`compose-preview/`](compose-preview/) | Renders Jetpack Compose `@Preview` functions on-device — no full app build or run. |
| [`ai-literacy-course/`](ai-literacy-course/) | Bundles Learn AI Anywhere's offline "Introduction to AI" course (26 videos + interactive activities) and plays it full-screen, fully offline. |
| [`layout-editor/`](layout-editor/) | Visual drag-and-drop editor for Android XML layouts. |
| [`ai-core/`](ai-core/) | Shared on-device LLM inference backend (bundled llama.cpp AAR) plus a Gemini API backend, exposed to other plugins as a runtime service. |
| [`ai-assistant/`](ai-assistant/) | In-IDE AI chat assistant with tool calling; talks to `ai-core` for inference over local or Gemini models. |
| [`ai-core/`](ai-core/) | The **Agent** chat (tool-calling assistant) plus the shared LLM inference **router** other plugins consume. Ships no model — install at least one backend plugin below. Mandatory for every AI feature. |
| [`ai-agent-local/`](ai-agent-local/) | On-device `.gguf` inference backend for `ai-core` (bundled llama.cpp AAR). Registers as `local`; needs no network. |
| [`ai-agent-gemini/`](ai-agent-gemini/) | Google Gemini API inference backend for `ai-core`. Registers as `gemini`; needs an API key and network access. |
| [`flutter-template/`](flutter-template/) | Adds Flutter starter project templates (Basic, BLoC, Provider, GetX, Riverpod) to the New Project screen. |
| [`code-suggestions-plugin/`](code-suggestions-plugin/) | Inline ghost-text code completions powered by AI. |
| [`speech-to-text-plugin/`](speech-to-text-plugin/) | Voice-to-code: converts speech to code with AI generation. |
Expand Down
File renamed without changes.
69 changes: 69 additions & 0 deletions ai-agent-gemini/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# AI Agent Gemini plugin for CodeOnTheGo

Google Gemini API inference for CodeOnTheGo's AI plugins. Registers itself as the
`gemini` backend with [`ai-core`](../ai-core/)'s `LlmInferenceService`, which is
what `ai-core`'s Agent chat, `code-suggestions-plugin`, `speech-to-text-plugin` and
`vector-search-plugin` actually talk to.

Calls the Generative Language REST API directly over `HttpURLConnection` rather
than the google-genai SDK: the SDK bundles OkHttp 4.x, but plugins run in the
host IDE's classloader where `okhttp3` resolves to the host's older OkHttp, and
that mismatch crashed generation with a `NoSuchMethodError`.

## Building

Prerequisites: Android SDK (API 33+), JDK 17. Create `local.properties` with
`sdk.dir=...`. This plugin uses the shared wrapper at the repo root:

```bash
cd ai-agent-gemini
../gradlew assemblePlugin # release -> build/plugin/ai-agent-gemini.cgp
../gradlew assemblePluginDebug # debug variant
```

## API key handling

The key is entered in **AI Core → Agent settings**, not here. It is stored
encrypted (AES/GCM under a hardware-backed Android Keystore secret) and sent as
an `x-goog-api-key` **header**, never in a URL query string.

`security/SecureApiKeyStore.kt` is the only copy of the crypto — this plugin owns
both the write and the read, so there are no constants to keep in sync with
another plugin. A key written under an earlier plugin id is adopted once by
`preferences/GeminiPreferences.kt` and re-encrypted here.

## Installation

Install **`ai-core` as well** — without the router this plugin has nothing to
register with. Order does not matter: this plugin re-registers when it sees
ai-core activate. Copy `build/plugin/ai-agent-gemini.cgp` to the device, install
via CodeOnTheGo's Plugin Manager, then restart the IDE.

## Cross-plugin contract

This plugin's own settings pane calls `GeminiBackend.listModels()` and
`listModels(String)` directly (see `BackendGeminiCatalogGateway` in
`settings/GeminiCatalogGateway.kt`) to populate the model picker and to verify a
key before it is saved. Those two signatures, and the `ListModels HTTP <code>`
message shape thrown by `fetchAvailableModels`, are a contract — the pane is
mounted by ai-core across the plugin classloader boundary, so
`proguard-rules.pro` pins the class and its public methods.

## Key classes

Every source file sits in a package named for its layer; nothing is loose at the
root of `com/itsaky/androidide/plugins/aiagentgemini/`.

- `plugin/GeminiPlugin.kt` — plugin entry point; registers the backend with ai-core
- `backend/GeminiBackend.kt` — the REST transport, streaming (SSE) and model catalog
- `errors/GeminiErrorFormatter.kt` — turns an API failure into one translated sentence
- `security/SecureApiKeyStore.kt` — AES/GCM at rest
- `preferences/GeminiPreferences.kt` — this plugin's settings store, plus the
one-time adoption of settings written under earlier plugin ids
- `prompt/GeminiSystemPrompt.kt` — the system prompt this cloud model is given
- `logging/` — `LOG_PREFIX` (`AiAgentGemini`), prefixing every logcat tag this plugin writes
- `settings/` — the settings pane this backend contributes to the selector

## License

GPL-3.0 — same as AndroidIDE / CodeOnTheGo.
126 changes: 126 additions & 0 deletions ai-agent-gemini/ai-agent-gemini.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>AI Agent Gemini Plugin</title>
<style>
body {
background: #ffffff;
color: #000000;
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
line-height: 1.6;
margin: 0;
padding: 1.25rem;
max-width: 820px;
}
h1 { font-size: 1.6rem; margin: 0 0 0.75rem; }
h2 { font-size: 1.2rem; margin: 1.75rem 0 0.5rem; border-bottom: 1px solid #e0e0e0; padding-bottom: 0.2rem; }
code {
background: #f4f4f4;
padding: 0.1rem 0.3rem;
border-radius: 3px;
font-family: "SF Mono", Menlo, Consolas, monospace;
font-size: 0.9em;
}
ul { padding-left: 1.25rem; }
li { margin: 0.3rem 0; }
table { border-collapse: collapse; margin: 0.5rem 0; width: 100%; }
th, td { border: 1px solid #ddd; padding: 0.4rem 0.6rem; text-align: left; vertical-align: top; }
th { background: #f4f4f4; }
.note {
border-left: 3px solid #999;
padding: 0.25rem 0.75rem;
margin: 1rem 0;
background: #fafafa;
}
</style>
</head>
<body>
<h1>AI Agent Gemini Plugin</h1>

<h2>Executive overview</h2>
<p><b>AI Agent Gemini</b> adds Google's <b>Gemini</b> models to CodeOnTheGo's
AI features. It registers itself as the <code>gemini</code> inference backend
with <b>AI Core</b>, which routes requests from the <b>Agent</b> chat, <b>Code
Suggestions</b>, <b>Speech to Text</b> and <b>Vector Search</b>. It also
contributes <b>its own settings pane</b> — the API key field and the model
picker — which AI Core mounts inside Agent settings.</p>
<p>Install <b>AI Core</b> alongside it — without the router this plugin has
nothing to register with. Install order does not matter.</p>
<p>Using this backend sends prompts, and any file contents a plugin includes in
them, to Google over HTTPS. Choose <b>AI Agent Local</b> instead if inference
must stay on the device.</p>

<h2>Core functionality</h2>
<ul>
<li><b>Cloud inference</b> via direct calls to the Gemini REST API, with text
completion, chat with history, and server-sent-events streaming.</li>
<li><b>Live model catalog</b> — lists the models the saved key can actually
use, so a retired model name is never offered.</li>
<li><b>Key verification before saving</b> — a newly entered key is checked
against the catalog, distinguishing a refused key from an unreachable
network.</li>
<li><b>Encrypted at rest</b> — the API key is stored as AES/GCM ciphertext
under a hardware-backed Android Keystore secret, and a pre-existing
plaintext key is upgraded in place on first read.</li>
<li><b>Translated, safe error messages</b> — an API failure becomes one
user-facing sentence; the raw HTTP error body stays in the log and never
reaches the chat transcript.</li>
</ul>

<h2>Technical architecture</h2>
<table>
<tr><th>Component</th><th>Role</th></tr>
<tr><td><code>GeminiPlugin</code></td><td>Plugin entry point. Registers the
backend with AI Core on activation, re-registering if AI Core activates
later; cancels in-flight requests and drops the decrypted key on
dispose.</td></tr>
<tr><td><code>GeminiBackend</code></td><td>The transport. Calls the
Generative Language REST API over <code>HttpURLConnection</code>, parses the
streaming response, and fetches the model catalog.</td></tr>
<tr><td><code>GeminiErrorFormatter</code></td><td>Classifies a failure
(retired model, quota, refused key, outage, unreachable) so it can be
reported as one translated sentence.</td></tr>
<tr><td><code>SecureApiKeyStore</code></td><td>AES/GCM encryption of the API
key under a hardware-backed Android Keystore secret owned by this
plugin.</td></tr>
<tr><td><code>GeminiSettingsFragment</code></td><td>The settings pane AI Core
mounts: key entry and verification, visibility toggle, and the model picker
driven by the live catalog.</td></tr>
</table>
<p><b>No third-party HTTP SDK.</b> The google-genai SDK bundles OkHttp 4.x, but
plugins run in the host IDE's classloader where <code>okhttp3</code> resolves to
the host's older OkHttp — a mismatch that crashed generation with a
<code>NoSuchMethodError</code>. <code>HttpURLConnection</code> has no
third-party dependency and works regardless of the host's OkHttp version.</p>

<h2>Usage</h2>
<ol>
<li>Obtain a Gemini API key from Google AI Studio.</li>
<li>Install <b>AI Core</b> and <b>AI Agent Gemini</b> via the Plugin
Manager, then restart the IDE.</li>
<li>Open <b>Preferences &rarr; Configuration &rarr; Agent</b> and select the
<b>gemini</b> backend. This plugin's own pane appears below it.</li>
<li>Enter the key and tap <b>Save</b> — it is verified against the API before
it is stored — then pick a model from the live list.</li>
</ol>
<div class="note">
The key is sent as an <code>x-goog-api-key</code> request header, never in a
URL query string — query strings leak into logs, proxies and crash reports.
</div>

<h2>Key benefits</h2>
<ul>
<li><b>High capability</b> — frontier-class models on a device that could not
run them locally.</li>
<li><b>Small footprint</b> — no bundled model or native library.</li>
<li><b>Credential hygiene</b> — encrypted at rest, header-only in transit,
dropped from memory when the plugin unloads.</li>
<li><b>No dead model names</b> — the picker reflects the live catalog for your
specific key.</li>
<li><b>Coexists with the local backend</b> — install both and switch in AI
Settings.</li>
</ul>
</body>
</html>
52 changes: 21 additions & 31 deletions ai-assistant/build.gradle.kts → ai-agent-gemini/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -5,29 +5,23 @@ plugins {
}

pluginBuilder {
pluginName = "ai-assistant"
pluginName = "ai-agent-gemini"
}

android {
namespace = "com.itsaky.androidide.plugins.aiassistant"
namespace = "com.itsaky.androidide.plugins.aiagentgemini"
compileSdk = 36

defaultConfig {
applicationId = "com.itsaky.androidide.plugins.aiassistant"
applicationId = "com.itsaky.androidide.plugins.aiagentgemini"
minSdk = 33
targetSdk = 36
versionCode = 2
versionName = "1.1.0"
}

buildFeatures {
viewBinding = true
buildConfig = true
versionCode = 1
versionName = "1.0.0"
}

buildTypes {
release {
// Disable minification to avoid lambda obfuscation issues with ClassLoader isolation
isMinifyEnabled = false
isShrinkResources = false
signingConfig = signingConfigs.getByName("debug")
Expand All @@ -46,18 +40,15 @@ android {
}
}

testOptions {
unitTests.isReturnDefaultValues = true
}

packaging {
resources {
excludes += setOf(
"META-INF/DEPENDENCIES",
"META-INF/LICENSE",
"META-INF/LICENSE.txt",
"META-INF/NOTICE",
"META-INF/NOTICE.txt"
"META-INF/NOTICE.txt",
"META-INF/INDEX.LIST"
)
}
}
Expand All @@ -66,26 +57,25 @@ android {
dependencies {
compileOnly(files("../libs/plugin-api.jar"))

// Use 'implementation' (not 'compileOnly') for androidx libraries.
// This is required for XML layouts: AAPT2 needs these dependencies at compile-time to process
// resource attributes and resolve xmlns declarations. This is standard across all CoGo plugins
// with XML layouts (random-xkcd, sketch-to-ui-plugin, Beepy). See investigation in Task 4.
// 'implementation' (not 'compileOnly') for the androidx/Material libraries: AAPT2 needs them
// at compile time to process the settings pane's layout, as in every CoGo plugin with XML.
implementation("androidx.appcompat:appcompat:1.6.1")
implementation("androidx.fragment:fragment-ktx:1.8.8")
implementation("com.google.android.material:material:1.10.0")
implementation("androidx.recyclerview:recyclerview:1.3.2")
implementation("androidx.constraintlayout:constraintlayout:2.1.4")

// Markdown rendering - plugin-specific library
implementation("io.noties.markwon:core:4.6.2")

// JSON serialization for session persistence
implementation("com.google.code.gson:gson:2.10.1")
implementation("org.jetbrains.kotlin:kotlin-stdlib:2.3.0")
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.7.3")

testImplementation(files("../libs/plugin-api.jar"))
testImplementation("junit:junit:4.13.2")
testImplementation("io.mockk:mockk:1.13.8")
testImplementation("org.json:json:20240303")
testImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:1.8.1")
testImplementation("androidx.arch.core:core-testing:2.2.0")
testImplementation("org.json:json:20231013")
}

// SecureApiKeyStore is no longer duplicated: this plugin holds the only copy, so there is nothing
// left to drift against. The parity check that guarded the ai-assistant copy went with that plugin.

// AAR metadata checks are disabled by convention for these application-as-library plugins.
tasks.matching {
it.name.contains("checkDebugAarMetadata") ||
it.name.contains("checkReleaseAarMetadata")
}.configureEach { enabled = false }
File renamed without changes.
21 changes: 21 additions & 0 deletions ai-agent-gemini/proguard-rules.pro
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# AI Agent Gemini Plugin ProGuard Rules

# Keep plugin entry point
-keep public class com.itsaky.androidide.plugins.aiagentgemini.plugin.GeminiPlugin {
public <methods>;
}

# Keep the backend: AI Core's settings pane calls listModels across the plugin
# classloader boundary, because listModels is not on LlmBackend. Renaming or
# stripping it breaks the model picker and key verification silently.
-keep public class com.itsaky.androidide.plugins.aiagentgemini.backend.GeminiBackend {
public <methods>;
}

# Keep the settings fragment: it is instantiated by name from getSettingsFragmentClassName().
-keep public class com.itsaky.androidide.plugins.aiagentgemini.settings.GeminiSettingsFragment {
public <init>(...);
}

# Keep plugin-api interfaces
-keep interface com.itsaky.androidide.plugins.** { *; }
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ dependencyResolutionManagement {
}
}

rootProject.name = "ai-assistant"
rootProject.name = "ai-agent-gemini"
Loading
Loading