Allow users to disable scrape, let the model continue even if scrape …

[fccview]

…fails and overall harden prompts

Summary by CodeRabbit

• New Features

  • Added ability to disable the scrape tool via DEGOOG_MCP_DISABLE_SCRAPE; when enabled, only the search tool is exposed.

• Improvements

  • Sanitized and standardized scrape error messages for clearer, more actionable output.
  • Updated search and scrape tool descriptions with stronger guidance on continuing from available results and reporting failed URLs appropriately.

• Documentation

  • Updated the README “Run” configuration table to include DEGOOG_MCP_DISABLE_SCRAPE.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 6d7349df-d720-4fe5-b646-4d9dd25aef20

📥 Commits

Reviewing files that changed from the base of the PR and between f517bf7 and 2405191.

📒 Files selected for processing (2)

• commands/handler_test.go
• tools/search.go

🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

• degoog-org/docs (manual)
• degoog-org/degoog (auto-detected)

🚧 Files skipped from review as they are similar to previous changes (2)

• tools/search.go
• commands/handler_test.go

---

📝 Walkthrough

Walkthrough

This PR adds a DEGOOG_MCP_DISABLE_SCRAPE environment variable that, when set to true, prevents the scrape tool from being registered while keeping the search tool available. The Config struct gains a DisableScrape boolean field populated by a new readBool helper that handles common truthy/falsey string values. Register now returns the list of registered tool names and conditionally omits scrape tool registration. SearchTool gains an optional scrapeEnabled parameter to switch between two description constants (SEARCH_DESC / SEARCH_DESC_NO_SCRAPE). Both tool descriptions are updated with agent guidance against inventing URLs and instructions for handling failures. Scrape error output is normalized through a new cleanScrapeError function that rewrites Docker DNS failure messages into a simplified form.

Possibly Related PRs

• degoog-org/mcp#1: Modifies commands/scrape.go's scrapeSummary and handler behavior, which are also changed in this PR for error sanitization and guidance text.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main objectives of the PR: allowing users to disable scrape functionality and enabling the model to continue even if scrape fails.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch develop

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

tests/config_test.go (1)

110-119: ⚡ Quick win

Expand bool parsing coverage to include falsey and invalid inputs.

Current table coverage only validates truthy inputs; readBool also has explicit falsey and invalid-fallback branches that should be locked down to prevent regressions.

♻️ Suggested test expansion

 func TestDisableScrapeBoolParsing(t *testing.T) {
-	for _, value := range []string{"1", "true", "TRUE", "yes", "on"} {
-		t.Run(value, func(t *testing.T) {
-			t.Setenv(config.ENV_DISABLE_SCRAPE, value)
-			if !config.Load().DisableScrape {
-				t.Fatalf("DisableScrape %q: want true", value)
-			}
-		})
-	}
+	cases := []struct {
+		name  string
+		value string
+		want  bool
+	}{
+		{"truthy_1", "1", true},
+		{"truthy_true_upper", "TRUE", true},
+		{"truthy_yes", "yes", true},
+		{"falsey_0", "0", false},
+		{"falsey_false", "false", false},
+		{"falsey_off", "off", false},
+		{"invalid_fallback_default_false", "maybe", false},
+	}
+	for _, tc := range cases {
+		t.Run(tc.name, func(t *testing.T) {
+			t.Setenv(config.ENV_DISABLE_SCRAPE, tc.value)
+			if got := config.Load().DisableScrape; got != tc.want {
+				t.Fatalf("DisableScrape %q: want %v, got %v", tc.value, tc.want, got)
+			}
+		})
+	}
 }

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/config_test.go` around lines 110 - 119, The
TestDisableScrapeBoolParsing test currently only validates truthy input values
for the DisableScrape boolean parsing. Expand the test to include additional
subtests covering falsey inputs (such as "0", "false", "FALSE", "no", "off")
that should result in DisableScrape being false, and also add subtests for
invalid or unexpected input values to verify the fallback behavior of the
readBool function. This ensures all branches of the boolean parsing logic are
tested and protected against regressions.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tools/search.go`:
- Around line 24-26: The SEARCH_DESC_NO_SCRAPE constant currently appends to
SEARCH_DESC, which means it inherits scraping-related instructions that conflict
with the no-scrape mode where scraping is unavailable. Refactor
SEARCH_DESC_NO_SCRAPE to be a complete, standalone description that does not
append to or inherit from SEARCH_DESC, ensuring it contains only instructions
appropriate for search functionality without any scraping capabilities.

---

Nitpick comments:
In `@tests/config_test.go`:
- Around line 110-119: The TestDisableScrapeBoolParsing test currently only
validates truthy input values for the DisableScrape boolean parsing. Expand the
test to include additional subtests covering falsey inputs (such as "0",
"false", "FALSE", "no", "off") that should result in DisableScrape being false,
and also add subtests for invalid or unexpected input values to verify the
fallback behavior of the readBool function. This ensures all branches of the
boolean parsing logic are tested and protected against regressions.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 0b768550-58e9-4c26-abe1-be09996469b3

📥 Commits

Reviewing files that changed from the base of the PR and between 972799d and f517bf7.

📒 Files selected for processing (8)

• README.md
• commands/handler_test.go
• commands/register.go
• commands/scrape.go
• internal/config/config.go
• tests/config_test.go
• tools/scrape.go
• tools/search.go

🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

• degoog-org/docs (manual)
• degoog-org/degoog (auto-detected)