-
-
Notifications
You must be signed in to change notification settings - Fork 778
Feat: Lightpanda extension #2192
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
nicktrn
merged 28 commits into
triggerdotdev:main
from
nrigaudiere:feat/lightpanda-extension
Jul 28, 2025
Merged
Changes from all commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
b3c5b9b
feat: add lightpanda structure
nrigaudiere 85198b8
chore: add lightpanda doc links
nrigaudiere 29dbf66
fix: lightpanda extension instructions
nrigaudiere 33ffbcf
feat: add Lightpanda guide and examples
nrigaudiere d5505b5
feat: lightpanda - add 3rd example
nrigaudiere f43f8af
feat: add lightpandaTask
nrigaudiere 11efe5c
fix: lightpanda 3rd example
nrigaudiere 806aeff
fix: lightpanda 1st example
nrigaudiere 76d936c
chore: add changeset
nrigaudiere 72225e7
add v4 tag to guide
nicktrn fcc50a6
fix: merge lightpanda docker instructions
nrigaudiere 60f3357
fix: add failsafes
nrigaudiere 281de09
add scrape warning
nicktrn 5aec497
lint
nicktrn 2568391
Merge remote-tracking branch 'origin/main' into pr/2192
nicktrn 32e4175
successful login also switches to that profile
nicktrn b272e30
Merge remote-tracking branch 'origin/main' into pr/2192
nicktrn 73379ce
Merge remote-tracking branch 'origin/main' into pr/2192
nicktrn c9511b1
update docs and links as this is v4 only
nicktrn 7011482
extension tweaks
nicktrn 636d71b
Merge remote-tracking branch 'origin/main' into pr/2192
nicktrn 54585cb
simplify extension
nicktrn 067f38a
update examples
nicktrn 3464e5e
update docs
nicktrn c7cb791
Merge remote-tracking branch 'origin/main' into pr/2192
nicktrn b11d9b7
remove from extensions list as v4 only
nicktrn f181664
remove from catalog
nicktrn 0a0c06a
update changeset
nicktrn File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "trigger.dev": patch | ||
| --- | ||
|
|
||
| Switch to profile after successful login |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@trigger.dev/build": patch | ||
| --- | ||
|
|
||
| Add Lightpanda extension |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| --- | ||
| title: "Lightpanda" | ||
| sidebarTitle: "lightpanda" | ||
| description: "Use the lightpanda build extension to add Lightpanda browser to your project" | ||
| tag: "v4" | ||
| --- | ||
|
|
||
| import UpgradeToV4Note from "/snippets/upgrade-to-v4-note.mdx"; | ||
|
|
||
| <UpgradeToV4Note /> | ||
|
|
||
| To use the Lightpanda browser in your project, add the extension to your `trigger.config.ts` file: | ||
|
|
||
| ```ts trigger.config.ts | ||
| import { defineConfig } from "@trigger.dev/sdk"; | ||
| import { lightpanda } from "@trigger.dev/build/extensions/lightpanda"; | ||
|
|
||
| export default defineConfig({ | ||
| project: "<project ref>", | ||
| build: { | ||
| extensions: [lightpanda()], | ||
| }, | ||
| }); | ||
| ``` | ||
|
|
||
| ## Options | ||
|
|
||
| - `version`: The version of the browser to install. Default: `"latest"`. | ||
| - `disableTelemetry`: Whether to disable telemetry. Default: `false`. | ||
|
|
||
| For example: | ||
|
|
||
| ```ts trigger.config.ts | ||
| import { defineConfig } from "@trigger.dev/sdk"; | ||
| import { lightpanda } from "@trigger.dev/build/extensions/lightpanda"; | ||
|
|
||
| export default defineConfig({ | ||
| project: "<project ref>", | ||
| build: { | ||
| extensions: [ | ||
| lightpanda({ | ||
| version: "nightly", | ||
| disableTelemetry: true, | ||
| }), | ||
| ], | ||
| }, | ||
| }); | ||
| ``` | ||
|
|
||
| ## Development | ||
|
|
||
| When running in dev, you will first have to download the Lightpanda browser binary and make sure it's in your `PATH`. See [Lightpanda's installation guide](https://lightpanda.io/docs/getting-started/installation). | ||
|
|
||
| ## Next steps | ||
|
|
||
| <CardGroup> | ||
| <Card title="Lightpanda" color="#6ac6e2" icon="bolt" href="/guides/examples/lightpanda"> | ||
| Learn how to use Lightpanda in your project. | ||
| </Card> | ||
| </CardGroup> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,227 @@ | ||
| --- | ||
| title: "Lightpanda" | ||
| sidebarTitle: "Lightpanda" | ||
| description: "These examples demonstrate how to use Lightpanda with Trigger.dev." | ||
| tag: "v4" | ||
| --- | ||
|
|
||
| import ScrapingWarning from "/snippets/web-scraping-warning.mdx"; | ||
| import UpgradeToV4Note from "/snippets/upgrade-to-v4-note.mdx"; | ||
|
|
||
| <UpgradeToV4Note /> | ||
|
|
||
| ## Overview | ||
|
|
||
| Lightpanda is a purpose-built browser for AI and automation workflows. It is 10x faster, uses 10x less RAM than Chrome headless. | ||
|
|
||
| Here are a few examples of how to use Lightpanda with Trigger.dev. | ||
|
|
||
| <ScrapingWarning /> | ||
|
|
||
| ## Limitations | ||
|
|
||
| - Lightpanda does not support the `puppeteer` screenshot feature. | ||
|
|
||
| ## Using Lightpanda Cloud | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| - A [Lightpanda](https://lightpanda.io/) cloud token | ||
|
|
||
| ### Get links from a website | ||
| In this task we use Lightpanda browser to get links from a provided URL. You will have to pass the URL as a payload when triggering the task. | ||
|
|
||
| Make sure to add `LIGHTPANDA_TOKEN` to your Trigger.dev dashboard on the Environment Variables page: | ||
| ```bash | ||
| LIGHTPANDA_TOKEN="<your-token>" | ||
| ``` | ||
|
|
||
| ```ts trigger/lightpanda-cloud-puppeteer.ts | ||
| import { logger, task } from "@trigger.dev/sdk"; | ||
| import puppeteer from "puppeteer-core"; | ||
|
|
||
| export const lightpandaCloudPuppeteer = task({ | ||
| id: "lightpanda-cloud-puppeteer", | ||
| machine: { | ||
| preset: "micro", | ||
| }, | ||
| run: async (payload: { url: string }, { ctx }) => { | ||
| logger.log("Lets get a page's links with Lightpanda!", { payload, ctx }); | ||
|
|
||
| if (!payload.url) { | ||
| logger.warn("Please define the payload url"); | ||
| throw new Error("payload.url is undefined"); | ||
| } | ||
|
|
||
| const token = process.env.LIGHTPANDA_TOKEN; | ||
| if (!token) { | ||
| logger.warn("Please define the env variable LIGHTPANDA_TOKEN"); | ||
| throw new Error("LIGHTPANDA_TOKEN is undefined"); | ||
| } | ||
|
|
||
| // Connect to Lightpanda's cloud | ||
| const browser = await puppeteer.connect({ | ||
| browserWSEndpoint: `wss://cloud.lightpanda.io/ws?browser=lightpanda&token=${token}`, | ||
| }); | ||
| const context = await browser.createBrowserContext(); | ||
| const page = await context.newPage(); | ||
|
|
||
| // Dump all the links from the page. | ||
| await page.goto(payload.url); | ||
|
|
||
| const links = await page.evaluate(() => { | ||
| return Array.from(document.querySelectorAll("a")).map((row) => { | ||
| return row.getAttribute("href"); | ||
| }); | ||
| }); | ||
|
|
||
| logger.info("Processing done, shutting down…"); | ||
|
|
||
| await page.close(); | ||
| await context.close(); | ||
| await browser.disconnect(); | ||
|
|
||
| logger.info("✅ Completed"); | ||
|
|
||
| return { | ||
| links, | ||
| }; | ||
| }, | ||
| }); | ||
| ``` | ||
|
|
||
| ### Proxies | ||
|
|
||
| Proxies can be used with your browser via the proxy query string parameter. By default, the proxy used is "datacenter" which is a pool of shared datacenter IPs. | ||
| `datacenter` accepts an optional `country` query string parameter which is an [ISO 3166-1 alpha-2](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) country code. | ||
|
|
||
| ```bash | ||
| # This example will use a German IP | ||
| wss://cloud.lightpanda.io/ws?proxy=datacenter&country=de&token=${token} | ||
| ``` | ||
|
|
||
| ### Session | ||
|
|
||
| A session is alive until you close it or the connection is closed. The max duration of a session is 15 minutes. | ||
|
|
||
| ## Using Lightpanda browser directly | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| - Setup the [Lightpanda build extension](/config/extensions/lightpanda) | ||
|
|
||
| ### Get the HTML of a webpage | ||
|
|
||
| This task will dump the HTML of a provided URL using the Lightpanda browser binary. You will have to pass the URL as a payload when triggering the task. | ||
|
|
||
| ```ts trigger/lightpanda-fetch.ts | ||
| import { logger, task } from "@trigger.dev/sdk"; | ||
| import { execSync } from "node:child_process"; | ||
|
|
||
| export const lightpandaFetch = task({ | ||
| id: "lightpanda-fetch", | ||
| machine: { | ||
| preset: "micro", | ||
| }, | ||
| run: async (payload: { url: string }, { ctx }) => { | ||
| logger.log("Lets get a page's content with Lightpanda!", { payload, ctx }); | ||
|
|
||
| if (!payload.url) { | ||
| logger.warn("Please define the payload url"); | ||
| throw new Error("payload.url is undefined"); | ||
| } | ||
|
|
||
| const buffer = execSync(`lightpanda fetch --dump ${payload.url}`); | ||
|
|
||
| logger.info("✅ Completed"); | ||
|
|
||
| return { | ||
| message: buffer.toString(), | ||
| }; | ||
| }, | ||
| }); | ||
| ``` | ||
|
|
||
| ### Lightpanda CDP with Puppeteer | ||
|
|
||
| This task initializes a Lightpanda CDP server and uses it with `puppeteer-core` to scrape a provided URL. | ||
|
|
||
| ```ts trigger/lightpanda-cdp.ts | ||
| import { logger, task } from "@trigger.dev/sdk"; | ||
| import { spawn, type ChildProcessWithoutNullStreams } from "node:child_process"; | ||
| import puppeteer from "puppeteer-core"; | ||
|
|
||
| const spawnLightpanda = async (host: string, port: string) => | ||
| new Promise<ChildProcessWithoutNullStreams>((resolve, reject) => { | ||
| const child = spawn("lightpanda", [ | ||
| "serve", | ||
| "--host", | ||
| host, | ||
| "--port", | ||
| port, | ||
| "--log_level", | ||
| "info", | ||
| ]); | ||
|
|
||
| child.on("spawn", async () => { | ||
| logger.info("Running Lightpanda's CDP server…", { | ||
| pid: child.pid, | ||
| }); | ||
|
|
||
| await new Promise((resolve) => setTimeout(resolve, 250)); | ||
| resolve(child); | ||
| }); | ||
| child.on("error", (e) => reject(e)); | ||
| }); | ||
|
|
||
| export const lightpandaCDP = task({ | ||
| id: "lightpanda-cdp", | ||
| machine: { | ||
| preset: "micro", | ||
| }, | ||
| run: async (payload: { url: string }, { ctx }) => { | ||
| logger.log("Lets get a page's links with Lightpanda!", { payload, ctx }); | ||
|
|
||
| if (!payload.url) { | ||
| logger.warn("Please define the payload url"); | ||
| throw new Error("payload.url is undefined"); | ||
| } | ||
|
|
||
| const host = process.env.LIGHTPANDA_CDP_HOST ?? "127.0.0.1"; | ||
| const port = process.env.LIGHTPANDA_CDP_PORT ?? "9222"; | ||
|
|
||
| // Launch Lightpanda's CDP server | ||
| const lpProcess = await spawnLightpanda(host, port); | ||
|
|
||
| const browser = await puppeteer.connect({ | ||
| browserWSEndpoint: `ws://${host}:${port}`, | ||
| }); | ||
| const context = await browser.createBrowserContext(); | ||
| const page = await context.newPage(); | ||
|
|
||
| // Dump all the links from the page. | ||
| await page.goto(payload.url); | ||
|
|
||
| const links = await page.evaluate(() => { | ||
| return Array.from(document.querySelectorAll("a")).map((row) => { | ||
| return row.getAttribute("href"); | ||
| }); | ||
| }); | ||
|
|
||
| logger.info("Processing done"); | ||
| logger.info("Shutting down…"); | ||
|
|
||
| // Close Puppeteer instance | ||
| await browser.close(); | ||
|
|
||
| // Stop Lightpanda's CDP Server | ||
| lpProcess.kill(); | ||
|
|
||
| logger.info("✅ Completed"); | ||
|
|
||
| return { | ||
| links, | ||
| }; | ||
| }, | ||
| }); | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.