(function () { const translations = { meta: { htmlLang: 'en', pageTitle: 'Universal Web-to-API Guide' }, ui: { navTitle: 'π Navigation', pageHeaderTitle: 'Universal Web-to-API Guide (v2.8.9)', projectLinkLabel: 'Project Link', projectLinkDescription: 'Need the source code, release notes, or the issue tracker? Start here.', projectLinkButton: 'Open Repository', languageLabel: 'Language', themeLight: 'Light Mode', themeDark: 'Dark Mode', hamburgerOpenLabel: 'Open navigation menu', hamburgerCloseLabel: 'Close navigation menu', siteCardHint: 'Click to copy the URL, then open it in the controlled browser', siteCardCopied: 'URL copied. Paste it into the controlled browser', siteCardCopyFailed: 'Copy failed. Please copy it manually and open it in the controlled browser', siteCardEmpty: 'No site list was loaded. Use the current dashboard config as the source of truth.' }, nav: { 'quick-start': 'π Quick Start', 'dashboard-tour': 'π₯οΈ Dashboard Tour', 'add-site-guide': 'π Adding a Site', 'connect-api': 'π Connect API', 'function-calling': 'π§° Function Calling', 'tab-pool': 'ποΈ Tab Pool', 'presets': 'ποΈ Presets', 'selectors': 'π Selectors', 'extractors': 'π§© Extractors', 'image-extraction': 'ποΈ Multimodal Extraction', 'response-detection': 'π Response Detection', 'workflow': 'π¬ Workflow', 'file-paste': 'π File Attach', 'stealth-mode': 'π‘οΈ Low-Interference Mode', 'commands': 'β‘ Automation Commands', 'ai-recognition': 'π― AI Recognition', 'env-config': 'βοΈ Environment Settings', 'browser-config': 'π Browser Constants', 'config-manage': 'πΎ Config Management', 'faq': 'β FAQ', 'author-note': 'β οΈ Notes From the Author' }, sections: {} }; translations.sections['quick-start'] = `
Welcome to Web-to-API. This project connects browser-based AI chat sites to a local OpenAI-compatible interface for personal testing, workflow integration, and client-side orchestration.
Recommended reading order: read the main workflow of this page first, then come back to the author note at the end. That section explains the maintenance scope, support expectations, and the most effective feedback channels.
π Core reminder: after startup the script opens a controlled browser window automatically. Keep that browser window open, because it is the foundation of the service. The tutorial can stay open in your normal browser; when the script is about to run, the controlled browser should contain nothing except the target site.
start.bat, first tell the two browser windows apart: the controlled browser used by the script, and the regular browser that shows this tutorial or that you use every day.chrome_profile/. On a first launch it is usually a blank, signed-out browser profile. If you did not already have a browser open, the script may also open a separate browser window for this tutorial, so it can look like two browsers appeared at once.https://gemini.google.com/. Before the script starts handling requests, make sure that controlled browser contains only the target site, not search pages, tutorial pages, mail, or any unrelated tabs.Base URL, API Key, and model examples there.π‘ Want to reuse an existing login state? Treat that as an advanced option. For normal use, it is safer to keep using the project's own chrome_profile/ directory and keep your session isolated there.
β οΈ Only do profile-copy reuse if you understand browser user-data structure and session-data risk. If you really need it, only copy your own local browser profile into a separate directory. Do not share, publish, or mix someone else's login-state data. The detailed steps stay in the Browser Configuration section later in this page.
Dashboard entry:
Open DashboardDashboard URL: http://127.0.0.1:8199/. You can edit configs and inspect logs there.
The following sites already have built-in adaptation and can usually be used directly:
Tip: the site cards below now copy the URL to your clipboard instead of opening in the current tutorial browser. Paste the copied URL into the controlled browser.
Sites highlighted in the current docs: ChatGPT, DeepSeek, Gemini, Claude, Kimi, Qwen, Grok, Doubao, AI Studio, and Arena AI.
The site cards below refresh from the current runtime config after the page loads. If you changed a domain or startup URL locally, trust the cards and the dashboard over any older screenshots.
β οΈ About arena.ai: this site is highly sensitive to IP quality. A good IP can keep chatting for a long time, while a poor IP may get blocked immediately. The most important factors are not just IP purity, but also traffic history and sharing level. If your IP quality is weak, Cloudflare challenges can still appear even with network interception disabled. That behavior comes from the site itself, not from this project.
If your target site is not listed above, the system can analyze the page automatically with a helper AI. See the AI Recognition section below.
`; translations.sections['dashboard-tour'] = `The dashboard is not just a settings page. It is the visual control center of the whole project.
INFO / DEBUG log phrasing in settings.Suggested reading order: for first-time use, start with the site config page and the tab pool. When debugging failures, go to logs first. When adapting a new site, read Settings β AI Recognition.
πΈ Dashboard overview: static/tutorial-dashboard-overview.png
This screenshot works well as a visual map of the control panel.
There are two main paths: automatic recognition and manual configuration. For a first attempt, automatic recognition is usually the fastest path.
HELPER_API_KEY, HELPER_BASE_URL, and HELPER_MODEL.config/sites.json, the backend reads the page HTML, asks the helper AI to analyze it, and writes a generated preset into the site config.Key point: automatic recognition starts on the first real request to an unknown domain. The Add Site button only creates an empty shell.
chat.example.com.| Key | Purpose | Priority |
|---|---|---|
input_box | Chat input field | β |
send_btn | Send button | β |
result_container | Container that holds the AI reply | β |
new_chat_btn | New-chat button | Optional |
message_wrapper | Outer container for one message | Optional |
[
{ "action": "CLICK", "target": "new_chat_btn", "optional": true, "value": null },
{ "action": "WAIT", "target": "", "optional": false, "value": 0.5 },
{ "action": "FILL_INPUT", "target": "input_box", "optional": false, "value": null },
{ "action": "CLICK", "target": "send_btn", "optional": true, "value": null },
{ "action": "KEY_PRESS", "target": "Enter", "optional": true, "value": null },
{ "action": "STREAM_WAIT", "target": "result_container", "optional": false, "value": null }
]
input_boxsend_btnresult_containerThis project exposes an OpenAI-compatible API.
βοΈ Common client settings:
OpenAI, OpenAI Compatible, or Custom.http://127.0.0.1:8199/v1http://127.0.0.1:8199/url/gemini.com/v1http://127.0.0.1:8199/tab/1/v1http://127.0.0.1:8199/v1/chat/completionssk-local. If auth is enabled, it must match your configured auth token.web-api or gemini-web. The actual response source still depends on the site and preset you opened.β Test flow: once the site is open and logged in, enter the URL and provider settings in your client and you can start testing immediately.
| Need | Recommended endpoint | Notes |
|---|---|---|
| Let the system choose an idle tab | http://127.0.0.1:8199/v1/chat/completions |
Use this when you do not care which site tab handles the request |
| Always use Gemini tabs | http://127.0.0.1:8199/url/gemini.com/v1/chat/completions |
Matches one available Gemini-related tab |
| Always use a specific tab | http://127.0.0.1:8199/tab/2/v1/chat/completions |
Useful when that tab already has a dedicated role or preset |
| Use one site with a forced preset | http://127.0.0.1:8199/url/gemini.com/pro/v1/chat/completions |
Useful when one site has multiple modes such as chat, pro, music, or video, and also works well as a direct Base URL |
The chat endpoints now support preset_name directly. This does not change the site's default preset. It only affects the current request.
Option A: put it directly in the path
curl "http://127.0.0.1:8199/url/gemini.com/pro/v1/chat/completions" ^
-H "Content-Type: application/json" ^
-d "{\"model\":\"any\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello\"}],\"stream\":false}"
Option B: put it in the URL query
curl "http://127.0.0.1:8199/url/gemini.com/v1/chat/completions?preset_name=pro" ^
-H "Content-Type: application/json" ^
-d "{\"model\":\"any\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello\"}],\"stream\":false}"
Option C: put it in the JSON body
{
"model": "any",
"messages": [
{ "role": "user", "content": "Hello" }
],
"stream": false,
"preset_name": "pro"
}
Priority rule: if all three are provided, the order is path preset > URL query > JSON body.
π‘ Practical advice: if your client lets you customize the Base URL, the cleanest choice is usually http://127.0.0.1:8199/url/gemini.com/pro/v1. If that is inconvenient, fall back to ?preset_name=pro or a JSON-body field.
| Method | Route | Purpose |
|---|---|---|
GET |
/v1/models |
Default model-list endpoint for OpenAI-compatible client checks |
POST |
/v1/chat/completions |
Default chat endpoint with automatic tab allocation |
GET |
/url/{domain}/v1/models |
Domain-routed model list, can use selector or tab_index |
POST |
/url/{domain}/v1/chat/completions |
Domain-routed chat, can use selector, tab_index, and preset_name |
GET |
/url/{domain}/{preset_name}/v1/models |
Domain + preset model-list route, useful when you want to use it directly as a Base URL |
POST |
/url/{domain}/{preset_name}/v1/chat/completions |
Domain + preset chat route, where the preset in the path has the highest priority |
GET |
/tab/{index}/v1/models |
Fixed-tab model-list endpoint |
POST |
/tab/{index}/v1/chat/completions |
Fixed-tab chat endpoint, can use preset_name |
Supported query parameters:
selector=first_idle: explicitly prefer an idle tab.selector=round_robin: rotate across matching site tabs.selector=random: randomly choose one matching site tab.selector is omitted on domain-routed endpoints, the server follows the current tab-pool allocation mode.tab_index=2: on a domain route, lock the request to one specific tab.preset_name=pro: force one preset for the current request only.β οΈ Route compatibility note: the server now supports putting the preset directly into the path, such as /url/gemini.com/pro/v1/chat/completions. If your client checks /models first, you can also use /url/gemini.com/pro/v1/models.
# 1. Default model list
curl http://127.0.0.1:8199/v1/models
# 2. Default chat endpoint
curl http://127.0.0.1:8199/v1/chat/completions ^
-H "Content-Type: application/json" ^
-d "{\"model\":\"any\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello\"}],\"stream\":false}"
# 3. Domain-routed model list
curl "http://127.0.0.1:8199/url/gemini.com/v1/models?selector=round_robin"
# 4. Domain-routed chat
curl "http://127.0.0.1:8199/url/gemini.com/v1/chat/completions?selector=first_idle" ^
-H "Content-Type: application/json" ^
-d "{\"model\":\"any\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello\"}],\"stream\":false}"
# 5. Domain route + fixed tab
curl "http://127.0.0.1:8199/url/gemini.com/v1/chat/completions?tab_index=2" ^
-H "Content-Type: application/json" ^
-d "{\"model\":\"any\",\"messages\":[{\"role\":\"user\",\"content\":\"Continue the previous turn\"}],\"stream\":false}"
# 6. Domain + preset model list
curl http://127.0.0.1:8199/url/gemini.com/pro/v1/models
# 7. Domain + preset chat
curl http://127.0.0.1:8199/url/gemini.com/pro/v1/chat/completions ^
-H "Content-Type: application/json" ^
-d "{\"model\":\"any\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello\"}],\"stream\":false}"
# 8. Fixed-tab model list
curl http://127.0.0.1:8199/tab/2/v1/models
# 9. Fixed-tab chat
curl http://127.0.0.1:8199/tab/2/v1/chat/completions ^
-H "Content-Type: application/json" ^
-d "{\"model\":\"any\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello\"}],\"stream\":false}"
# 10. Fixed-tab chat with a forced preset
curl "http://127.0.0.1:8199/tab/2/v1/chat/completions?preset_name=pro" ^
-H "Content-Type: application/json" ^
-d "{\"model\":\"any\",\"messages\":[{\"role\":\"user\",\"content\":\"Continue the previous turn\"}],\"stream\":false}"
β οΈ About SillyTavern: the built-in API test can be unreliable. It is better to send a real conversation instead of using the test button.
β οΈ Context-length note: many websites still limit how much text fits in a single input box. For very long input, enable File Attach so the site can process that content through its own attachment flow.
The project supports the OpenAI-style tools format and also the older functions / function_call fields, so most clients that already support Tool Calling can connect directly.
On the model-output side, the backend now prefers a project-local XML block format: <adapter_calls> / <call> / <arg>. The older XML tags <tool_calls> / <invoke> / <parameter> are still accepted for compatibility.
Unlike the earliest versions, this no longer has to fail in a single shot. The backend now includes internal repair retries after invalid tool-call output, and you can tune the strategy directly in Dashboard β Settings β Environment Settings β Function Calling.
What this is: a compatibility layer that lets you keep using familiar OpenAI-style tool definitions.
Preferred output shell: <adapter_calls> β <call name="..."> β <arg name="...">.
β οΈ Important boundary: this is not native API tool calling. The backend rewrites your tool definitions into prompt instructions, the website model tries to follow them, and the backend then parses the result back into OpenAI-style tool_calls. Reliability depends heavily on the model's own reasoning and formatting discipline.
| Setting | Where | What it controls |
|---|---|---|
| Retry strategy | Settings β Environment Settings β Function Calling | Whether the repair round sends only minimal correction feedback or the original conversation plus feedback. |
| Prompt padding obfuscation | Settings β Environment Settings β Function Calling | Randomizes the extra prefill and tail prompt blocks and inserts a few zero-width characters. |
| Prompt padding switch | Settings β Environment Settings β Function Calling | Turns those extra prompt blocks on or off. When off, only the retry strategy prompt remains. |
| Internal repair retries | Settings β Environment Settings β Function Calling | How many repair rounds are allowed after validation fails. Set it to 0 to disable internal repair. |
| Single Tool Result limit | Settings β Environment Settings β Function Calling | Stops oversized tool output from being pushed back into the website model. |
| Strategy | Best for | Behavior |
|---|---|---|
| Focused repair (default, recommended) | Most normal tool-calling flows | Sends only the validation errors and a compact repair context. Usually the most stable option. |
| Full context | Cases where the model keeps repairing incorrectly because it lacks broader context | Sends the original conversation together with repair feedback. More context, but also more room for the model to drift again. |
| Factor | Higher success | Lower success |
|---|---|---|
| Model strength | Strong models such as GPT-5 or Claude 4.5 | Smaller or specialized models |
| Tool count | 1-3 tools | 10+ tools |
| Parameter shape | Flat and simple | Deep nesting and complex objects |
| Naming clarity | search_web, get_weather | func1, tool_x |
| Prompt length | Focused system prompts | Very long role or policy setup |
Expectation management: if you see wrong function names, missing arguments, plain chat replies instead of tool calls, or backend parse failures, look at the website model first. In many cases it simply did not produce a stable <adapter_calls> block or valid JSON fallback.
search, calculate, or get.The tab pool is the project's main scheduling mechanism. The script scans supported AI-site tabs in the browser and gives each recognized tab a persistent index.
| Route | Path format | Description |
|---|---|---|
| Default route | /v1/chat/completions |
Automatically uses one idle tab |
| Fixed domain route | /url/{domain}/v1/chat/completions |
Matches a tab from the specified domain, such as /url/gemini.com/v1/chat/completions |
| Fixed tab route | /tab/{index}/v1/chat/completions |
Uses a specific tab index and queues if that tab is busy |
π‘ When to use a fixed tab:
| Parameter | Example | Effect |
|---|---|---|
selector |
?selector=round_robin |
Choose how the server picks among multiple matching tabs |
tab_index |
?tab_index=2 |
Further pin a domain route to one specific tab |
preset_name |
?preset_name=pro |
Temporarily override the preset for only this request |
π‘ Recommended mental model: /url/gemini.com/... decides which site, selector or tab_index decides which tab, and preset_name decides which config for this one request.
In the Tabs panel you can:
β οΈ Notes:
chrome://newtab are not added to the pool.Presets let you create multiple independent configurations for the same site and assign different presets to different tabs.
| Tab | Preset | Difference |
|---|---|---|
| Tab #1 | Pro chat | Full workflow, longer timeout, deep extractor |
| Tab #2 | Fast vision | Simpler workflow, multimodal extraction enabled, shorter timeout |
| Tab #3 | Coding assistant | File attach enabled, higher thresholds, network interception mode |
1. Create a preset
2. Assign a preset to a tab
3. Call the site by domain or fixed tab
# Assume one Gemini tab already exists and tab #2 uses the "Fast vision" preset
# Use the Gemini domain route
curl http://127.0.0.1:8199/url/gemini.com/v1/chat/completions
# Use the fixed vision tab
curl http://127.0.0.1:8199/tab/2/v1/chat/completions
4. Override the preset for just one request
# Option A: path-based preset (best when used directly as a Base URL)
curl "http://127.0.0.1:8199/url/gemini.com/pro/v1/chat/completions" ^
-H "Content-Type: application/json" ^
-d "{\"model\":\"any\",\"messages\":[{\"role\":\"user\",\"content\":\"Write me a short intro\"}],\"stream\":false}"
# Option B: query parameter
curl "http://127.0.0.1:8199/url/gemini.com/v1/chat/completions?preset_name=pro" ^
-H "Content-Type: application/json" ^
-d "{\"model\":\"any\",\"messages\":[{\"role\":\"user\",\"content\":\"Write me a short intro\"}],\"stream\":false}"
# Option C: JSON body
curl http://127.0.0.1:8199/url/gemini.com/v1/chat/completions ^
-H "Content-Type: application/json" ^
-d "{\"model\":\"any\",\"messages\":[{\"role\":\"user\",\"content\":\"Write me a short intro\"}],\"stream\":false,\"preset_name\":\"pro\"}"
# Fixed tab + forced preset
curl "http://127.0.0.1:8199/tab/2/v1/chat/completions?preset_name=pro" ^
-H "Content-Type: application/json" ^
-d "{\"model\":\"any\",\"messages\":[{\"role\":\"user\",\"content\":\"Continue the previous turn\"}],\"stream\":false}"
Calling rule: if you explicitly pass preset_name, that preset is forced for the current request. If path, query, and body all provide one, the order is path > query > body. If you do not pass it, the old behavior remains: tab-bound preset first, then site default preset.
π‘ Tip: every preset contains its own selectors, workflow, stream config, multimodal extraction, and file-attach settings. Editing one preset does not affect the others.
{
"gemini.google.com": {
"default_preset": "Fast vision",
"presets": {
"Main": {
"selectors": { ... },
"workflow": [ ... ],
"stream_config": { ... },
"image_extraction": {
"enabled": false,
"modalities": {
"image": { "enabled": false, "run_policy": "disabled" },
"audio": { "enabled": false, "run_policy": "disabled" },
"video": { "enabled": false, "run_policy": "disabled" }
}
},
"file_paste": { "threshold": 50000 }
},
"Fast vision": {
"selectors": { ... },
"workflow": [ ... ],
"image_extraction": {
"enabled": true,
"modalities": {
"image": { "enabled": true, "run_policy": "on_signal" },
"audio": { "enabled": true, "run_policy": "generic_only" },
"video": { "enabled": true, "run_policy": "generic_only" }
},
"audio_capture_enabled": false,
"mode": "first"
},
"file_paste": { "threshold": 10000 }
}
}
}
}
β οΈ Warning: deleting a preset cannot be undone, and at least one preset must remain.
Each site and each preset needs a set of CSS selectors so the program knows where to type and where to click.
For the first pass, focus only on input_box, send_btn, and result_container. Once these three are correct, the page usually becomes testable.
The local page is no longer just a toy demo. It now acts as a real selector workbench where you can switch target fields, change page state, inspect suggested candidates, and test whether dynamic classes break your selector.
"selectors": {
"input_box": "textarea[id='prompt']",
"send_btn": "button.send-btn",
"result_container": ".markdown-body",
"new_chat_btn": "button.new-chat",
"temp_chat_btn": "button.temp-chat"
}
π‘ How to find more stable selectors:
textarea, button, input[type=file], aria-label, and data-testid first.result_container, start with a larger wrapper around the whole answer. Targeting only a small p or span tends to miss content.input_box.send_btn.result_container.new_chat_btn, message_wrapper, and upload-related selectors when needed.π§ͺ Test selectors: the dashboard test button now opens a richer selector testing workbench. It shows not just hit count, but also matched element details, suggested candidates, and stability warnings, and it can feed a candidate back into the current field.
π‘ Custom selectors: you can define extra selectors such as a temporary-chat button and then reference them inside the workflow.
When an AI reply comes back messy, incomplete, or loses formatting, start with the extractor. If the issue lives in the underlying response body, move on to parsers and network interception.
{
"extractor_id": "deep_mode_v1",
...
}
result_container, which is enough for most normal chat pages.π‘ Configuration advice: if the full answer is already visible on the page and only the extracted result looks wrong, stay in the extractor layer. If default mode is not enough, switch to deep_mode. If the real issue is that the answer is not rendered yet, or you want to read the raw JSON / text directly, move on to parsers and network interception.
β οΈ Complex output reminder: even deep_mode can still struggle with some difficult code blocks. If you need more faithful code, formulas, or raw streaming output, continue to the network interception mode below. Just remember that unsupported sites should not enable it casually.
The control panel now calls this feature Multimodal Extraction. It is responsible for extracting images, audio, and video from the page, then returning stable local URLs or Markdown references that downstream clients can use directly.
"image_extraction": {
"enabled": true,
"modalities": {
"image": {
"enabled": true,
"run_policy": "on_signal",
"quick_probe_timeout_seconds": 1.0,
"late_wait_timeout_seconds": 45
},
"audio": {
"enabled": true,
"run_policy": "probe_if_trigger_found",
"quick_probe_timeout_seconds": 1.0,
"capture_timeout_seconds": 12
},
"video": {
"enabled": true,
"run_policy": "on_signal",
"quick_probe_timeout_seconds": 1.0,
"late_wait_timeout_seconds": 90
}
},
"selector": "img",
"audio_selector": "audio, audio source",
"video_selector": "video, video source",
"container_selector": ".img-grid",
"audio_capture_enabled": false,
"download_blobs": true,
"mode": "all",
"max_size_mb": 10
}
Compatibility note: the stored config key is still image_extraction. Legacy boolean values such as "image": true are still normalized automatically, but the recommended format is now a per-modality strategy object so "site supports this modality" is separate from "run long waits or audio capture every turn".
| Policy | Meaning | Use it when |
|---|---|---|
disabled | Turns this modality off completely. | The site does not need image, audio, or video extraction. |
generic_only | Runs one quick DOM scan only; no long wait, read-aloud click, or recording. | A preset may return media, but you do not want each round to pause, such as Gemini direct media extraction. |
on_signal | Scans quickly first; heavier waiting runs only after a clear generation signal. | Image/video generation, or sites whose parser reports media generation state. |
probe_if_trigger_found | Audio-oriented. Quickly checks for a read-aloud trigger and captures only if one is found. | The page has a read-aloud button, but most turns should not pay the capture cost. |
always_probe | Actively probes whenever the modality is enabled. | You know every turn produces that media and accept the waiting cost. |
A signal means evidence worth waiting for, not merely that the site supports a modality. Common signals include media or placeholder nodes in the current reply DOM, parser-reported media_generation_state.pending, pending-generation words in the reply, image-generation request hints, or a quick audio probe finding a read-aloud trigger.
π‘ Key change: the pipeline now starts with a quick scan controlled by quick_probe_timeout_seconds. It only enters late_wait_timeout_seconds long waits or capture_timeout_seconds audio recording when the policy and signal allow it. Supporting audio/video no longer means every round waits for audio/video.
How to use it in the UI:
generic_only and keep page playback audio capture off.Saved locations:
image/.download_images/ and exposed through local URLs.π‘ Response shape: images are returned as embedded Markdown images, while audio and video are returned as Markdown links such as [audio_0](/download_images/xxx.mp3) or [video_0](/download_images/xxx.mp4).
π‘ Practical advice: decide which modality you truly need, then choose the most conservative policy that works. Use generic_only when you only want existing DOM media, on_signal when generated media may need waiting, and audio capture only on sites with a stable read-aloud entry point.
Besides reading existing audio / video nodes, the system can click a page-level read aloud / listen / TTS button and capture the spoken audio. This path is controlled separately by audio_capture_enabled; enabling the audio modality alone does not force it to run.
"image_extraction": {
"modalities": {
"audio": {
"enabled": true,
"run_policy": "probe_if_trigger_found",
"quick_probe_timeout_seconds": 1.0,
"capture_timeout_seconds": 12
}
},
"audio_capture_enabled": true,
"audio_trigger_selector": "button[aria-label=\"Read aloud\"]",
"audio_trigger_labels": ["Read aloud", "Listen", "TTS"],
"audio_network_capture": {
"enabled": true,
"transport": "page_websocket_probe",
"url_patterns": ["voicegenie"],
"extractor": "voicegenie_ogg_pages",
"timeout_seconds": 2.5,
"settle_seconds": 0.35
}
}
Order of operations: after the text reply finishes, the system runs the audio trigger probe according to the audio policy. If clicking succeeds, it first tries direct in-page network capture, such as WebSocket TTS frames; only then does it fall back to recording page playback.
Key fields:
audio_capture_enabled: enables the read-aloud click and audio capture path. Gemini direct extraction presets should keep it off by default.audio_trigger_selector: preferred read-aloud button selector. Use a precise selector when the site markup is stable.audio_trigger_labels: fallback text labels when the selector is unstable.audio_network_capture.enabled: tries network capture before page playback recording.audio_network_capture.url_patterns: captures only audio network streams whose URL matches these keywords.audio_network_capture.extractor: network stream aggregator. Doubao read-aloud currently uses voicegenie_ogg_pages.Doubao read-aloud preset: the www.doubao.com read-aloud preset already uses this path. Its default strategy is: run only after a trigger is found, prefer network capture, and fall back to page playback recording.
Playback note: if you want to hear the audio from the controlled browser, make sure that controlled Chrome was not started with --mute-audio. Refreshing the page cannot remove process-level mute; restart the controlled Chrome instance on port 9222.
The system needs to know when the AI starts speaking and when it has really finished. There are two modes here: the default DOM mode and the more advanced network interception mode.
If the reply is visible on the page and you can capture it reliably, DOM mode is already good enough. It is also the easiest starting point when you adapt a new site for the first time.
Move to network interception when code blocks keep breaking, formulas keep losing structure, DOM extraction becomes unreliable, or the site returns one complete JSON / text payload at a time.
| Mode | Streaming | Description |
|---|---|---|
| DOM mode (recommended default) | β Yes | Watches page changes like a human eye and works well for most normal chat scenarios |
| Network interception mode | β Yes, site-dependent | Intercepts browser requests directly and parses the response body, better for advanced extraction on adapted sites |
DOM mode works by βwatching the pageβ: as long as the content is still changing, the system assumes the AI is still typing. It is the best default for plain-text chats, roleplay, and most everyday use cases.
Finished = (stable checks β₯ target count) AND (silence duration > silence timeout)
How it works:
| Scenario | Silence timeout | Stable checks | Initial wait |
|---|---|---|---|
| Fast models (GPT-4o) | 3-5 s | 3-5 | 60 s |
| Slow reasoning (o1 / Claude) | 10-15 s | 8-12 | 300 s |
| Long-form or code generation | 8-15 s | 6-10 | 180-300 s |
β οΈ Prerequisite: network interception must be used together with a site parser. Do not enable it casually on unsupported sites, or you may get errors or raise the chance of anti-bot / Cloudflare challenges.
Network interception works like reading the browser's underlying response directly. It intercepts XHR / Fetch traffic, reads the raw JSON or text, and turns it into final output or incremental chunks. On adapted sites, it is often faster, more stable, and better at preserving code blocks and formulas.
parser and a good listen_pattern.π‘ How to enable it: open the site configuration in the dashboard, go to Streaming Configuration, and switch the mode to Network Interception.
π‘ Recommended order: lock the request keyword first, choose the parser second, and tune timeouts after that. Many failed attempts happen because the first two fields are still wrong.
| Field | Description | Example |
|---|---|---|
| URL match pattern | Intercept only requests whose URL contains this text; start with the most stable path keyword | GenerateContent |
| Response parser | The parser ID used to read the intercepted response body | lmarena / custom parser |
| First response timeout | Maximum wait for the first useful network response | 300 s |
| Silence timeout | How long to wait after the last new chunk before considering the reply finished | 3 s |
| Polling interval | How often the listener checks for new intercepted data | 0.5 s |
π‘ Arena parser note: arena.ai already ships with two parsers. lmarena is for direct single-column mode, and lmarena_side_left is for side-by-side mode and reads the left output automatically. In most cases, keep the default mapping.
β οΈ arena.ai: in practice, it is still safer to stay on DOM mode for this site. Extra browser connections from network monitoring can raise the chance of Cloudflare challenges.
If the target site returns JSON, SSE, or plain text from the network layer, you usually need a custom parser. The fastest workflow now is: try the built-in debug capture first, and only fall back to the manual console listener when the built-in capture misses the request.
Recommended first choice: turn on the built-in parser debug capture in Dashboard β Settings β Browser Constants. When a network parser is hit, the project can automatically save raw bodies and parser debug data into logs/network_parser_debug/.
NETWORK_DEBUG_CAPTURE_ENABLED.NETWORK_DEBUG_CAPTURE_PARSER_FILTER to a parser ID such as deepseek so you only capture the site you are debugging.β οΈ Important: built-in capture is much simpler, but it is not guaranteed to catch every site or every response shape. Some websites still need manual in-page interception, especially when their traffic is hidden behind unusual wrappers or paths that the normal listener cannot see cleanly.
NETWORK_DEBUG_CAPTURE_ENABLED.NETWORK_DEBUG_CAPTURE_PARSER_FILTER with the parser you are testing, such as deepseek.logs/network_parser_debug/ and collect the generated JSON debug files.app/core/parsers/base.py, app/core/parsers/__init__.py, and one or two similar parser files.app/core/parsers/xxx_parser.py and tell you how to register it in __init__.py.listen_pattern and parser ID, then test again.Use this only when the built-in capture did not record the request or did not preserve enough structure.
Manual helper script: the project already includes static/ζ¦ζͺθ―·ζ±εηε¨.txt. You can copy it from this tutorial section or open the original text file directly.
Console.exportRequests() in the console. The browser will download a requests JSON export.listen_pattern and parser ID to use.β οΈ Maintainer-only workflow: the exported JSON can contain live session structure, request metadata, prompt fragments, and response content. Skip this unless built-in debug capture was not enough, and only use it on your own local session data.
π‘ Tell AI these requirements:
done=True to avoid duplicate output.images field.app/core/parsers/__init__.py to register the parser.You can send AI a prompt like this:
This is a network parser debug sample from a new site. Please refer to the existing parsers in app/core/parsers and help me add a new parser.
Requirements:
1. Create a new parser class in app/core/parsers/xxx_parser.py and inherit from ResponseParser
2. Inspect the raw response carefully and decide whether it is plain JSON, an SSE/event stream, or another text format
3. After the first successful parse, return done=True to avoid duplicate appends
4. If images are included, fill the images field
5. Tell me what code I need to change in app/core/parsers/__init__.py to register it
6. Tell me what listen_pattern and parser I should fill into the site configuration
Attached:
- debug dump files from logs/network_parser_debug (preferred), or exported requests json from the manual listener if built-in capture missed it
- one or two reference parser files
- app/core/parsers/base.py
- app/core/parsers/__init__.py
`;
translations.sections['workflow'] = `
The workflow defines a sequence of actions. Each preset can have its own independent workflow.
"workflow": [
{ "action": "CLICK", "target": "new_chat_btn" },
{ "action": "WAIT", "value": 0.5 },
{ "action": "FILL_INPUT", "target": "input_box" },
{ "action": "CLICK", "target": "send_btn" },
{ "action": "STREAM_WAIT" }
]
| Action | Description |
|---|---|
CLICK | Click an element, where target is a selector key |
FILL_INPUT | Fill the input box with the user prompt |
WAIT | Wait for a fixed number of seconds |
KEY_PRESS | Simulate a key press such as Enter |
JS_EXEC | Run JavaScript inside the current page when simple clicks are not enough |
STREAM_WAIT | Wait until the reply is finished |
Visual editor update: the workflow visualizer now keeps abstract steps such as WAIT, KEY_PRESS, and JS_EXEC as their own balls instead of silently dropping or merging them. The saved JSON order now matches the visual order much more closely.
πΈ Workflow visualization: static/workflow-visualization.png
This screenshot shows how the visual workflow mode helps locate page elements and action steps.
When your prompt is too long for the site's input box, the system can write it into a temporary .txt file and try to send it through the site's own upload entry instead of forcing all of the text into the input box.
| Field | Default | Description |
|---|---|---|
| Enabled | Depends on site preset | Whether file-attach mode is enabled |
| Threshold | 50000 chars | Global default threshold; site presets can override it |
| Hint text | Focus entirely on the file content | Extra instruction appended after the file is attached to remind the model to read the attachment |
π‘ Upload order: the system tries file_input first, then generic input[type=file], then upload_btn, then drop_zone. Only after all of those fail does it fall back to the Windows clipboard path, and it will only continue sending after the attachment state looks stable. That readiness check is now also configurable per site.
For new sites, do not stop at enabled: true. You still need to confirm that the site has a usable upload_btn, file_input, or drop_zone selector, and also how the site signals that the attachment is really present in the composer.
You can now open Config -> Network listener mode -> Attachment send confirmation -> Advanced attachment rules and define site-specific readiness rules. The most useful fields are:
attachment_selectors: preview nodes, chips, or attachment cards that count as a successful uploadpending_selectors: spinners, progress bars, and loading states that mean the upload is still in progressbusy_text_markers: text markers on the composer or container that still indicate a busy statesend_button_disabled_markers: class / aria tokens on the send button that mean the site still does not allow sendingrequire_attachment_present: require a real attachment preview before the workflow can continuecontinue_once_on_unconfirmed_send: whether the system may still risk a one-time send when readiness was not confirmedβ οΈ Why the hint text matters: after the upload succeeds, the system appends a short reminder such as βfocus entirely on the file contentβ so the model does not ignore the attachment and only react to the short text in the box.
Gemini example: on sites like gemini.google.com, a preview chip or image preview is usually a stronger success signal than βthe network looks idleβ. If failed uploads leave the send button gray or disabled, add those button tokens to send_button_disabled_markers and disable continue_once_on_unconfirmed_send to avoid sending before the attachment is really there.
aistudio.google.comgemini.google.comchat.deepseek.comwww.doubao.comchat.qwen.aiPractical advice: first confirm that the target site really supports file uploads, then make sure at least one reliable upload selector works, and only then test long prompts. Turning on the flag alone is usually not enough.
Stealth mode makes clicks, cursor movement, scrolling, and timing behave more like gradual manual interaction so some sites are easier to drive reliably.
Instead of βteleport and click immediatelyβ, stealth mode uses smoother motion and pauses so page interaction is less abrupt.
| Action | Normal mode | Stealth mode |
|---|---|---|
| Click | Immediate CDP command | Press -> tiny movement -> release |
| Move | Instant jump | Bezier-like human motion |
| Idle | No movement | Small random drift |
| Scroll | Direct call | Wheel-event style interaction |
| Delay | Minimal | Randomized 0.1-0.3 s |
π Where to enable it: select a site in the dashboard and check Stealth Mode at the top. That is the current UI label, and the setting is stored per preset.
This project applies a small DrissionPage patch so network monitoring can reuse the browser's main connection instead of opening extra ones, which improves compatibility.
start.bat applies the patch after dependency installation.python patch_drissionpage.pypython patch_drissionpage.py --restoreβ οΈ Important: re-apply the patch after every DrissionPage upgrade. The patch is idempotent, so repeated runs are safe.
β οΈ arena.ai note: even with stealth mode enabled and network monitoring disabled, repeated chatting can still trigger Cloudflare after roughly ten messages in half an hour. That indicates the site is simply a poor fit for long continuous test sessions, and lowering frequency is still the safer approach.
The command system lets you define triggers and actions so tabs can recover automatically, switch routes, or send alerts.
π Where to configure it: open the dashboard β Commands.
β οΈ Built-in commands: the project ships with several predefined commands, most of them disabled by default. Many of them are highly tailored to Gemini and Arena workflows. If you do not understand what a command does yet, leave it disabled.
| Mode | Description | Best for |
|---|---|---|
| Simple mode | Pick a trigger and configure an action chain | Common automation needs |
| Advanced mode | Write JavaScript or Python code directly | Complex custom logic |
| Trigger | Description | Example |
|---|---|---|
request_count | Total requests hit a threshold | Run once every 10 turns |
error_count | Consecutive errors hit a threshold | Recover after 3 failures |
idle_timeout | A tab stays idle for too long | Reclaim a tab after 5 minutes |
page_check | Specific text appears on the page | React to Cloudflare prompts |
command_triggered | Fire after another command fires | Chain recovery steps |
command_result_match | Match the result of an upstream command | Branch on CSS_FAILED |
network_request_error | Intercept bad network status codes by URL rule | React to 429 or 5xx |
π‘ page_check matching: it checks document.body.innerText first and falls back to document.title only when necessary. As of 2.5.9, you can also use || for OR and && for AND inside the match text field.
chatgpt.com.priority; larger numbers win.CMD_REQUEST_PRIORITY_BASELINE in .env.| Action | Description |
|---|---|
clear_cookies | Clear cookies for the current tab |
refresh_page | Refresh the page |
new_chat | Click the new-chat button |
run_js | Execute JavaScript in the page |
wait | Wait for a number of seconds |
execute_preset | Switch to a preset |
execute_workflow | Run a preset workflow immediately |
navigate | Open a target URL |
switch_proxy | Switch proxy node through Clash |
send_webhook | Send an external alert |
execute_command_group | Run a grouped command set in sequence |
abort_task | Stop the current task and optionally stop following actions |
You can use command_result_match as a branch controller. It watches the result of another command and triggers only when the expected value is matched.
| Field | Meaning | Suggested usage |
|---|---|---|
| Observed command | The upstream command to watch | Choose one with stable outputs |
| Action ref | Optional specific step inside that command | Useful for JS probe steps |
| Rule | equals, contains, or not_equals | Prefer equals for status strings |
| Expected value | The result you want to match | Examples: SUCCESS, CSS_FAILED |
send_webhook supports alert templates such as {{tab_index}}, {{domain}}, {{network_status}}, and {{network_url}}.
{"msg":"Tab #{{tab_index}} on {{domain}} hit {{network_status}}, URL={{network_url}}"}
Prerequisite: Clash must be running with External Controller enabled, for example:
external-controller: 127.0.0.1:9090
secret: ""
| Setting | Default | Description |
|---|---|---|
| Clash API URL | http://127.0.0.1:9090 | External Controller address |
| Proxy group | Proxy | Proxy group name in Clash |
| Switch mode | Random | Random, round-robin, or fixed node |
| Refresh after switching | Enabled | Refresh the page after the node changes |
CSS_FAILED β clear cookies β refresh β execute a recovery preset.// JavaScript example
document.cookie.split(";").forEach(c => {
document.cookie = c.trim().split("=")[0] +
"=;expires=Thu, 01 Jan 1970 00:00:00 UTC;path=/";
});
location.reload();
# Python example
logger.info(f"Current URL: {tab.url}")
if session.error_count > 2:
tab.run_js("location.reload()")
logger.info("Page refreshed after repeated errors")
β οΈ Safety: only use trusted scripts in advanced mode. Python scripts run in a restricted backend sandbox by default; if unsafe mode is explicitly enabled, they have full system access.
For sites outside the supported list, the system can call a helper AI to analyze the page and identify key elements such as the input box, send button, and reply container.
π Where to configure it: open the dashboard β Settings β AI Recognition.
The flow is simple: open an unknown site -> send the first real request -> the system asks a helper AI to analyze the page -> the generated selectors are written into the site config.
| Key | Description | Required |
|---|---|---|
input_box | User input field | β |
send_btn | Send button | β |
result_container | Container that holds AI output only | β |
new_chat_btn | Button that starts a new chat | β |
message_wrapper | Outer container for one full message | β |
generating_indicator | Visible indicator shown while generating | β |
| Field | Description |
|---|---|
| Order | Controls recognition priority |
| Key | The element identifier to search for, such as input_box or send_btn |
| Description | Plain-language guidance that helps the AI find the element accurately |
| Enabled | Whether the recognition item is active |
| Actions | Edit or remove the item |
config/sites.json.π‘ Tip: if you only use already supported sites, you can ignore this feature completely.
π‘ Manual flow: if you prefer not to configure a helper API, you can still let the project generate an empty site shell, then fill selectors manually with the selector tester or workflow visualizer.
Edit these in Dashboard β Settings β Environment. Changes here require a restart.
| Setting | Default | Description |
|---|---|---|
| Listen host | 127.0.0.1 | Use 0.0.0.0 to allow external access |
| Listen port | 8199 | HTTP service port |
| Debug mode | On | Enables /docs and detailed error output |
| Log level | INFO | DEBUG, INFO, WARNING, or ERROR |
| Setting | Default | Description |
|---|---|---|
| Enable auth | Off | Require a Bearer token for API calls |
| Bearer token | Empty | Must be set when auth is enabled |
| Setting | Default | Description |
|---|---|---|
| Enable CORS | On | Allow cross-origin requests or not |
| Allowed origins | * | Comma-separated list, or * for all origins |
| Setting | Default | Description |
|---|---|---|
| Enable proxy | Off | Route browser traffic through a proxy |
| Proxy URL | socks5://127.0.0.1:1080 | Supports socks5:// and http:// |
| Bypass list | localhost,127.0.0.1 | Addresses that should skip the proxy |
| Setting | Default | Description |
|---|
This is an advanced option. If you want the script to inherit your existing login state, cookies, and extensions, point it to a copied user-data directory rather than your live system Chrome profile.
β οΈ Important: starting from Chrome 136, Chrome tightened remote-debugging behavior. If you point BROWSER_PROFILE_DIR to the live system User Data folder, Chrome may open while the debugging port never becomes usable.
β οΈ Session-data reminder: copied browser profiles can contain login state, cookies, local cache, and extension data. Only copy your own local profile into a dedicated directory for local debugging. Do not upload it, commit it, or pass it to someone else.
C:\\Users\\YourName\\AppData\\Local\\UniversalWebApiProfile.Local State and the profile folder you want into that new directory..env and restart start.bat.BROWSER_PATH=C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe
BROWSER_PROFILE_DIR=C:\\Users\\YourName\\AppData\\Local\\UniversalWebApiProfile
BROWSER_PROFILE_NAME=Default
Field meaning:
BROWSER_PATH: browser executable path.BROWSER_PROFILE_DIR: the root user-data directory.BROWSER_PROFILE_NAME: a profile name such as Default or Profile 1.Common mistake: do not set BROWSER_PROFILE_DIR to ...\\User Data\\Default. The directory should point to the root, while the exact profile name belongs in BROWSER_PROFILE_NAME.
If you do not need your own login state: the simplest option is to leave BROWSER_PROFILE_DIR empty and keep using the project's built-in chrome_profile/ directory.
| Setting | Default | Description |
|---|---|---|
| API Key | Empty | API key for the helper AI |
| Base URL | http://127.0.0.1:5104/v1 | Base URL of the helper API |
| Model | gemini-3.0-pro | Model used for page analysis |
| Max HTML length | 120000 | Longer HTML is truncated to save tokens |
Edit these in Dashboard β Settings β Browser Constants. Changes here take effect immediately.
| Setting | Default | Description |
|---|---|---|
| Debug port | 9222 | Chrome DevTools remote debugging port |
| Connection timeout | 10 s | Timeout for connecting to the browser |
Randomized delays that help the automation behave more like a human user
| Setting | Default | Description |
|---|---|---|
| Stealth delay min | 0.1 s | Minimum stealth-mode delay |
| Stealth delay max | 0.3 s | Maximum stealth-mode delay |
| Action delay min | 0.15 s | Minimum delay for clicks and inputs |
| Action delay max | 0.3 s | Maximum delay for clicks and inputs |
| Setting | Default | Description |
|---|---|---|
| Default wait | 3 s | Default timeout when searching for an element |
| Fallback wait | 1 s | Retry timeout after the first failure |
| Cache lifetime | 5 s | How long cached element positions stay valid |
These switches only change how logs read in the dashboard. They do not change runtime behavior, and the original log text is still preserved for troubleshooting.
| Setting | Default | Description |
|---|---|---|
| INFO cute mode | Off | Polishes common INFO logs into friendlier, easier-to-scan wording. |
| DEBUG cute mode | Off | Polishes the main DEBUG paths into more readable hints, such as βXiaolu starts splitting the text nowβ; useful when you stare at logs for a long time. |
Controls polling frequency and timeout thresholds while waiting for replies
| Setting | Default | Description |
|---|---|---|
| Check interval min | 0.1 s | Fastest polling interval |
| Check interval max | 1.0 s | Slowest polling interval |
| Default interval | 0.3 s | Initial polling interval |
| Silence timeout | 8.0 s | Main finish condition when content stops changing |
| Silence fallback | 12 s | Extra patience for slower models |
| Max timeout | 600 s | Absolute timeout for one request |
| Initial wait | 180 s | Max wait before any response appears |
| Stable checks | 8 | How many unchanged checks are needed to finish |
Usually you do not need to change these values
| Setting | Default | Description |
|---|---|---|
| Re-render wait | 0.5 s | Extra wait after page re-rendering |
| Shrink tolerance | 3 | Allowed number of content-shrink events |
| Minimum valid length | 10 | Shortest response considered valid |
| Initial element wait | 10 s | How long to wait for the response element to appear |
| Max anomaly count | 5 | Abort after too many anomalies |
| Max missing-element count | 10 | Abort after repeated missing-element checks |
Use these when you are developing or fixing a network parser. Captured files are written to logs/network_parser_debug/.
| Setting | Default | Description |
|---|---|---|
| Enable response debug capture | Off | When a network parser runs, save the raw body, request metadata, and parser debug summary for later inspection. |
| Max body chars | 200000 | Maximum raw response length to save per capture file. Raise this if the site streams long replies and the useful tail gets cut off. |
| Parser filter | Empty | Leave blank to capture all parsers, or set a specific parser ID such as deepseek to reduce noise. |
π‘ Recommended usage: when adapting a new site, enable parser debug capture first and reproduce one real reply. If no useful dump appears, then switch to the manual listener script from the tutorial's parser guide.
config/ directory regularly.Edit this in Dashboard β Settings β Update Whitelist. After you save it, the settings are written into config/update_settings.json and take effect during the next automatic update.
config/, app/, or static/, the whole directory is preserved.config/sites.json and config/commands.json are not whitelisted, the updater tries to merge release updates while preserving local changes.config/update_settings.json itself is always preserved automatically.Default preserved items:
config/sites.local.jsonconfig/commands.local.jsonchrome_profile/ (local session data; keep it private and out of version control)venv/logs/image/updater.py.git/__pycache__/*.pycbackup_*/If all you want is to preserve login state and your own configuration, the default selection is usually enough.
β οΈ Warning: the visual editor currently has no undo feature. After saving, you cannot quickly revert changes unless you already have a backup.
A: The built-in API test often sends a request that is too short. This is especially common with AI Studio. In practice, it is better to send a real chat message instead of using the test button.
/v1/chat/completions): uses one idle tab automatically./url/gemini.com/v1/chat/completions): matches one tab from that site./tab/1/v1/chat/completions): always uses tab #1.A: Usually not. Most defaults are already tuned for common usage.
http://127.0.0.1:9090/proxies.[CMD] logs in the dashboard for exact failures.User Data directory, which often breaks remote debugging on Chrome 136+.BROWSER_PROFILE_DIR points to the wrong level; it should point to the root user-data directory, not ...\\User Data\\Default.π¬ Feedback channels:
Function calling still has plenty of rough edges and depends heavily on the model's own ability to understand instructions. Treat it as a feature that needs testing, not as something you can trust blindly.
The author mainly uses Gemini, so other sites may not receive timely updates. For problems, the QQ group is a better feedback channel than GitHub Issues or Discord.
Many issues do not require waiting for the author. Selector failures, wrong configs, and broken workflow order are often faster to diagnose by giving logs and element trees to another AI first.
When a site suddenly stops working, parser bugs are possible, but selector failures are more common.
Updating selectors is usually simple:
Parser issues are harder: they often require network interception, response analysis, and reverse engineering.
If you can fix it yourself, please do: if you already solved the problem, sending the updated config or code back to the author is very helpful.
If you report a bug, describe the issue clearly and attach DEBUG-level logs whenever possible. If wording is difficult, ask another AI to help organize the report first.
This tutorial tries to cover the major features, but some details are still brief. Contributions to improve the docs are welcome.
A lot of this project's design comes from roleplay-oriented use cases. Features outside that core workflow may exist in a more lightly maintained state.