LlmBench combines account-backed provider WebViews with a native chat/compare layer. Provider-specific behavior stays behind bounded adapters instead of leaking across the UI.
Account-backed WebViews#
The current Android client includes WebView entries for ChatGPT, Claude, Gemini, DeepSeek, Kimi, Mistral Vibe, Qwen, Microsoft Copilot, Z.ai, Grok, Character.AI, Venice, and Meta AI.
"Integrated" does not guarantee that every provider permits fresh embedded sign-in. Authentication remains between the provider page and that provider, and LlmBench does not scrape passwords, cookies, OAuth tokens, or page contents.
| Provider | Status | Uploads | Activity tracking | Notes |
|---|---|---|---|---|
| ChatGPT | Full | Yes | Generating + unread | Persistent provider-scoped session |
| Claude | Full | Yes | Generating + unread | Persistent provider-scoped session |
| Gemini Chat | Partial | Yes | Generating + unread | Fresh Google OAuth may be blocked in embedded user-agents |
| DeepSeek | Full | Yes | Generating + unread | Persistent session |
| Kimi | Full | Yes | Generating + unread | Persistent session |
| Mistral Vibe | Full | Yes | Generating + unread | Provider-scoped activity probe |
| Qwen | Partial | Provider-documented; embedded flow unverified | Not yet | Embedded sign-in/upload verification pending |
| Microsoft Copilot | Partial | Provider-documented; embedded flow unverified | Not yet | Provider-owned auth hosts are bounded in-WebView |
| Z.ai | Partial | Page-driven; embedded flow unverified | Not yet | Provider-owned auth hosts are bounded in-WebView |
| Grok | Partial | Provider-documented; embedded flow unverified | Not yet | Activity probe still needs verification |
| Character.AI | Partial | Provider-documented image attachments; embedded flow unverified | Not yet | Sign-in/upload verification pending |
| Venice | Partial | Provider-documented; embedded flow unverified | Not yet | Sign-in/upload verification pending |
| Meta AI | Partial | Not verified | Not yet | Provider-owned Meta login surface |
Native and free-provider layer#
The native chat layer is already implemented, not merely planned:
- Gemini through
generateContentREST with streaming support. - OpenAI through the Responses API with
store=falseand bounded visible-text replay. - Claude through its native API transport.
- OpenRouter Free through the shared OpenAI-compatible gateway adapter.
- AIHubMix Free through the same gateway boundary.
The two gateway aggregators stay outside the default All Models comparison so the same underlying model is not routed twice.
Provider tweaks and diagnostics#
Provider tweaks live in a small, auditable in-app registry. Scripts are static, scoped to matching provider hosts, and applied after page load; remote userscript code is never fetched. Diagnostics expose only privacy-safe integration metadata such as provider host, WebView package/version, capability counts, activity-tracking support, and file-picker events. They do not collect page text, full URLs, form values, file names, cookies, or authentication tokens.
Compatibility for newer providers is intentionally marked partial until embedded sign-in, uploads, and provider-specific generation tracking are verified in the Android WebView.
Gemini and embedded Google sign-in#
Gemini Chat is integrated, but Google may reject OAuth flows from embedded user-agents. That is a provider policy boundary, not something LlmBench attempts to bypass.