Runner H and Surfer-H: A Browser Agent and Its Holo1 Models

Matthew Berman demonstrates H Company’s browser agent and reviews its research on visual navigation. The paper’s benchmark configuration and the demo’s missing links show why results need context.

MBMatthew BermanAnime · Forward Future
Chaîne
Matthew Berman
Publié
10 juin 2025
Vidéo
11:24
Lecture
5 min
Diapositives
5
Original
anglais
Ouvrir sur YouTube
En faisant défiler, la vidéo se réduit en bas à droite et continue. Les sous-titres suivent votre langue de lecture.
Langue de lectureDisponible en 2 langues
Cet article n’est pas encore disponible en français ; il s’affiche dans l’original en anglais.
RÉSUMÉ

En 30 secondes

In this sponsored June 2025 video, Matthew Berman introduces H Company’s Runner H beta and asks it to gather Pokémon card listings in a Google Sheet. He explains the Surfer-H visual browser agent, its policy, localizer and validator, and Holo1’s open model weights. Berman highlights company benchmark results and a QA testing product. The linked research paper shows that the widely cited 92.2% WebVoyager score used GPT-4o validation and up to ten attempts, while the demonstration sheet itself contained two missing links.

  1. 1A browser taskBerman asks Runner H to collect ten Pokémon listings in Google Sheets, after authorizing the connection.
  2. 2Open-weight visual modelsHolo1 provides visual navigation and localization capabilities, with released model weights and an evaluation dataset.
  3. 3Three-part agent loopA policy selects actions, a localizer finds screen coordinates, and a validator decides whether to return the answer or continue.
  4. 4Read the benchmark configurationThe paper’s 92.2% result uses GPT-4o as validator and up to ten attempts, not an entirely Holo1-based setup.
  5. 5A partially complete sheetThe output has ten entries, but two links are unavailable because of CAPTCHA.

À retenir

  1. Runner H was presented as a browser-agent beta, with Google Sheets authentication needed for the demonstrated task.
  2. Surfer-H uses screenshots and a policy, localizer and validator to choose actions, locate controls and assess answers.
  3. Holo1 model weights and the WebClick evaluation dataset were released, subject to their respective license terms.
  4. The paper’s 92.2% WebVoyager score used Holo1-7B for policy and localization, GPT-4o validation and up to ten attempts.
  5. The demonstrated sheet contained ten entries but two lacked links because of CAPTCHA, so the request was not fully completed.
  6. The company’s benchmark and product claims, pricing and beta availability reflect the 2025 launch.
TEXTE INTÉGRAL

Lire le texte remanié, chapitre par chapitre

Transcription seule
01

Runner H and a Pokémon card task

Berman introduces the H Company partnership and launches a browser task. He authorizes Google Sheets and watches the agent navigate.

H Company has released Runner H, a browser agent that takes a natural-language task and attempts to carry it out on the web. In this June 2025 video, Matthew Berman says he partnered with the company to introduce the launch. He describes the product as free and in beta at the time, and also reviews its research and released model weights.

For a demonstration, Berman asks the agent to find ten Pokémon card listings on eBay and create a Google Sheet with their links. Runner H asks him to connect Google Sheets; he authorizes the integration before the task proceeds. He notes that multiple browser agents can be started in parallel, although the video follows this particular task.

The interface displays a plan, thoughts, notes and web actions alongside a live browser view. Berman watches the agent search for listings and says he will return to the result later. The visible trace makes the steps inspectable, but the interface alone does not establish that every extracted item is accurate.

02

Holo1 and Surfer-H

He introduces the Holo1 weights and explains the visual agent’s three components. Screenshots provide the agent’s view of the page.

Berman introduces Surfer-H and the Holo1 family of vision-language models, including navigation and localization demos. Localization means identifying screen coordinates for an element to click. In one example, a model receives a screenshot of a travel site and a hotel-booking request, then suggests selecting an August 6 checkout date for a three-night stay starting August 3.

Holo1 navigation example
DIAPOSITIVE 1 / 5Holo1 navigation exampleA screenshot and a booking request produce a proposed calendar-selection action.

The [research paper](https://arxiv.org/abs/2506.02865) describes Holo1 as an open-weight model collection and introduces the WebClick evaluation dataset. The weights are available for use and further development, but readers should check the license for the specific model: the [Holo1-3B license](https://huggingface.co/Hcompany/Holo1-3B/blob/main/LICENSE) includes terms inherited from its Qwen base model. Open weights do not by themselves establish that the entire hosted product is open source.

Berman explains the motivation for visual browser agents: many websites are designed for people and do not offer suitable standardized APIs for every task. Rather than build a separate integration for each site, a visual agent can operate graphical controls by clicking, scrolling and typing. This approach can broaden access to web workflows, although it still depends on what the interface exposes.

Surfer-H has three main modules: a policy, a localizer and a validator. According to the paper, it operates from website screenshots without needing the document object model or accessibility tree. A vision-language policy proposes the next action, and the localizer translates a description of a target element into screen coordinates.

Actions such as opening a URL, scrolling, waiting or clicking are executed in sequence. When the policy produces an answer, a validator assesses whether it is suitable. If it rejects the answer, feedback goes into the agent’s memory and execution continues until a result is accepted or a time or cost budget is reached. The modules can use different compatible models.

Surfer-H agent loop
DIAPOSITIVE 2 / 5Surfer-H agent loopThe paper diagrams memory, policy, localizer, browser actions and validator feedback.

Berman walks through the paper’s diagram: the task and observations enter memory, the policy chooses an action, and a localizer supplies coordinates when a click or text entry requires them. The browser’s next screenshot feeds the loop, while the validator can return a final answer or send feedback back to memory.

03

Localization and benchmark claims

Berman reviews click accuracy and WebVoyager cost comparisons. The research paper provides essential details about attempts, validation and estimated costs.

The paper compares Holo1-3B and Holo1-7B with similarly sized models on UI localization benchmarks, including the company’s WebClick dataset. Berman highlights their higher average click accuracy in the displayed table. These are the authors’ results on particular benchmark sets, not a guarantee that either model will correctly click every control on an arbitrary website.

UI localization benchmark table
DIAPOSITIVE 3 / 5UI localization benchmark tableThe researchers compare click accuracy across the listed models and benchmark sets.

Berman also describes a cost-versus-accuracy chart and cites the company’s 92.2% WebVoyager result at an estimated $0.13 per task, alongside comparisons with other model configurations. These estimates reflect the paper’s inference-cost assumptions and its tested workload, not a universal price for every production task.

The paper’s configuration matters: the 92.2% figure used Holo1-7B for policy and localization but GPT-4o for validation, with up to ten attempts across 643 WebVoyager tasks. It was not an entirely Holo1-based run, despite the wording shown in the video. The paper reports 69.6% on one attempt for that configuration; using Holo1-7B for all three modules produced 80.4% after ten attempts at an estimated $0.06 per task. The standard Holo1 policy also trained on WebVoyager-related tasks, so the score should not be generalized as unseen-task performance.

04

Results, integrations and Tester H

The generated sheet contains missing links. Berman shows controls and integrations, then introduces the private-beta testing product.

Generated Pokémon listings sheet
DIAPOSITIVE 4 / 5Generated Pokémon listings sheetThe sheet displays ten rows, including two links unavailable because of CAPTCHA.

Returning to the eBay task, Berman opens the generated Google Sheet. It has ten listing rows, but two entries say their links are unavailable because of CAPTCHA. The agent produced a useful partial result after the connection was authorized, but it did not deliver all ten requested links, and the video does not independently verify the remaining listings.

Berman shows options for highly involved, moderately involved or fully automated interaction, along with uploaded documents and a files view. He points to integrations with Google Sheets, Docs, Drive, Notion, Slack and Zapier, which require authentication. He also says payments were marked as coming soon at the time; the video does not demonstrate a completed payment.

Finally, Berman introduces Tester H, then in private beta, as a way to automate website or application quality assurance. The example specifies a natural-language test on an Airbnb page: navigate to a listing, scroll sideways, click the first photo with an orange bed and check that the image appears. He closes by encouraging viewers to explore the released models and tools.

Tester H example
DIAPOSITIVE 5 / 5Tester H exampleA natural-language test is paired with a browser action trace.

Sur la même chaîne

12 diapositives34:21EN
Matthew Berman· 27 oct. 2026Matthew Berman Examines Anthropic’s IPO Filing, AI Growth and GovernanceMatthew Berman, Sam Altman · Forward FutureThe presenter reviews the S-1’s financial metrics, compute strategy and voting structure while questioning the tension between rapid AI development and safety.AnthropicAI SafetyIA en entreprise+2
2 diapositives0:36EN
Matthew Berman· 25 oct. 2026Kimi K3.1 and a Rubik’s Cube Solver DemonstrationMatthew Berman · Forward FutureMatthew Berman shows a scrambled virtual cube being solved and gives the Kimi K3.1 test a favorable assessment.ÉvaluationsAgents de codeOpen-weight models
6 diapositives25:50EN
Matthew Berman· 24 oct. 2026Kimi K3.1 Pro Tested: Strong 3D Demos, Uneven Results and Competitive PricingMatthew Berman · Forward FutureMatthew Berman tests the open-weight model on coding, 3D rendering and visual explanations, then compares benchmark results and prices.Open-weight modelsBenchmarksAI cost & performance+1
2 diapositives32:20EN
Matthew Berman· 23 oct. 2026Who Is Local AI For? The Costs, Trade-offs and a Hybrid FutureMatthew Berman · Forward FutureMatthew Berman weighs data control and independence against hardware cost, setup complexity and inference speed.Local inferenceOpen-weight modelsAI cost & performance+1
6 diapositives1:03:37EN
Matthew Berman· 23 oct. 2026Matthew Berman’s Team Tests Kimi K3.1 on Animation, Games, and ChartsMatthew Berman, Alex, Brian · Forward FutureA live team AMA turns into an informal test of Kimi K3.1, from personal-agent workflows to motion graphics, a browser game, and a benchmark chart.Agents de codeGenerative mediaÉvaluations+2
0:53EN
Matthew Berman· 22 oct. 2026Sam Altman on AI and the Discovery of New KnowledgeSam Altman · OpenAIAltman expects progress in mathematical discovery to extend to biology, materials and energy.AI for scienceMathematicsComputational Biology+1