Philipp Schmid: Seven Things an AI Coworker Needs

Two years ago, we mostly asked chatbots simple questions. Then assistants gained tools and coding agents began writing software. Now we can give an agent its own computer and a task, and return hours later to see what it has produced. I want to show how to build an autonomous Wall Street coworker.

A coworker needs a brain, a computer for artifacts and iteration, playbooks for unfamiliar workflows, tools, credentials, house rules and a schedule. Those seven requirements apply to digital employees too. We will build one using Gemini and work through each part.

Before that, we recently announced Gemini 4. I highlighted its unified one-million-token context window and the extra room it gives the model to reason through difficult math. The Interactions API is a short way to get started: obtain an API key and send a request. The price I quoted at the talk was $2 for input and $10 for output.

Google teams had been testing the model internally. I described work involving Chromium, making the open Gemma model more efficient to run, improvements to the ADK framework, and internal code. One colleague also prompted Gemini to build a Python 3D ray tracer and used generated narration to explain it. The point is to be ambitious about the problems you try.

In a conventional tool-calling loop, the model returns a function call, your application executes it, and the result goes back to the model. Managed Agents provides a remote sandbox where Gemini can run tools and work with files directly. Instead of supplying a model ID to the Interactions API, you specify an agent ID, prompt it and use the managed environment.

Our coworker is called Buck. To define Buck, we give the agent an ID, select a base agent and model, and provide system instructions. The remote agent uses the same harness as Antigravity. We then call our own agent by its ID through the Interactions API.

A computer also needs its working environment. Managed Agents can receive inline files, a public or private GitHub repository containing skills, or a Google Cloud Storage bucket. Skills, hooks and system instructions live as files; the agent can also create skills or store learnings from previous sessions in its environment.

I separate knowledge by when it is needed. System instructions accompany every request, so keeping them concise matters for cost. Skills have short descriptions that tell the agent when to read them. Rather than making a skill into a dense document, split detailed context into references and scripts that can be loaded as needed.

For example, our portfolio-audit skill tells the agent when and how to run an included Python script, what to do with its results, and where to store them. It gives a new coworker a repeatable playbook without loading every detail for every task.

The next requirement is tools. Agents can use built-in capabilities such as Google Search and code execution, an MCP server, or custom functions. Managed Agents can connect to Workspace MCP tools for products such as Gmail and Sheets, and developers can provide their own servers or functions.

Custom functions still follow the familiar pattern: the agent makes a call and the application returns a result. This is useful when a capability needs to run outside the agent's sandbox rather than on its computer.

Credentials are particularly easy to mishandle. I described three supported approaches: bearer tokens for APIs, OAuth for services and MCP tools, and environment variables for command-line utilities. The managed environment separates credentials from the sandbox using an external network proxy.

You create credentials outside the sandbox and define where they may be used, such as a particular domain. They are write-only: after setting one, you can replace it but cannot retrieve it. An environment variable visible to the agent contains a dummy value; on an allowed outbound request, the proxy replaces that value with the real credential.

I argued that domain-scoped substitution helps prevent a malicious prompt or MCP server from simply reading and sending the real token elsewhere. For example, an instruction to print an environment file would see the dummy value rather than the externally stored secret. The protection depends on the proxy and the configured credential restrictions.

Hooks provide another layer of house rules. A matcher can identify tool calls or commands that need monitoring or human approval, and the hook applies a deterministic check. Hook files are protected so that the agent cannot simply modify them to bypass the rule.

A coworker also needs a schedule. The available trigger I described is a cron schedule that runs the agent with a specified input in the same environment—for example, periodic due diligence or an audit. Webhooks were also being developed so that external services could trigger a workflow.

For the demo, I asked Gemini to help build a browser-based Google Sheets replica and simulated a scheduled trigger. A real Managed Agents API call told Buck to use the portfolio-audit skill. The agent read the skill, called financial services, checked changes in the market and stock prices, and updated the sheet.

The same trigger could run at 8:30 each morning. With Slack access, the agent could post a summary of what changed to a trading channel. I also showed a recommendation request that used the conversation's earlier context and discussed three positions while respecting the system instructions about not taking actions.

The infrastructure for this coworker is in the managed environment: Gemini provides the brain, the sandbox provides the computer, and skills and tools provide the playbooks and capabilities. Credentials, hooks and triggers supply controlled access, rules and proactive execution. That is the complete coworker I wanted to build.