How Factory Separates AI Agents from the Compute They Run On

Nikhil Harithas leads forward-deployed engineering at Factory. He describes the work as a feedback loop: learn what makes deployment difficult for complex customers and use that information to guide product development.

His example is Blackstone, a large financial institution with regulatory and security constraints. Harithas says Factory’s defaults did not fit: the customer wanted its own compute, inference and identity, with single-tenant deployment.

Factory’s coding agent, Droid, can receive a delegated task through tools such as Slack or Jira and return a pull request. That requires permissions, so the enterprise design repeatedly raised two questions: on whose behalf does the agent act, and where does it run?

Harithas argues that agents should not indefinitely borrow an individual’s identity. He frames agents as applications that can use established software and identity practices, adapted to their less deterministic behavior.

Different customers wanted Kubernetes, EC2, other clouds or sandbox providers. Factory’s earlier bring-your-own-compute approach installed an agent on a machine but left customers to manage the machine’s lifecycle, which proved complicated.

Factory’s proposed answer is Open Compute, a project intended to abstract that lifecycle for arbitrary agents. At the time of the talk it was in private preview, with an open-source release planned for the following weeks.

The goal is to let the customer choose where the workload runs and apply its own security constraints inside its VPC, without necessarily moving every part of Factory’s control layer on premises. Harithas describes this as separating a compute plane from the control and data planes.

In the simplified architecture, a client sends a session request to Factory’s backend. A compute gateway asks an Open Compute controller in the customer environment to provision a machine, while a relay carries communication between the client and the agent daemon.

The controller calls a driver that wraps the chosen compute provider. An EC2 driver can launch an instance; other drivers can target Kubernetes or a sandbox provider. The agent need not implement a separate lifecycle for each environment.

The provisioned machine establishes an outbound WebSocket connection to Factory’s relay. Harithas says this avoids requiring an inbound connection into the customer environment after setup, a distinction that matters to security teams.

The client communicates through the relay to the agent running in the customer’s environment. Model calls can be configured separately, including a bring-your-own-key arrangement with the customer’s inference service.

Harithas says this gives customers control over the agent execution environment without deploying the entire Factory product on premises. He presents customer ownership of models, evaluations and compute as a longer-term direction, rather than claiming every component is already customer-hosted.

The planned interface treats the computer as the unit of work. It supports lifecycle operations such as creating, pausing, resuming and destroying compute, using templates that define installed software and build requirements.

Snapshots and forks matter for longer or delegated work. A session can preserve state, branch it and hand a subtask to another agent. Factory also envisions customer-controlled snapshot storage and integration with the customer’s secret store.

Harithas stresses that the project manages computers rather than Factory-specific sessions, making the abstraction applicable beyond its own product. Drivers implement the provider-specific side of the interface.

Provisioning is more than one API call: acknowledgments may be lost, providers throttle requests and processes restart. A controller must keep track of desired state, retry and route users to the correct machine.

Long-running work adds another challenge. Harithas describes replacing a time-limited sandbox by snapshotting and restoring it while a multi-day agent mission continues. He also discusses distributing versioned infrastructure packages during the private preview.

A precise driver specification can make new compute backends easier to add and test. Harithas closes with the goal of allowing agents to run across customers’ preferred environments without tying the agent itself to one provider.