In 30 seconds
First Person Labs co-founder Ken Liu wants a personal context store that lets different agents use a person’s preferences and ongoing work without repeatedly asking for them. Second combines permitted computer activity and connectors, and Liu describes local storage, encrypted cloud options and confidential inference as ways to protect the resulting data. The hosts press on a separate problem: a model can still be tricked into disclosing information, which Liu says cannot be definitively solved. Continuous analysis is also costly, so he expects different models and analysis frequencies to serve different tasks.
- 1A shared personal model can travel between agentsLiu says Second can provide preferences such as travel needs to an existing agent instead of making users answer the same questions repeatedly.
- 2Connecting events over time is harder than sending notificationsLiu distinguishes reactive alerts from reflective help that combines earlier goals with current activity and preferences.
- 3Protecting storage and inference requires separate choicesHe describes local data and an optional encrypted cloud design, alongside a choice of inference providers and confidential computing.
- 4Prompt injection remains a separate limit on data sharingLiu distinguishes deterministic restrictions from model reasoning and says adversarial instructions are not definitively solved.
- 5Round-the-clock analysis can be expensiveA hypothetical ten-agent calculation produces widely different daily costs depending on the model; Liu expects a mix of models and frequencies.
- 6Other applications can query permissioned personal contextLiu describes interfaces for recent activity, reminders and priorities, and expects developers to build their own applications on top.
Takeaways
- A shared personal context can reduce repeated explanations when moving between specialized agents.
- Encrypted storage and confidential inference address different risks from an agent being tricked into revealing data.
- Liu says prompt injection cannot be definitively solved; deterministic restrictions remain necessary alongside model reasoning.
- Continuous analysis trades richer, faster context against inference cost.
Read the edited text by chapter
A portable model of the user
Liu describes a shared context layer that other agents can consult instead of relearning preferences.
Ken Liu says computers already contain a great deal of context about their users. His proposed Second system builds a model of a person's preferences, commitments and goals, then makes that context available to other agents the person chooses to use.
A host asks how that would help an existing agent book a flight. Liu says it could provide details such as a traveler's preferred seat, class and arrival time, avoiding repeated questions. The aim is portable context rather than starting over with each tool.
One host describes switching back to Codex after using Second and asks whether it could bring over earlier work. Liu says the coding agent could query Second for the relevant context.
Liu sees a personal model as something built in stages: first a record of artifacts, relationships and preferences, then a deeper understanding of how a person communicates. He imagines it becoming more representative of its user over time.
He compares the intended relationship to Batman's butler Alfred: someone who understands the principal's wishes and discretion well enough to act without having every preference restated. That trust depends on a good model of the person.
From computer activity to proactive help
The system analyzes digital events and connects them over time; early users describe unprompted reminders.
The hosts say an early version impressed them not just by recalling preferences but by connecting digital channels. Liu says Second analyzes a stream of computer events, such as application switches and typing, alongside information from connectors.
Those events do not arrive with a ready-made explanation. Liu describes building an understanding in layers: activity in a messaging application might be a conversation, while movement between a browser and a coding tool might reflect research. He argues that much of this context already exists on the computer.
An early user describes an unsolicited weather update while planning an event: Second monitored the forecast and noted when a tent might be needed. Another host says his own assistant has surfaced relevant details he might have forgotten.
Liu distinguishes simple reactive notifications from what he calls reflective help. The latter connects a goal mentioned months earlier with current work, priorities and preferences. He hopes such connections can lead to useful interventions without a separate request each time.
A host restates the product as a personal context store that can interact with several specialized agents. He asks whether users will eventually need a separate Second for every domain of life.
Liu expects agents to specialize, including some that focus on shopping or particular services. He envisions Second as a single trusted store for personal context, with temporary subagents created as needed instead of multiple competing personal vaults.
Protecting the context store
Liu discusses local storage, encrypted cloud storage and confidential inference, then distinguishes those protections from model decisions.
Joel de la Garza raises a security concern: a system that sees so much of a person's activity requires a different level of trust. He asks how the product is built and where its data goes.
Liu says ambient AI calls for choices about both stored data and model inference. He describes a local option in which a user chooses an inference provider and supplies their own tokens while activity data remains on the laptop; requests still go to the chosen provider.
The company also discusses confidential inference, in which protected hardware runs a model and can provide evidence about its execution environment. Liu contrasts this with a provider's zero-data-retention policy, which is a promise backed by policy or legal obligations rather than the same hardware mechanism.
The hosts ask whether the protection can be checked technically. Liu says this involves checking hardware and signatures, and expects confidential inference to become more common for personal data. The guarantees depend on the system and providers actually used.
For storage, Liu describes local data as one option and an encrypted cloud backup as another. In the latter design, data can be encrypted before upload and decrypted for an authorized user or in an attested secure environment.
A host asks a harder question: even if storage and transport are protected, how does the model avoid disclosing a secret to someone pretending to have permission? De la Garza recounts asking his own Second what Joel was hiding.
Liu separates deterministic safeguards from decisions made through model reasoning. He says sensitive system passwords can be excluded by design, while other sharing decisions may depend on context and policies that cannot enumerate every situation.
Prompt injection and inference cost
The participants discuss the limits of reasoning-based access controls and the expense of continuous analysis.
The hosts press on prompt injection, such as an instruction disguised as a play that tries to coax the model into revealing information. Liu says it is not a problem that can be definitively solved and expects better reasoning to improve the defense.
The product is intended to work with different models, Liu says, including eventually local inference. Some tasks require stronger models, but he does not want the personal context to be tied permanently to one provider.
Continuous analysis raises a cost question. Liu offers a hypothetical of ten agents each producing 60 tokens per second around the clock: at the prices he cites, one model could cost about $1,000 a day and a cheaper one about $60. He says early users may be busy professionals rather than everyone.
A host asks whether a lighter service could reduce the bill. Liu says there is a trade-off between how deeply and frequently the system analyzes a person’s history in advance and how much work a model must do later when answering a question from raw logs.
Liu says the product is available but the team needs to expand inference capacity and support before rolling it out broadly. Because it handles unusually personal data, he wants the rollout to keep user control and security in view.
Continuity and a mix of models
Liu describes development and cross-device uses, with model choice depending on the complexity and frequency of each task.
For the company's own development work, Liu says accumulated context has made coding agents more useful: he can share a screenshot and a short instruction instead of explaining the entire task. He also describes starting work on a laptop and continuing on a phone with the same context.
The hosts ask whether every operation needs an expensive frontier model. Liu says recognizing a calendar move may require much less capability than inferring a person's goals for the week. The latter can also run less frequently.
He expects the system to mix models according to task and cost, with retrieval handled separately from deeper reasoning. Designing for portability from the start should let the team swap in new models as their capabilities and prices change.
Building applications on personal context
The application gathers permissioned activity and exposes interfaces for other tools and applications.
A user installs an application, grants permissions and specifies tokens, Liu says. Connectors to services such as messaging apps can add information, but the computer's activity stream is also part of the input. Background agents then analyze events.
The resulting interfaces can expose recent activity, upcoming priorities, reminders and relationships to an authorized agent. Liu imagines developers building applications on those interfaces rather than making one chat screen the only way to use the system.
As one example, he describes a designer whose button and color preferences were captured in Second; a personal application could then draw on them. Liu compares the search for useful applications to early personal computers before spreadsheets gave many people a compelling reason to use them.
An assistant across devices
The participants imagine more natural interfaces while describing the limitations of the current API-based approach.
The hosts ask about a more visible, embodied assistant. Liu says the familiar science-fiction ideal depends not only on intelligence but on continuity across devices. He imagines different devices eventually becoming inputs and outputs for the same personal context.
For now, a host asks how to use Second from an iPad in the kitchen. Liu says an internet-connected agent that can call its cloud API can access it; he describes native applications and possible devices as later possibilities rather than current features.
A small interface over a difficult system
Liu says the company narrowed its own product surfaces to focus on the context and infrastructure problems.
The discussion turns to the tension between infrastructure security and a pleasant interface. Liu says the team initially tried to build more of the interaction layer but found the details difficult and time-consuming.
First Person Labs narrowed its own surfaces to text and an API so others could build experiences on the context layer. Liu says the software can technically act as a laptop agent, but the team wants to keep its own interaction model minimal while improving the underlying system.
Liu says Second is available at 2.computer and that the team hopes to grant access as capacity permits. He invites people interested in the work to try it or join the company.
More from this channel
16:19EN
1:03:18EN
46:06EN
46:11EN
56:06EN
59:06EN