Building AI Characters With the a16z Companion App

I want to show you how to put together the [a16z Companion App](https://github.com/a16z-infra/companion-app), a starter project for characters you can chat with. You can give a character a personality and backstory and store conversation history. The repository describes it as a developer tutorial; the interfaces and integrations in this walkthrough reflect July 2023.

The sample app comes with several companions using hosted ChatGPT and Vicuna models. I was told a fully local version, including the databases, was on the authors’ roadmap, but that is a plan rather than something demonstrated here. I chat briefly with Alex, whose responses refer to painting and science-fiction books.

To get started, clone the project repository and change into its directory. The app combines several services: Next.js for the application, Clerk for authentication, Pinecone or Supabase for vector storage, LangChain for model orchestration, OpenAI or Replicate for text generation, and Upstash for conversation history. Fly and Twilio are listed for deployment and SMS, with some integrations optional.

After cloning, run `npm install` and open the folder in your editor. Rename or copy `.env.local.example` to `.env.local` and fill in the settings for the services you choose. I use Pinecone in this walkthrough; the example also offers Supabase, which I note can be hosted locally.

For authentication, create a Clerk application and choose sign-in methods. I select email and Google, then copy the publishable and secret keys into their corresponding environment variables, leaving the shown sign-in and sign-up paths as they are. These setup screens and options may look different now.

Next, add an OpenAI API key to the environment file. I use OpenAI here; the project also supports Vicuna 13B hosted on Replicate, which requires a Replicate token. The demonstration runs the web server locally, but these model calls and other selected services remain hosted.

Create a Pinecone index and supply its name, environment and API key in the environment file. The project’s setup specifies 1,536 dimensions for its example embeddings. The starter-plan details shown are historical; if using Supabase instead, configure that provider according to the project instructions.

For conversation storage, create an Upstash Redis database, select regions and the shown SSL and eviction settings, then copy its REST URL and token into the environment file. The video shows a hosted Redis service; using it does not make this a fully local stack.

The repository provides separate commands to generate embeddings for Pinecone and Supabase. I run the Pinecone command, which loads embeddings from the character information into the vector store, then start the development server with `npm run dev`. I open the local address in a browser and sign in through Clerk.

The local app displays Alex and Sebastian using ChatGPT, and Rosie and Lucky using Vicuna 13B. I say conversation history is stored even though the interface does not display all of it at this point. The repository explains that a conversation queue and retrieved character information provide context; storage should not be confused with unlimited model memory.

To change a personality, open its text file under `companions`. Alex’s file describes a fictional character who enjoys painting, programming and science fiction. You can edit that description or create a new personality, then update `companions.json` with metadata such as the image, model and optional phone number.

The project includes a Twilio integration for texting a character, but I could not get it working: my attempts returned a 401 error. I plan to keep investigating and add an update if I solve it. The browser-based companion is the part I successfully demonstrate here.