Trying an Open-Source Answer Engine with Groq and Ollama

RÉSUMÉ

En 30 secondes

In this 2024 tutorial, Matthew Berman runs Developers Digest’s Next.js answer engine locally with Groq-hosted Mixtral, OpenAI embeddings, Brave Search and Serper. A ramen query produces an answer with sources, images and videos. His attempt to replace hosted inference with Ollama reaches the local endpoint but does not produce a working answer in the interface. The app code is open source, while the demonstrated setup uses external APIs that may have costs and send data to providers.

1Answer-engine interfaceBerman compares Perplexity with the open-source project using a ramen query.
2External servicesThe working setup combines Groq inference, OpenAI embeddings, Brave Search and Serper.
3InstallationHe clones the repository, installs dependencies, configures API credentials and starts the development server.
4Local-model attemptHe downloads a Mistral model using Ollama and tries directing the application at its local endpoint.
5Unresolved resultOllama appears to receive requests, but the interface never displays an answer during the demonstration.

À retenir

The answer engine combines search results with generated answers, sources, images and videos.
The working 2024 demonstration runs the app locally but uses hosted inference, embeddings and search services.
Berman’s attempted Ollama integration did not produce a working answer in the video.
A local application is not automatically free or private when it calls external providers.
Check the project’s current documentation and provider terms before using historical setup steps.

ThèmesSearch & retrieval Local inference Open Source RAG et connaissances AI cost & performance

TEXTE INTÉGRAL

Lire le texte remanié, chapitre par chapitre

Afficher la transcription

Transcription seule

A search and answer demonstration

Berman compares Perplexity with Answer Engine and shows the latter returning a response and source media.

Perplexity answers a search query by assembling a response and citing sources, instead of presenting only a list of links. I demonstrate with a question about making ramen, which returns instructions and images. I then try a similar open-source project called Answer Engine.

Running the application on localhost, I submit the same query. It displays an answer, links to sources, images and playable videos. In this working example, the app runs locally but uses Mixtral through Groq for inference, along with other online services.

DIAPOSITIVE 1 / 1Answer Engine search resultsThe application displays a generated answer, cited sources, related videos and images for a ramen query.

The project is by Developers Digest and uses Next.js, the Vercel AI SDK, LangChain, Groq, OpenAI embeddings, Brave Search and Serper. The source code is available, but open-source application code does not make every dependency open source, local or free. The example also does not independently verify the accuracy of every generated statement or citation.

Installing the application

He clones the repository, installs packages, configures provider keys and starts a local development server.

I clone the repository, enter its directory and install the dependencies with `bun install`. I open it in VS Code and configure the environment file with provider credentials for inference, embeddings and search. API keys should be kept private; the video’s screens show credentials, so avoid copying or sharing them.

I start the development server with `bun run dev`, open localhost and repeat the ramen query. The app returns results in this configuration. I call it free in the video, but third-party usage limits, provider charges and local computing costs depend on the services and accounts used.

Trying Ollama locally

Berman attempts to replace hosted model inference with Ollama. The attempt remains unresolved in the video.

Next I try to run the language model locally with Ollama. I download a Mistral model and change the application’s model endpoint to Ollama’s local OpenAI-compatible address. The 2024 setup still uses OpenAI embeddings and online search services, so replacing inference alone would not make the entire workflow local.

When I try the local setup, Ollama appears to receive the request, but the application does not display an answer. I restart and modify the model references, then retry; the issue remains unresolved. I speculate about compatibility or the model’s response, but the video does not establish the cause or a working fix.

During troubleshooting I bind Ollama to all network interfaces. That can expose a service beyond localhost and is not necessary to assume for a local-only setup; access controls and network configuration matter. I invite others to investigate the integration. The project and its dependencies may have changed since this 2024 demonstration, so consult the current repository instructions.