A 2023 Google Memo on Open Models, LoRA and AI’s Competitive Moat

I review a leaked internal essay arguing that Google and OpenAI face a competitive challenge from rapidly improving open models. [SemiAnalysis](https://newsletter.semianalysis.com/p/google-we-have-no-moat-and-neither), which republished the document in May 2023, said it verified the source and explicitly described it as a Google employee’s opinion, not the position of the entire company. The memo’s forecast remains an argument, not a demonstrated outcome.

The author argues that while Google and OpenAI watch one another, a third group is advancing: the wider community building models such as Alpaca, Vicuna and GPT4All. I see the speed of iteration as striking. “Open source” is used broadly in the discussion; access to code, weights, training data and commercial-use rights varied between projects.

The memo points to compact models running on consumer devices, including a reported Pixel 6 example at five tokens per second, and to personal adaptation on a laptop. Those examples depend on the model, quantization and hardware. They do not mean every foundation model can run on any device or be trained from scratch on a laptop.

The author also notes unrestricted image models and a multimodal ScienceQA result, arguing that open communities are advancing quickly. I emphasize customization and accessibility. The existence of unrestricted releases does not settle questions about responsible deployment, safety, copyright or a particular license.

The memo contrasts small, inexpensive community experiments with expensive large-model development and recommends working with outside developers, enabling third-party integrations and taking smaller variants seriously. Those cost figures describe particular claims and exclude many upstream costs; they do not show that a $100 experiment reproduces a frontier model trained for millions.

A chart compares LLaMA, Alpaca, Vicuna, Bard and ChatGPT, placing Vicuna near 92% of ChatGPT in that illustration. [LMSYS](https://www.lmsys.org/blog/2023-03-30-vicuna) derived the figure from GPT-4 scores on 80 questions and cautioned that the approach was not a mature comprehensive evaluation. It is not a universal capability percentage or proof of parity on coding, mathematics or all other tasks.

Meta announced LLaMA in February 2023 and provided access to researchers under a noncommercial research license; its weights subsequently leaked. The base model was not instruction- or conversation-tuned in the way later derivatives were. Community projects then explored instruction tuning, quantization, evaluation, multimodality and other methods.

I mention GPT4All’s opt-in conversation sharing as an example of community data collection and call it reinforcement learning from human feedback. Collecting conversations can support later training, but it is not by itself RLHF: that method requires an appropriate feedback or preference process and a training procedure.

The memo argues that the barrier to experimentation fell sharply, allowing individuals to adapt models with relatively modest resources. That is an important change for experimentation, but it should not be confused with the compute, data and expertise required to pretrain the original foundation models.

The author compares the language-model moment with the earlier community around Stable Diffusion and contrasts it with DALL-E. I regard image-generation progress as evidence of fast iteration. Their relative quality and cultural impact are assessments, not universal benchmark results; my suggestion that Midjourney was built on Stable Diffusion is not established by the memo.

The memo highlights low-rank adaptation, or LoRA. Instead of updating every model weight, LoRA represents selected updates with low-rank matrices, reducing the number of trainable parameters. Depending on the setup this can reduce adaptation memory and cost, but exact savings vary and it does not make the original pretraining free ([LoRA paper](https://arxiv.org/abs/2106.09685)).

I contrast repeatedly adapting a smaller model with retraining a large model from scratch. The author argues that fast iteration can let smaller models benefit from new datasets and ideas more quickly. Whether that closes a particular quality gap depends on task, data, architecture and resources; it is a prediction rather than a general law.

I highlight data quality as an important ingredient and discuss the availability of public datasets, alongside emerging restrictions or prices for access to online platforms. High-quality data can be valuable, but saying quality always scales better than quantity is too broad; dataset rights, composition and scale all matter.

The memo argues that proprietary firms face competition from free alternatives and that researcher departures can spread technical knowledge. I describe the concern as “brain drain.” People’s expertise travels with them, but that does not imply they may lawfully disclose confidential information, or that competitors automatically know everything a former employer knows.

The essay notes that much early community activity was built on leaked LLaMA weights, while companies may face different license and compliance constraints. I suggest Meta could benefit from the ecosystem surrounding its model. However, owning a base model does not automatically grant unrestricted ownership or use rights over all outside datasets, code and derivatives; the applicable licenses matter.

I compare the strategic value of an ecosystem with Google’s roles around Android and Chromium. The memo’s broader point is that a company can benefit when external developers build on a shared platform. The governance and licensing arrangements of those projects differ and do not prove the same outcome for any model.

The author also worries that researcher movement can weaken the value of secrecy and urges Google to learn from external work. This is the memo author’s assessment, not evidence that a company’s confidential work is automatically shared whenever an employee moves.

I review the memo’s timeline: LLaMA’s February launch and early March leak, followed by Alpaca, quantization, Vicuna, GPT4All, Cerebras models, parameter-efficient fine-tuning, Koala and OpenAssistant. The dates capture rapid 2023 development, but each project had different capabilities and terms.

I describe Stanford’s Alpaca as laptop fine-tuning. In fact, [Stanford reported](https://crfm.stanford.edu/2023/03/13/alpaca.html) its initial Alpaca 7B fine-tuning used eight 80 GB A100 GPUs for three hours, while later community work demonstrated other lower-resource approaches. Its reported inexpensive adaptation did not include the cost of pretraining LLaMA.

I close by saying that open models were still behind GPT-4 in 2023, but I expect fast iteration to narrow the gap. That is my prediction from the time of the video. Progress depends on the task and evaluation, and the eventual competitive outcome cannot be inferred from the early timeline alone.