Xie Qingchi’s AI Paper Reading Route: From GPUs to Multimodal Models

Zhang Xiaojun invites Xie Qingchi, a product lead at Meituan’s Light Year, to share his journey through AI papers and a learning route for understanding technical developments.

Xie studied computer science as an undergraduate and graduate student before working in product and startups. He reads papers to understand how technical boundaries shape product choices.

He recalls revisiting foundations such as linear algebra and probability and using translation tools to lower the English-reading barrier. This is his experience, not a prescription for everyone.

He suggests using AI to explain terminology and equations, while checking generated explanations against the original papers and other reliable sources because they can be wrong.

At greater depth, mathematics helps with equations and notation. When a paper is difficult, readers can start with the problem it addresses and gradually learn the details.

The first section traces changes in model paradigms, connecting GPU computing, vision, sequence modeling, Transformers and later methods on a timeline.

He describes how early work such as Brook made GPU computation more accessible and reduced barriers to programming graphics hardware directly.

On AlexNet, he identifies growth in data, compute and model scale as important conditions, illustrating the influence of hardware and datasets on research.

Sequence-to-sequence models encode inputs and then generate outputs. Earlier methods could be limited by fixed-length representations; attention provides a way to access different parts of the input.

He introduces knowledge distillation: a student model can learn a teacher’s output distribution rather than only final labels. Results depend on training data and task.

Residual connections in ResNet provide a way to train deeper networks and alleviate optimization degradation; they do not guarantee that arbitrary models improve with depth.

On Transformers, he emphasizes that architecture and engineering evolve together and that designs require experiments; researchers may not fully explain every choice.

He introduces AlphaGo Zero’s self-play and reinforcement-learning approach, showing how a system can improve through computation in a specific rule-governed setting.

He next introduces mixture-of-experts models, which can activate parts of a model for different inputs and trade off total parameter scale against per-step computation.

On chain-of-thought prompting, he considers how examples or requests for intermediate steps can change outputs; that does not guarantee every generated step is correct.

LoRA adapts a model by training smaller low-rank updates, reducing the number of parameters updated. Performance still depends on model, task and training setup.

He describes ReAct’s approach of interleaving reasoning and action. Tool use can give models external information, but reliability and permissions remain system concerns.

The Bitter Lesson emphasizes the long-term role of scalable general methods and compute. Xie says hand-designed features can be useful at one point and displaced as computing conditions change.

The second section turns to infrastructure and data: larger models raise issues of memory, parallelism, data processing and cluster reliability as well as algorithms.

Using ZeRO, he explains distributed training: when a single GPU cannot hold all training state, parts can be partitioned across devices.

He discusses scaling laws and Chinchilla: under a limited compute budget, model size and training data must be considered together. Results from different experiments should not be applied mechanically.

Open-data work such as LAION shows how datasets and community resources can broaden research participation; provenance, quality and rights still need careful treatment.

He discusses web-data cleaning and RefinedWeb, explaining that large-scale data requires filtering, deduplication and quality control as well as volume.

Large GPU clusters must also manage communication efficiency and failures: adding devices does not necessarily increase speed linearly, and faulty hardware or data can disrupt training.

The third section follows language models from word vectors and neural machine translation to GPT, BERT and post-training.

Word2Vec studies how to represent words as vectors useful for downstream tasks. Vector relationships are learned representations, not complete explanations of meaning.

Large-scale neural machine translation illustrates the engineering challenge of combining sequence models, attention and parallel computing into an online service.

He introduces GPT-1’s decoder architecture and next-token prediction, discussing pretraining followed by adaptation to tasks.

BERT used a different pretraining approach and advanced language-understanding tasks. Xie places it among several parallel research directions of the time.

He recalls GPT-2’s exploration of larger, more varied data and researchers’ interest in performing tasks without separate fine-tuning for each one.

GPT-3 scaled further and spurred research on few-shot prompting and changes in capabilities; relationships between scale and performance must be tested by task.

InstructGPT and later work used methods including human feedback in post-training to better follow instructions; alignment does not mean a model cannot make mistakes.

He mentions open post-training research such as Tulu 3, emphasizing that public recipes and data aid the community while results still depend on the setup.

The fourth section traces multimodal research from video recognition and generative adversarial networks to diffusion, vision Transformers, CLIP and text-to-image generation.

Early deep-video research explored neural networks for video and required task-appropriate datasets and computing resources.

He introduces the generator and discriminator in generative adversarial networks and the instability that can arise when training them together.

Diffusion models build on adding noise gradually and learning a reverse generation process. They later became widely used for images, with quality and efficiency varying by implementation.

Vision Transformers divide images into patches and turn them into sequence representations for a Transformer, allowing vision tasks to draw on related architectures and training experience.

CLIP learns related representations from image-text pairs, providing a way to connect language and images and influencing later generation systems.

He explains how text-conditioned diffusion uses language to influence generation and how latent-space methods reduce the cost of operating directly in pixel space.

DiT explores using a Transformer as the core network of a diffusion model, connecting vision-Transformer and generative-model research.

After the paper survey, he recalls rereading and checking the material and says returning to original questions helps reveal relationships among papers.

He is optimistic about room for further technical progress while acknowledging that specific advances and timing are uncertain. History can improve questions, not eliminate uncertainty.

On team size, he says tools may reduce some collaboration and development costs, but staffing still depends on the business, organization and scope of service.