AlphaFold 2 and the Protein-Folding Breakthrough: Why Lex Fridman Calls It a Landmark for AI and Biology

Fridman begins by noting that 2020 has been a hard year and that positive news from engineering and science is welcome, citing SpaceX alongside DeepMind's announcement that AlphaFold 2 had effectively solved the 50-year-old protein-folding grand challenge. Here, he explains, solved means that the computational prediction approaches roughly the performance of slower and far more expensive experimental methods such as X-ray crystallography.

The evidence is the CASP competition. In the 2018 edition, AlphaFold scored 58 on the hardest class of proteins, while this year it reached 87, a huge improvement that is also 26 points better than the nearest competitor. He cautions that the internet is full of hype, but believes this is a genuine leap.

Fridman, speaking as an AI researcher, says a strong case can be made that this is the biggest advance in structural biology in the past decade or two and one of the biggest in the recent history of AI. Many people compare it to the ImageNet or AlexNet moment, but he argues the ranking depends on how much weight you give direct real-world impact, since AlexNet was ultimately demonstrated on a toy-scale image classification problem.

He playfully lists the competition: AlphaZero and self-play reinforcement learning, transformers and GPT-3 in language, Tesla Autopilot and Waymo deploying machine learning in safety-critical systems, in-the-wild natural language processing in smart speakers, and Boston Dynamics' Spot as a hardware achievement. He also predicts at least one Nobel Prize will come from work derived from these methods, possibly with a machine learning system playing a central role.

Fridman gives a primer: amino acids are the building blocks, with 21 in eukaryotes, and proteins are chains of them that act as structural parts, catalysts and movers inside cells. Folding takes an amino acid sequence to a 3D structure. A sequence usually maps to one structure, the structure determines function, and many diseases are caused by misfolded proteins.

The search space is vast, which he illustrates with a figure of 10 to the 143 possible folds against roughly 10 to the 80 atoms in the universe, and points to Levinthal's paradox. He suspects the combinatorics exceed those of chess but notes biology is far weirder. On data, about 200 million proteins have been mapped but only roughly 170,000 have 3D structures, because X-ray crystallography reportedly costs around $120,000 and takes about a year per protein.

No AlphaFold 2 paper existed yet, only a blog post, so Fridman first explains AlphaFold 1. It had two steps: a convolutional network took amino acid sequences plus features such as multiple sequence alignments and output a distance matrix with confidence distributions between residues, and then a non-learned gradient-descent optimization folded a 3D structure to match those distances.

For AlphaFold 2 he offers clearly labelled speculation. The clear change is attention: convnets are out and transformers are in, echoing what has happened in language and much of deep learning. The multiple sequence alignment also seems to be part of learning rather than just a feature source, with iterative passing of information between the sequence representation and the residue-to-residue representation.

The blog's use of the term spatial graph rather than a distance matrix hints that more than pairwise distances may be involved, though he cannot say whether this is a clever generalization. He draws two lessons from recent deep learning history: attention and transformers bring a big boost, and making as much of the problem learnable as possible often pays off, which he has seen in computer vision segmentation.

Because structure gives function, predicting structures for perhaps millions of proteins could reveal the unknown functions of genes encoded in DNA and clarify the causes of diseases tied to misfolding. It could also speed up designing new proteins that alter other proteins' function, for example to fix misfolded ones. Fridman mentions papers that discuss insecticidal proteins, frost-protective coatings, tissue regeneration, supplements and new biomaterials, while admitting he knows little about those areas.

Looking further out, he sees the lasting impact as advancing end-to-end learning for very complicated life-science problems: predicting protein complexes, modelling the environment's effect on folding, and eventually physics-based simulation of biological systems from cells to organs. He jokes about simulating the organ producing the video and muses about biological and artificial life. For him it extends machine learning beyond games like StarCraft, chess and Go into real-world systems, alongside efforts like Tesla Autopilot.