AlphaFold 3 Is Here! Closed Source but Available for Scientific Research. Is DeepMind's Subsidiary the First to Promote Commercial Use?

On May 8th local time, Google DeepMind and its subsidiary Isomorphic Labs jointly released AlphaFold 3.

DeepMind said that AlphaFold 3 successfully predicted the structure and interaction of all life molecules (proteins, DNA, RNA, ligands, etc.) with unprecedented accuracy. Compared with existing prediction methods,AlphaFold 3 found at least 50% more interactions between proteins and other molecular types, and for some important interaction categories, its prediction accuracy doubled.

Readers who follow AI for Science are no strangers to AlphaFold, which was praised as a "milestone breakthrough" when it was first released. AlphaFold 2 was named "Breakthrough of the Year" by Science and "Method of the Year" by Nature in 2021, leading the way and constantly breaking the application limits of AI + biology.

Today, AlphaFold 3 has once again refreshed people's cognition. How did it achieve "all molecules of life are predictable"? What breakthroughs have been made in technology and performance? What role does Isomorphic Labs, which was born out of DeepMind, play?

Pairformer module replaces Evoformer, diffusion module replaces structure module

The researchers made significant improvements to the structure and training procedures of AlphaFold 2 to accommodate more general chemical structures and improve the data efficiency of model learning. Now, given a list of molecular inputs, AlphaFold 3 can generate their joint 3D structure and reveal how they fit together.

As shown in the figure above, the overall structure of AlphaFold 3 is similar to AlphaFold 2, both of which have a large backbone for evolving pairing representations of chemical complexes. AlphaFold 3 replaces AlphaFold 2's Evoformer with a simpler Pairformer module.This reduces the amount of multiple sequence alignment (MSA) processing and the number of MSA modules is reduced to 4. AF3's MSA representation processing uses the cheaper pair-weighted averaging method, and only the pair-wise representation is used in subsequent processing steps.

The researchers observed in AlphaFold 2 that removing most of the complexity of the structure module had little effect on prediction accuracy, while retaining the backbone framework and side chain torsion representation increased the complexity of general molecular graphs.

So in AlphaFold 3,Engineers used the Diffusion Module to directly predict raw atomic coordinates, replacing the AlphaFold 2 structure module that operates based on amino acid-specific frameworks and side chain torsion angles.The latter uses a pairwise representation to generate unambiguous atomic positions. The multiscale nature of the diffusion process (low noise level induced network improves local structure) also enables the elimination of stereochemical losses and special treatment of bonding patterns in the network, making it easy to cope with arbitrary chemical compositions.

Specifically,The diffusion module is able to operate directly on raw atomic coordinates and coarse abstract labeling representations, without the need for rotation frames or any isovariant processing.The researchers first trained the diffusion model to receive "noisy" atomic coordinates and then predict the true coordinates. This process requires the model to learn protein structures at various length scales, where the denoising task under small noise emphasizes understanding very local stereochemistry, while the denoising task under high noise emphasizes the large-scale structure of the system.

At inference time, the model first samples random noise and then performs cyclic denoising to generate the final structure. It is worth mentioning that AlphaFold 3 is a generative training procedure that produces a distribution of answers. This means that for each answer, even if the model is uncertain about its position, it is also able to determine its local structure (such as the geometry of the side chain bonds). Therefore,AlphaFold 3 avoids both torsion-based parameterization of residues and structural violation penalties while also handling the complexity of general ligands.

As shown in the figure above, in the performance comparison of predicting Protein-dsDNA (protein-double-stranded DNA interaction), AlphaFold 3 has a success rate of 64.8%, while RosettaAlphaFold2NA has only 28.3%. In the prediction of Protein-Antibody (interaction between protein and antibody), AlphaFold 3 has an accurate success rate of 62.9%, while other systems have only 29.6%.

* RosettaAlphaFold2NA combines Rosetta's classic modeling technology with AlphaFold 2.

AlphaFold's growth history: an excellent student with four consecutive leaps in 6 years

After AlphaGo defeated the international Go master Lee Sedol, the "Alpha series" officially came into people's attention. According to DeepMind, in 2016, almost immediately after AlphaGo became famous, the team began researching protein folding problems.

At the 13th CASP (Critical Assessment of protein Structure Prediction) at the end of 2018,AlphaFold came first among 98 contestants, accurately predicting the structures of 25 out of 43 proteins.The second-place contestant in the same group only correctly predicted 3 options.

At that time, the "first generation" AlphaFold had already demonstrated amazing strength. AlphaFold 1 was trained based on thousands of known proteins, using neural networks to predict the distances between amino acid pairs and the angles between the chemical bonds connecting them, and then adjusting the preliminary structure to find the most energy-efficient arrangement.

But the team found that AlphaFold 1's approach, which combines local physics with guide potentials derived from pattern recognition, tends to over-consider interactions between residues that are close together in the sequence, compared to interactions between residues that are far apart along the chain. As a result, AlphaFold 1 tends to choose models with slightly more secondary structure (alpha helices and beta sheets) than is actually the case (a kind of overfitting).

From a technical perspective, the software design used by AlphaFold 1 consists of multiple modules, each of which is trained individually to generate guided potential and then combined with physics-based energy potential.

Therefore, AlphaFold 2, released in 2020, combines sub-networks with a single differentiable end-to-end model.The system is entirely based on pattern recognition and is trained in an ensemble fashion as a single ensemble structure.

AlphaFold 2 achieved highly accurate predictions of protein monomer structures, and the DeepMind team then further turned its attention to the prediction of complexes. In October 2021, DeepMind released an update called AlphaFold-Multimer.It is an expansion of AlphaFold 2 and can model complexes of various proteins.

One-click deployment of AlphaFold 2 tutorial:

https://openbayes.com/console/public/tutorials/m6k2bdSu30C

The researchers tested 4,433 protein complexes and examined the prediction accuracy of AlphaFold-Multimer at the contact interfaces of heteropolymers and homopolymers, reaching 67% and 69% respectively, with highly accurate predictions accounting for 23% and 34% respectively.

Subsequently, AlphaFold, which had been silent for two years, once again amazed everyone. In addition to further improving the accuracy of protein structure prediction, it also added RAN prediction capabilities. On the last day of October 2023,DeepMind released the latest progress of AlphaFold (the paper called it AlphaFold-latest, which now seems to be AlphaFold 3).

Paper address:https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/a-glimpse-of-the-next-generation-of-alphafold/alphafold_latest_oct2023.pdf

DeepMind said that the new generation of AlphaFold model can predict almost all molecules in the Protein Database (PDB) with prediction accuracy at the atomic level, which not only opens up a new understanding of multiple key biological macromolecule categories, but also significantly improves the prediction accuracy. These biological macromolecule categories include ligands (small molecules), proteins, nucleic acids (DNA and RNA), and biological macromolecules with post-translational modifications (PTMs).

After much anticipation, the new generation of AlphaFold 3, which has whetted everyone's appetite since the end of 2023, has finally appeared. Its capabilities obviously did not disappoint the industry and academia, but there are still more important matters that require joint efforts from all parties to promote -How to get AlphaFold 3 out of the laboratory and into the pharmaceutical production line, how to enable more scientific research teams to use this advanced tool to optimize the research process, etc.Continued attention and investment are still needed.

DeepMind subsidiary Isomorphic Labs emerges

It is worth noting that a team that cannot be ignored also appeared in the release of AlphaFold 3 - Isomorphic Labs.

This company, which was born out of DeepMind, was founded in November 2021.Its name is inspired by the potential isomorphic mapping between biology and information science. Relying on AlphaFold, Isomorphic Labs focuses on the field of AI medicine, with the mission of using artificial intelligence and machine learning methods to accelerate and improve the drug discovery process in order to find treatments for some of the most devastating diseases in humanity.

In May 2022, Isomorphic Labs announced the first phase of its management team, which is star-studded.

Demis Hassabis, known as the "Father of AlphaGo", serves as the company's CEO; Chief Scientific Officer is Miles Congreve, who has participated in the design of 20 clinically evaluated drugs and is the co-inventor of Kisqali® (Ribociclib), a breast cancer treatment drug that has been marketed; Chief Technology Officer Sergei Yakneen has more than 20 years of rich experience in engineering, machine learning, products, life sciences and medical research; Max Jaderberg, a PhD from Oxford University, serves as the Director of Machine Learning...

London-based Isomorphic has been around for about two years, with no news of any work beyond the initial official announcement of the management team. It wasn’t until an exclusive interview with Endpoints News in 2023 that Demis Hassabis first revealed his vision for Isomorphic.

In the interview, when talking about the company’s main work since November 2021, Demis Hassabis said,“We’re focusing first on the small molecule space, which is where we can make the most impact quickly. Then we’re also very interested in the interaction space—protein-ligand interactions, protein-protein interactions, and the dynamic nature of biology. And then also going into the chemical space, understanding compound structures, how they bind to targets, binding affinity, etc.”

Regarding whether it is currently working with industry to build a pipeline, Demis Hassabis said, "We will select targets and programs that we think are particularly suitable for our technology roadmap and collaborate with large pharmaceutical companies on interesting targets. More information can be disclosed at a later time."

In January 2024, Isomorphic Labs announced two $3 billion drug development agreements with Eli Lilly and Novartis. Demis Hassabis revealed that "although Isomorphic had previously held preliminary talks with other large pharmaceutical companies, Eli Lilly and Novartis have great sincerity in cooperation." At the same time, the primary goal of this cooperation is clear: to manufacture drugs, rather than being limited to pilot projects or academic cooperation.

Specifically, the collaboration with Eli Lilly involves the development of treatments for a variety of disease-related proteins and pathways. In this regard, Demis Hassabis emphasized: "Applying our proprietary technology platform, next-generation AlphaFold, and large-scale computing capabilities to Eli Lilly's development projects will jointly advance breakthrough drug design."

The collaboration with Novartis focuses on discovering small molecule therapeutics against three undisclosed targets. Fiona Marshall, President of Novartis BioMedical Research, said: "This collaboration combines the unique strengths of both companies, from artificial intelligence and data science to medicinal chemistry and deep disease area expertise, to advance new possibilities in AI-driven drug discovery."

Today, the release of AlphaFold 3 is bound to further enhance the technical strength of Isomorphic Labs, but from the application perspective, the investment boom in AI pharmaceuticals has declined in recent years. Faced with high R&D costs, how to transform advanced technology into actual output is an important issue related to the future direction of the company. With the strong support of DeepMind, we also hope to see AlphaFold 3 truly landed in the industry as soon as possible.

One more thing: AlphaFold Server

AlphaFold 3 is the first AI system to surpass physics-based biomolecular structure prediction tools, and there are currently no plans to open source the complete code.However, the research team released a public interface for the model, the AlphaFold Server, which supports non-commercial research, opening the door to researchers around the world.

Visit AlphaFold Server official website:

alphafoldserver.com

With just a few clicks of the mouse, biologists can use AlphaFold 3 to model structures composed of proteins, DNA, RNA, and selected ligands, ions, and chemical modifications, and predict how proteins interact with other molecules in the cell. The platform can help scientists come up with novel hypotheses to test in the lab, speeding up workflows and can be used by scientists regardless of whether they have sufficient computing resources or expertise in machine learning.

In this regard, Céline Bouchoux, a research scientist in the Uhlmann laboratory at the Francis Crick Institute, praised: "With AlphaFold Server, it is no longer just about predicting structures, but generously providing access: allowing researchers to ask bold questions and accelerate discovery."

There is no doubt that the advent of AlphaFold 3 is not only a major leap forward in scientific exploration, but also a key tool for opening a new era of biomedical research and development. It has made significant breakthroughs in simulating many different types of molecular interactions, which is crucial for research and development projects such as accurately determining drug targets.

The DeepMind team is full of expectations for it: "We have just begun to explore the potential of AlphaFold 3 and can't wait to see what happens in the future."

References:
1.https://cloud.tencent.com/developer/article/2017961
2.https://hub.baai.ac.cn/view/31181
3.https://zh.wikipedia.org/wiki/AlphaFold
4.https://mp.weixin.qq.com/s/18cNw-E-5vU3vKb1J4WWKg