BBERT Unearths Taxonomic and Functional Clues From 100bp Short Reads to Tackle the Challenge of Annotating Massive Soil Microbiome Data

Microbial ecosystems possess extremely complex compositional structures. Understanding this complexity is crucial for elucidating the various processes that sustain the biosphere, and this work relies on the analysis of massive sequencing datasets. Today, such data resources are abundant: thousands of publicly available metagenomic datasets covering diverse biomes on Earth can be freely accessed and downloaded.

Among these data, soil metagenomics holds particularly important research value. Shotgun metagenomic sequencing in complex environments such as soil can reveal exceptionally rich biodiversity. However, due to the lack of sufficiently comprehensive reference databases, most short reads generated by shotgun sequencing cannot obtain accurate taxonomic or functional annotations, making it difficult to comprehensively elucidate the true structure and function of microbial communities.

Against this backdrop, a research team from the Hebrew University of Jerusalem proposed BBERT, a nucleotide large language model optimized for short-read analysis. Researchers conducted tests on large-scale soil metagenomic datasets and found that BBERT can identify grammatical features of bacterial sequences without relying on reference databases, and accurately determine their taxonomic domain, coding potential, and reading frame directly from short reads as short as 100 bp. Furthermore, the BBERT model can directly convert short metagenomic reads into bacterial amino acid sequences, providing support for subsequent research and applications. BBERT also demonstrates advantages in de novo metagenomic assembly: it reduces mismatches and gaps during the assembly process while shortening runtime. Researchers further validated these findings using metagenomic data from root nodules of wild legumes, demonstrating that data filtering via BBERT not only significantly improves binning quality but also dramatically accelerates de novo assembly speed.

The related research results have been published in Nucleic Acids Research under the title "Fast and accurate taxonomic domain assignment of short metagenomic reads using BBERT."

Research Highlights:

• BBERT simultaneously overcomes the limitations of previous large language model-based methods in terms of throughput and accuracy.
• Using BBERT for pre-filtering prior to metagenomic analysis can improve functional annotation speed and de novo assembly quality, reduce assembly errors, and enhance binning quality.
• BBERT provides a rapid, reference-database-free taxonomic method for short reads, potentially unlocking the research value of existing large archives of metagenomic data.

Paper Link:

https://academic.oup.com/nar/article/54/19/gkag1002/8971853?login=false

Dataset: From Training on Bacterial Genomes to Validation in Complex Metagenomic Environments

During the development and evaluation of BBERT, researchers curated and constructed two types of datasets.

• The first type was used for model pre-training and includes genomic data from 1,416 bacterial species. To ensure the data adequately covered the bacterial phylogenetic tree, two genomes were randomly sampled from each bacterial family in the RefSeq database.
• The second type was used for model calibration and evaluation, comprising 899 bacterial species from 396 different genera, and 1,200 eukaryotic species from 823 different genera. Notably, these two datasets are completely separated at the genus taxonomic level, with no overlap in genera between them.

Simultaneously, to construct a curated reference genome dataset representative of eukaryotic diversity in soil environments, researchers first determined a target taxon list by reviewing 18S rRNA and ITS amplicon metabarcoding studies of soil microbial communities. Relevant studies included the Earth Microbiome Project, the pan-European LUCAS soil survey, and surveys of global soil protist and fungal diversity.

Through a comprehensive analysis of the aforementioned literature, eight major eukaryotic lineages consistently exhibiting high abundance in soil microbial communities were identified, namely:

• Cercozoa
• Ciliophora
• Oomycota (heterokonts)
• Glomeromycota (arbuscular mycorrhizal fungi)
• Mucoromycota
• Ascomycota
• Free-living Nematoda
• Amoebozoa

Additionally, the dataset included the Chlorophyta (green algae) relevant to soil environments, as well as terrestrial plant taxa for which genome sequencing data were already available.

To account for potential randomness during species grouping, all genome accessions in the calibration and evaluation datasets were divided into 20 equal groups at the genus taxonomic level, with 20 groups each for bacteria and eukaryotes. Within each group, species abundances were randomly assigned according to a log-normal distribution. Subsequently, groups from the two datasets were paired to form 20 combined subsets. Each subset possessed independent bacterial and eukaryotic genus compositions. Among these 20 subsets, the first 10 (Datasets 1–10) were used for model fine-tuning, while the remaining 10 (Datasets 11–20) were reserved for model evaluation.

BBERT Framework: Enabling Large Language Models to Learn Bacterial Features from DNA Sequences

BBERT is a model developed based on the BERT architecture, designed for out-of-distribution (OOD) detection of nucleotide sequences derived from bacterial genomes. Its core concept involves training exclusively on bacterial sequences to determine whether a given sequence is of bacterial origin. To construct a large language model capable of OOD detection for bacterial genomes based on nucleotide sequences, researchers adopted a three-stage development workflow:

• Pre-training the BERT architecture using only bacterial genome reads;
• Fine-tuning for the bacterial/non-bacterial classification task;
• Evaluating model performance on bacterial read classification tasks and comparing it with BERTax, Tiara, and REMME.

① Model Pre-training

The pre-training of BBERT utilized a bacterial dataset containing synthetic short nucleotide sequences generated through sampling, with each sequence being 100 bp in length, aimed at simulating metagenomic sequencing reads and potential sequencing errors. During training, researchers employed masked language modeling as the optimization objective, masking 30% of the tokens in the input sequences and optimizing the model's ability to infer these masked tokens based on context. Through this process, the model learned to represent missing nucleotides based on existing read information.

The optimization process for pre-training proceeded in stages of 5–10 epochs per stage until optimal results were achieved, while saving all intermediate checkpoints (model weights). Typically, a continuous decrease in training loss was observed alongside steadily improving classification accuracy. Training was terminated when progress plateaued and no significant improvement in model performance was observed.

② Model Fine-tuning

Next, to enable BBERT to perform bacterial read classification tasks, researchers added a lightweight classification head to the model: a linear layer was added on top of the token embeddings generated by the BBERT encoder, used to directly predict whether a read is of bacterial origin.

The classification head was trained via supervised learning. During training, the pre-trained BBERT model weights were not frozen; instead, the encoder and classification head were jointly fine-tuned, allowing the feature representations learned by the encoder to adapt to the current classification task. To prevent unstable impacts on the pre-trained weights, different learning rates were set for the classification head and the encoder: the encoder learning rate was 1×10⁻⁶, and the classification head learning rate was 1×10⁻⁵, using the AdamW optimizer.

The study found that the strategy of freezing the bottom two layers of the Transformer encoder while fine-tuning the remaining layers yielded better results. The model finally outputs the probability that a read is of bacterial origin, denoted as bact_prob; when bact_prob exceeds 0.5, the read is classified as bacterial.

③ Model Evaluation and Comparison

Finally, researchers calculated and reported overall classification performance metrics on the datasets, comparing the benchmark results with BERTax, Tiara, and REMME.

Results: BBERT’s Classification Accuracy, Functional Annotation, and Assembly Efficiency Were Verified

① BBERT Outperforms State-of-the-Art Deep Learning Classification Tools

First, in the bacterial and non-bacterial read classification task, BBERT demonstrated strong recognition capabilities—as shown in the figure below, BBERT outperformed BERTax, Tiara, and optimized REMME on standard classification metrics including accuracy, precision, recall (comparable to REMME), and AUROC. The average domain classification accuracy of BBERT across 10 test sets was 89.7% ± 1.7%, compared to 86.8% ± 1.7% for REMME, 79.2% ± 4.7% for BERTax, and 72.6% ± 3.4% for Tiara.

In terms of identifying coding sequences and their reading frames, BBERT’s performance surpassed REMME, which was the only comparable tool with this functionality (Figure E below, coding accuracy: BBERT 88.1% ± 0.9%, REMME 74.4% ± 2.1%; Figure F below, reading frame accuracy: BBERT 96.3% ± 0.3%, REMME 45.4% ± 0.4%).

In terms of computational performance, BBERT also demonstrates good practicality. On a compute node equipped with a dedicated NVIDIA H200 GPU, BBERT can process 45,912 reads per second (rps), whereas BERTax and Tiara achieve 413 rps and 347 rps, respectively. However, after code optimization, REMME is slightly faster, reaching 50,745 rps. When running on a Windows 10 personal computer equipped with an NVIDIA RTX 3090 GPU, BBERT achieves a processing speed of 15,373 rps. This means that users can perform analyses using BBERT even without relying on high-performance computing clusters, a capability that most similar tools struggle to provide.

② BBERT is a bacterial identification tool with high sensitivity

The researchers also applied BBERT to a curated set of 1,969 publicly available soil metagenomic datasets, using the number of prokaryotic bases estimated by SingleM as a reference, to further validate its performance on these real-world data lacking true taxonomic labels. As shown in the figure below, without filtering, the total number of bases in the metagenomic data was 31% higher than the number of prokaryotic bases estimated by SingleM (slope: 1.31±0.01); after filtering with BBERT, this difference narrowed to 9% (slope: 1.09±0.01).

Unlike SingleM, Kraken2 supports comparisons at the individual read level, so the researchers also directly compared BBERT with Kraken2. The results showed that for reads identified by Kraken2 as bacterial in origin, BBERT's judgments were largely consistent. Meanwhile, BBERT was able to identify a substantial portion of reads that Kraken2 could not classify (see figure below); conversely, there were very few reads that Kraken2 could identify but BBERT failed to identify.

Therefore, BBERT is a bacterial identification tool with high sensitivity, capable of classifying reads at the read level like Kraken2, while not being limited by incomplete reference database coverage.

③ BBERT can directly translate short DNA reads into amino acid sequences

Given that BBERT can identify bacterial coding reads and their reading frames, it can also directly translate short metagenomic reads into amino acid sequences. The study found that compared to annotating unfiltered DNA reads using BLASTx, this approach yields largely consistent annotation results but with faster processing speeds: approximately 2.6 times faster in fast mode and approximately 6.3 times faster in sensitive mode (see figure below). This means that BBERT can not only help researchers filter data but may also further optimize metagenomic functional annotation workflows.

④ BBERT accelerates metagenomic de novo assembly and reduces assembly errors

In terms of metagenomic assembly, the benefits of BBERT are more intuitive. The research team found that in unfiltered data, a large number of non-target sequences increase the assembly burden and may introduce erroneous gaps and mismatches. After filtering simulated metagenomic data with BBERT, the unweighted average bacterial genome coverage decreased from 68.23% to 65.22%, representing a coverage loss of approximately 4.4% (see figure below). At the same time, the assembly time was reduced from 66±8 hours to 25±2 hours, resulting in an efficiency improvement of approximately 2.7 times.

Assembly accuracy also improved. Calculated as the unweighted average across 512 genomes and normalized by reference genome length, the results show that the number of gap openings per 100 bp of reference sequence decreased from 0.51 to 0.36, a reduction of 0.15, representing a relative decrease of 29.3%; the number of mismatches decreased from 3.31 to 2.64, a reduction of 0.67, representing a relative decrease of 20.2%. Together, these two metrics decreased by approximately 21.4%.

⑤ Real Nodule Metagenomic Data

To evaluate the potential benefits of BBERT filtering in real datasets, researchers further applied it to metagenomic data from legume plant nodules. After BBERT filtering, the volume of the co-assembly result shrank by 61%, decreasing from 577.6 Mb to 222.4 Mb; the number of contigs decreased by 57%; and the GC content increased from 40.4% to 58.5%.

This change is consistent with the approximately 40% GC content of legume plant genomes and the approximately 62% GC content of Rhizobiales genomes, indicating that the majority of the filtered reads originated from the host. Additionally, BBERT filtering doubled the N50 value of the assembly from 0.9 kb to 1.8 kb, halved the assembly runtime, and reduced peak memory usage from 188 GB to 82 GB.

Conclusion

BBERT demonstrates another possibility for large language models to process short DNA sequences: even when facing 100 bp reads lacking reference database support, it can complete tasks such as domain classification, coding potential identification, and open reading frame prediction by learning the intrinsic features of the sequences. More importantly, this capability can not only be used to identify non-target sequences in metagenomes but also further improve downstream workflows such as functional annotation, genome assembly, and binning.

The study also suggests that BBERT's recognition bias for special sequences such as mobile genetic elements may reflect the distinct evolutionary characteristics of these sequences compared to host genomes. In the future, the sensitivity of large language models to unique DNA sequence patterns may provide new tools for tracing horizontal gene transfer events. As related models continue to develop, their applications are expected to expand from microbial taxonomy to broader ecological genomics research.

References:

https://phys.org/news/2026-10-ai-tool-scientists-bacteria-hidden.html

https://academic.oup.com/nar/article/54/19/gkag1002/8971853?login=false