t-SNE plot of embeddings from GeneEmbed (Splice Sites Donors Dataset)

About

Understanding and analyzing genetic sequences is a fundamental challenge in computational biology. With the vast amount of genomic data being generated, researchers need powerful tools to transform raw nucleotide sequences into meaningful numerical representations. GeneEmbed is an advanced nucleotide embedding model designed to convert genetic sequences into fixed-length numerical vectors. Comprising 350 million parameters, it generates 1024-dimensional embedding, providing a compact yet highly informative representation of nucleotide sequences. The embedding model is built off of the Pathogen AI Foundation Model. An encoder only model that achieves impressive performance on a low compute budget.

Embedding Models

Embedding models are machine learning techniques that map complex data—such as text, images, or DNA sequences—into a numerical format that preserves meaningful relationships between inputs. GeneEmbed learns to represent nucleotide sequences in a way that captures underlying biological patterns, sequence similarities, and functional relationships, making genomic data more interpretable and useful for computational applications.

Applications

The embeddings generated by GeneEmbed 1.0 can be leveraged for a variety of downstream tasks, including:

Genomic Similarity Analysis – Identifying related sequences and clustering similar genetic structures.
Variant Effect Prediction – Assisting in the interpretation of mutations and their potential impact on biological functions.
Pathogen Detection – Enhancing rapid classification and identification of microbial sequences for epidemiological and biosecurity applications.
Genetic Feature Extraction – Providing feature-rich inputs for machine learning models in genomics, improving the accuracy and efficiency of bioinformatics pipelines.