Protein Language Models

Hugging Face now supports protein language models, enabling users to analyze protein sequences using deep learning. This includes models like ProtBERT for understanding protein sequences analogous to NLP models, opening new avenues for research in bioinformatics and computational biology. The platform also supports training and fine-tuning of transformer variants for codon-level language modeling, with CodonRoBERTa-large-v2 showing strong performance in perplexity and correlation metrics. This post details the development of an end-to-end pipeline for protein engineering, including structure prediction (ESMFold), sequence design (ProteinMPNN), and mRNA optimization (CodonRoBERTa-large-v2). The team scaled their codon optimization models to 25 species, trained 4 production models in 55 GPU-hours, and built a species-conditioned system.