The immune response of an organism when it is infected by a pathogen is based on the recognition of small portions of its proteins. This raises two questions: what protein portions are relevant to this process? And what immune cells are able to recognize them? In this talk, I will discuss models to answer those two questions that are based on the machine learning method known as Restricted Boltzmann Machine and that are learned from large protein sequence datasets. These models provide flexible and interpretable frameworks to characterize and predict immune recognition of both cancer and infections.