"Sound recognition relies not only on spectral but also on temporal cues as evidenced by the profound perceptual impact of time-reversion on common sounds. To address the coding principles underlying such temporal asymmetries in perception, we monitored the activity of more than 4000 auditory cortex neurons using two-photon calcium imaging in awake mice while playing sounds ramping up or down in amplitude. We observed clear asymmetries in the cortical population responses, including higher cortical saliency of ramping up sounds which matches perceptual saliency measures both in mice and humans. Analysis and modeling of cortical activity patterns revealed spatially segregated ensembles of neurons working as nonlinear temporal feature detectors that filter temporal modulations in an asymmetric manner. In contrast to standard receptive fields models of auditory cortex function, the dynamics of these ensembles build divergent sound representations for opposite amplitude modulations explaining the emergence of qualitatively distinct percepts."