Performance is central to processes management and event data provides the most objective source for analyzing and improving performance. Current process mining techniques give only limited insights into performance by aggregating all event data for each process step. In this paper, we investigate process performance of all process behaviors without prior aggregation. We propose the performance spectrum as a simple model that maps all observed flows between two process steps together regarding their performance over time. Visualizing the performance spectrum of event logs reveals a large variety of very distinct patterns of process performance and performance variability that have not been described before. We provide a taxonomy for these patterns and a comprehensive overview of elementary and composite performance patterns observed on several real-life event logs from business processes and logistics. We report on a case study where performance patterns were central to identify systemic, but not globally visible process problems.