Fitness landscapes, which map genotypes or phenotypes to fitness, have developed from a theoretical metaphor into a popular subject of experimental study. The quantification of the shape of fitness landscapes carries the promise to inform us about the nature of adaptation and the predictability of evolution; yet, the true dimensionality of fitness landscapes is so immense that even large data sets can only cover a small area of the total sequence or phenotype space. Thus, the question emerges whether we can use theoretical or statistical approaches to extrapolate from the observed landscape to the surrounding area or even the whole organism's fitness landscape, and what a given experimental landscape can teach us about evolution in natural populations. I will discuss these questions under consideration of various experimental data sets, and highlight the challenges when trying to bridge our theoretical and empirical knowledge of fitness landscapes.