Climate change will strongly influence agricultural practices in the future. In order to promote resource-efficient agriculture, it is important to analyse the impact of climate variation on crop yields. In this study, we report yields of spring wheat, winter barley, maize, potato and sugar beet from the long-term crop rotation and fertilization experiment Demo in Switzerland and analyze their response to different climate variables (e.g., annual and seasonal temperature, precipitation, evapotranspiration, number of heat days and days of heavy rainfall). In addition, we investigate the impact of readily plant-available soil potassium (K) on the relationship of crop yields and precipitation. Annual and summer temperatures increased by 1 °C and 1.5 °C, respectively, over the observation period, and both the number of heat days and days of heavy rainfall increased in summer. Rising summer temperatures have a negative impact on all crop yields, which was most prominent for spring wheat, potato and maize. Annual, spring and summer precipitation show varying effects on different crops. For maize, soil K has a mediating effect on yield reductions under low spring precipitation. Yields are significantly reduced by 1 t ha−1 per 100 mm reduction of precipitation below a soil K threshold of 7 mg K kg−1 soil. Based on these results and the future climate scenarios for Switzerland, crop rotations with less heat-sensitive species and early-maturing varieties should be considered. In order to keep future irrigation demands and costs as low as possible, the soil K fertility classes in the Swiss K fertilization guidelines might need to be revisited. Our study is one of a few long-term observations that show the impact of climate variation on crop yields and highlights the potential of K management as a climate change adaptation measure.