Soybean growth is determined by the interaction of genetic, environmental, and management factors. In the context of future climate and climate extremes, understanding genotype by environment interaction (GxE) will be crucial for selecting resilient breeding lines and optimizing management practices to minimize stress. This requires an in depth elucidation of stressful weather conditions and differing temporal responses of genotypes to those conditions. In field studies, however, the environment is often treated as a static factor, and the specific effects of weather variability on crop growth remain poorly understood. Here, we present a longitudinal dataset comprising 17,247 high-resolution RGB images of soybean breeding lines collected throughout eight years in Eschikon, Switzerland. Top-of-canopy images were acquired throughout the entire growing seasons and complemented by hourly weather data, enabling a comprehensive analysis of soybean growth dynamics under varying field conditions. High spatio-temporal image resolution allows detailed analysis of growth dynamics and GxE, supporting identification of stress-tolerant genotypes to improve yield prediction and yield stability.