Abstract Agricultural policy in Europe is the subject of extensive debate due to the unmet agri-environmental goals, despite high levels of support. For example, the public is critical of agriculture’s impact on the environment and animal welfare. At the same time, farmers protest due to low incomes and high bureaucracy. This apparent polarisation may hinder progress in agricultural policy reform, and a more nuanced understanding of the perceptions of both non-farmers and farmers is needed. This study systematically compares non-farmers’ and farmers’ preferences and levels of consensus across a comprehensive set of agricultural policy goals using surveys in Switzerland (N = 1668). We also assess the polarisation among farmers, non-farmers, and the entire population. We find consensus and polarisation in the perception of agricultural policy goals among Swiss farmers and the general public. Both prioritise domestic food production and farmers’ income as main policy goals. However, non-farmers put more emphasis on low food prices and agri-environmental goals than farmers. We find strong divides within the farming population, particularly regarding the need to mitigate climate change. Our analysis provides key insights for policymakers. Firstly, the broad consensus on different agricultural policy goals underlines the legitimacy of focusing on multiple objectives that support the multifunctional role of agriculture. Secondly, farmers are not homogeneous but have differing views, particularly regarding environmental objectives, especially those related to reducing greenhouse gas emissions. Policies need to take this heterogeneity into account, for example, by more efficient allocation of agri-environmental payments. Thirdly, while farmers and non-farmers generally agree on the importance of securing farmers’ incomes and food production, there are divergent views when it comes to reducing food prices and environmental impacts. Future agricultural policy reforms must find a balance between environmental goals and safeguarding farmers’ incomes, while also ensuring affordable food prices.