Plant sexual deception, the floral mimicry of female insects to attract mate-searching males for pollination, is a long-studied reproductive strategy with a poorly understood genomic basis. Here, we assembled the genomes of a sexually deceptive, a semi-deceptive, and a derived nondeceptive floral form of the South African daisy Gorteria diffusa to chromosome-scale and near-chromosome scale, respectively. We located several previously identified genes involved in the development of deceptive floral traits, including tandem duplications of GdbHLH and GdMYBSG6 transcription factors regulating the complex coloration of sexually deceptive floral structures. Using additional genotyping-by-sequencing data of six G. diffusa floral forms, we further identified several large inverted genomic segments with a high fixation index (FST), which seem to play a role in maintaining the distinct identity of some floral forms in zones of secondary contact. Finally, genome synteny analyses revealed that the genome of the derived nondeceptive floral form is contracted and shows signs of recent genome-wide deletion of long terminal repeat retrotransposons. Our results provide insight into the genomic elements underlying plant sexual deception as well as some of the structural genomic differences between sexually deceptive and nondeceptive floral forms.