The Brazilian dry forest (Caatinga) is located in one of the world’s largest tropical semiarid regions, and it occurs on two large geological environments named the crystalline and sedimentary basins. In order to determine the structure and the main drivers of the composition of communities of arbuscular mycorrhizal fungi (AMF) in the Caatinga, we collected soil samples from the rhizosphere of Jatropha mollissima, J. mutabilis, and Mimosa tenuiflora, species that occur in crystalline and sedimentary areas. Ninety-six AMF taxa were identified from soils collected directly in the field and trap cultures. Acaulospora, Glomus, and Rhizoglomus represented almost 49% of the taxon richness. The composition of the AMF communities differed between the crystalline and sedimentary areas and between the rhizospheres of the three plant species. Coarse sand, total sand, natural clay, calcium, soil particles density, flocculation, pH, and base saturation were the principal edaphic variables related to the distribution of these organisms. We registered nine and 17 AMF species classified as indicators, for the geological environments and plant species, respectively. Glomerospores of Glomerales predominated in crystalline basins, whereas glomerospores of Gigasporales prevailed in sedimentary areas; among the plant species, lower number of glomerospores of Archaeosporales and Glomerales was recorded in the rhizosphere of J. mollissima. The results show that the AMF community composition is shaped by geological environments and plant hosts. In addition, soil characteristics, mainly physical attributes, significantly influence the structure of Glomeromycota communities occurring in areas of the Brazilian semiarid.