Arbuscular mycorrhizal fungi (AMF) are the most prominent mycobionts of plants in the tropics, yet little is known about their diversity, species compositions and factors driving AMF distribution patterns. To investigate whether elevation and associated vegetation type affect species composition, we sampled 646 mycorrhizal samples in locations between 1000 and 4000 m above sea level (masl) in the South of Ecuador. We estimated diversity, distribution and species compositions of AMF by cloning and Sanger sequencing the 18S rDNA (the section between AML1 and AML2) and subsequent derivation of fungal OTUs based on 99% sequence similarity. In addition, we analyzed the phylogenetic structure of the sites by computing the mean pairwise distance (MPD) and the mean nearest taxon difference (MNTD) for each elevation level. It revealed that AMF species compositions at 1000 and 2000 masl differ from 3000 and 4000 masl. Lower elevations (1000 and 2000 masl) were dominated by members of Glomeraceae, whereas Acaulosporaceae were more abundant in higher elevations (3000 and 4000 masl). Ordination of OTUs with respect to study sites revealed a correlation to elevation with a continuous turnover of species from lower to higher elevations. Most of the abundant OTUs are not endemic to South Ecuador. We also found a high proportion of rare OTUs at all elevations: 79–85% of OTUs occurred in less than 5% of the samples. Phylogenetic community analysis indicated clustering and evenness for most elevation levels indicating that both, stochastic processes and habitat filtering are driving factors of AMF community compositions.