Estimating assemblage similarity is the starting point of multivariate analyses. Beta diversity is also often assessed based on this estimation. However, the accuracy of these estimates is unknown because only a proportion of the species is normally sampled. Statistical methods were recently proposed to predict the “true” similarity between assemblages, but their effectiveness and limitations are poorly understood. In this study, I simulated three groups of assemblages: 1) highly diverse ones with similar species-occurrence distributions (SODs); 2) a succession series with increasing richness and changing SODs; and 3) local assemblages containing random subsets of similar size from the regional species pool and with different SODs. I tested Chao’s versions of the Jaccard and Sørensen indices based on incident frequencies against the standard estimates and the true values. I found that 1) the two indices were moderately underestimated among the Group-1 assemblages and Chao’s methods reduced the bias; 2) both indices could be over- or under-estimated for assemblages in Group 2 or 3, frequently severely so, and Chao’s methods failed to reduce the bias or even increased it; and 3) accuracy improved with increasing sampling effort. These results highlight the challenges in comparing disparate assemblages and the importance of adequately sampling.