Urban microclimate simulations require a rationale for determining the appropriate level of geometric detail to ensure reliable results. This study investigates the influence of building modeling level of detail (BMLoD) on the predictive accuracy of computational fluid dynamics (CFD) simulations. We established six BMLoD stages (1.0–3.5) for a high-density urban district, incorporating LiDAR-based geometric data and precise boundary conditions. Simulation results were validated against field measurements of air temperature at three distinct sites. The findings demonstrate that while increasing geometric detail generally improves accuracy, BMLoD 1.5—incorporating accurate building heights—achieved a significant leap in performance, reducing the root mean square error (RMSE) by approximately 50% compared to the baseline model (BMLoD 1.0). By contrast, models with higher complexity (BMLoD 2.0–3.5) showed only marginal improvements, with temperature differences converging within 0.2 °C. Consequently, this study suggests that prioritizing accurate vertical information is the most practical approach for urban thermal environment analysis, as BMLoD 1.5 provides sufficient fidelity for reliable microclimate modeling. Therefore, for efficient urban microclimate assessment, accurately reflecting building heights is more critical than elaborating on detailed geometric features.