This study aims to identify the impacts of urban and building forms on the energy consumption of buildings. It applies microscopic elements such as urban forms, urban tissues, building forms, and building character as factors affecting the energy consumption of buildings. To this end, the 2018 energy consumption of the Old Downtowns of Daejeon City and Jaecheon City were analyzed. Methods: This study performs a log transformation of dependent variables in order to apply ordinary least squares (OLS), a method utilized in many studies focusing on the energy consumption of buildings. Results: The analysis results show that most but not all elements related to urban forms and tissues demonstrate a meaningful relationship to the energy consumption of buildings. As for urban form-related elements, buildings with high density and that are divided into numerous blocks show high energy consumption efficiency. Among the building-related elements, time-worn masonry buildings generally demonstrate a low energy consumption efficiency. It is also found that a difference in urban forms has an effect on the energy consumption of buildings, which confirms the fact that improvements in urban form should be accompanied with an endeavor to boost the energy efficiency of building units. These results are expected to contribute to the establishment of a foundation for more sustainable and resilient cities through the reduction of the energy consumption of buildings.