In countries located in active seismic zones, it is very important to carry out a risk analysis of the existing building stock and to priorities buildings according to their risk status in order to implement effective and improvable measures. In this context, the earthquake risks of hundreds of thousands of buildings in cities, especially residential buildings, need to be assessed, especially in countries that experience major earthquakes, such as Türkiye. Due to the uneconomical and time-consuming nature of detailed analyses for such a large building stock, Rapid Seismic Assessment Methods (RSAM) are very useful for prioritising at-risk structures. However, rapid assessment methods can also require the use of many technical experts, which can lead to differences in interpretation based on their knowledge, expertise and experience in the relevant field. It is very important that the assessment is as standardized as possible to be able to predict the presence of significant structural irregularities for a large building stock and to make decisions on seismic risk quickly and measurably. To achieve this, this paper proposes a two-step deep learning-based framework to automatically extract structural features such as soft stories, short columns, and standard floor windows from building facade images and to estimate the total building height. In the first step, the You Only Look Once version 5 (YOLOv5) object detection model is used to identify key architectural elements associated with seismic vulnerability. In the second step, the detected architectural elements along with the facade image are analyzed to estimate building height. The framework is trained and evaluated using a dataset of 4500 facade images collected from Google Street View (GSV). The results demonstrate the method's potential for large-scale, standardized, and rapid seismic risk assessment in urban environments.