Abstract: The data of building structure is the basic information of seismic fortification and to gain the data has important practical significance. In this paper we introduced a system of unmanned aerial vehicle remote sensing (UAVSS) and technique of investigating building information by using unmanned aerial vehicle (UAV). Firstly, taking some flight missions as application examples, we conducted some processing steps, which include screening photos, matching posture, stitching images and texture mapping. Then we obtained the orthographic mosaic images and 3-Dimensional scene models in building compact districts. Secondly, the structural types of buildings are interpreted by visual method using 3-Dimensional scene models, and the visual results are compared with realistic consequences of ground surveys. Finally, the visual interpretation accuracy is calculated. The value of accuracy is 91.17% and the Kappa coefficient is 0.80. The result shows that the light-small UAV is flexible and convenient. Using 3-Dimensional scene models which are acquired by UAV, we can directly and accurately interpreted the information of building structure. The approach can make up for the inadequacy of traditional field investigation and provide significant reference frame for a wide range of building aseismic capability assessment.