Ming and Qing Dynasty Official-Style Architecture Roof Types Classification Based on the 3D Point Cloud

Abstract

The Ming and Qing Dynasty type of official-style architecture roof can provide plenty of prior knowledge relating to the structure and size of these works of architecture, and plays an important role in the fields of 3D modeling, semantic recognition and culture inheriting. In this paper, we take the 3D point cloud as the data source, and an automatic classification method for the roof type of Ming and Qing Dynasty official-style architecture based on the hierarchical semantic network is illustrated. To classify the roofs into the correct categories, the characteristics of different roof types are analyzed and features including SoRs, DfFtR, DoPP and NoREs are first selected; subsequently, the corresponding feature extraction methods are proposed; thirdly, aiming at the structure of the ridges, a matching graph relying on the attributed relational graph of the ridges is given; based on the former work, a hierarchical semantic network is proposed and the thresholds are determined with the help of the construction rules of the Ming and Qing Dynasty official-style architecture. In order to fully verify the efficiency of our proposed method, various types of Ming and Qing Dynasty official-style architecture roof are identified, and the experimental results show that all structures are classified correctly.

1. Introduction

1.1. Backgrounds

The Ming and Qing Dynasty official-style architecture, which mainly includes the royal palaces, official buildings and Buddhist/Taoist temples, is an important carrier for traditional Chinese culture and is considered as the last peak of Chinese architectural history [1,2]. Timberwork is the main construction material for this ancient Chinese architecture, and wooden posts, beams, lintels and joists make up the framework of a house. Suffering from natural disasters and man-made destruction, a mass of the ancient architecture is damaged or has disappeared due to the fact that these construction materials are vulnerable to weathering and fires [3]. To preserve the information of this ancient architecture, digital documentation has become increasingly necessary. The 3D point cloud, which can be captured from terrestrial or airborne LiDAR, and UAV-based SfM processing have become one of the most reliable data sources, and play a more and more important role in the field of digital documentation [4,5,6]. Recent studies on the 3D point cloud have achieved great progress in 3D architectural documentation [7]. However, the ancient buildings in the Ming and Qing Dynasty contain a large number of components with irregular shapes, and this brings great difficulties in reconstructing the 3D parametric model, which has further application [8,9].

Roof types have been widely used in 3D building and parametric modeling and have led to many achievements. Kada and McKinley [10] firstly segmented the point cloud and determined that the roof type relies on the number of segments; then, the roof plane parameters were estimated. Mass and Vosselman [11] selected the heights of the point cloud as weight functions in moment equations to compute the information on the roof type and shape parameters. On this basis, the gable roof was reconstructed in [11]. Much of the literature indicates that roof types play an important role in 3D reconstruction [12,13,14,15].

Similarly, the roof types (form) of the Ming and Qing Dynasty official-style architecture can also improve the efficiency of parametric modeling. Moreover, in the Ming and Qing Dynasty, the hierarchy was strictly divided and the construction of the Ming and the Qing Dynasty official-style architecture followed certain rules, which can be searched for in YingzaoFashi (Building Standards) [16] of the Song Dynasty or Gongchengzuofa zeli (Structural Regulations) [17] published by Qing. The roof types (form) of the Ming and Qing Dynasty official-style architecture can provide more information, including the structure, topological relation and the size of the components. Based on the information from the roof types, Yuan Shen [18] calculated the parameters of the traditional Chinese curvilinear roof based on the roof types (form). Other 3D reconstruction methods for ancient Chinese architecture using roof types can be seen in [19,20,21].

Hence, understanding how to distinguish the roof types of the Ming and Qing Dynasty official-style architecture from the point cloud is an important task, and has a great significance in the field of 3D modeling.

1.2. Related Works

1.2.1. Roof Type Classification

Focusing on the roof type, several classification methods have been proposed by researchers. The previous classical methodologies for roof type classification mainly rely on the texture, geometry, height and other features to form building roof hypotheses. These features can come from images, the 3D point cloud or DSM. For example, Kushwaha et al. [22] applied the inclination to categorize roof points into flat, inclined or dome-type roofs; Mohajeri et al. [23] made use of the number of roof surfaces and the distribution of the binned slope angles to complete the roof classification based on the DSM; Zang et al. [24] designed a new feature to describe the roof and obtain better performance compared with the histogram of oriented gradient (HOG), scale-invariant feature transform (SIFT) and local binary pattern (LBP) features. The methods in [25,26,27] provide other examples.

Despite the data sources being different, features are always the key to roof type recognition in classical roof classification methods. Aiming at the different roof types, the features must be designed carefully. This principle also applies to the roof classification of the Ming and Qing Dynasty official-style architecture. However, the differences between different roof types may be very small. How to select and extract the proper features from a 3D point cloud to distinguish the roof types is still a challenging task.

In recent years, with the advances in the field of artificial neural networks, some researchers have turned their attention towards deep learning for roof types classification. Axelsson et al. [28] applied a pre-trained deep convolutional neutral network (CNN) to classify the most common slope and flat roof types based on the aerial images. Similarly, based on a pre-trained CNN framework, Partovi et al. [29] accomplished building roof types classification. In addition to using a pre-trained CNN framework directly, Partovi et al. extracted the deep features from deep layers of different pre-trained CNN models and made use of an SVM classifier to recognize the building roof type. Jeremy et al. [30] proposed an automatically labelling building roof shape method from publicly available GIS data. In this method, a diverse annotated roof image dataset is created, the multiple CNN architectures are trained and tested, and the experimental results show that the fusion of the satellite image and LiDAR data can provide greater classification accuracy than using either data type alone. To improve the learning efficiency and prediction accuracy, Bittner et al. [31] propose a Multi-Task conditional generative adversarial network (cGAN) for simultaneous space borne DSM refinement and roof-type classification.

The methods based on deep learning for roof classification shows promising results; however, there are some limitations. On one hand, although some roof type training datasets have been published [32], these datasets mainly consist of images and there is rarely a point cloud dataset focusing on the Ming and Qing Dynasty official-style architecture roof. On the other hand, the direct method of applying deep learning on a point cloud is to convert the data into volume representation. Comparing with the 2D image, the amount of 3D volume data will become very large, very fast (although a graphics processing unit (GPU) has been developed). To speed up processing, it is necessary to compromise and adopt a lower resolution (some methods use 64 × 64 × 64). This strategy will cause the loss of details and bring the cost of quantization errors. In fact, some experimental results show that the classical methods may obtain better performance than the deep learning methods in the field of architectural heritage point cloud classification [33].

1.2.2. Object Recognition Based on 3D Point Cloud

Identifying roof types of the Ming and Qing official-style architectures from a point cloud is an object recognition/classification problem. Classical object recognition methods based on the 3D point cloud mostly compare/learn the geometric, shape, structural attributes or multiple attributes to complete the object recognition and classification [34]. From the perspective of the features, these methods can be divided into four categories: object recognition methods based on local features, object recognition methods based on global features, object recognition methods based on graph matching and object recognition methods based on machine learning.

The process of the object recognition methods based on local features can be divided into three stages: key point detection, feature description and matching. Many mature methods which can complete specific functions in each stage have been proposed. In the key point detection stage, the common key point detection algorithms mainly include intrinsic shape signatures (ISSs) [35], Harris 3D [36], local surface patches (LSPs) [37]; fast point feature histograms (FPFHs) [38], SHOT [39] and rotational projection statistics (ROPS) [40] which are used for feature description; the feature matching can be completed by the threshold method, nearest neighbor (NN) and nearest neighbor distance ratio (NNDR) [41]. Due to the fact that this type of method makes use of the local features of the key points, edges and patches to complete the target object recognition directly, it inevitably needs to search for the feature points from the entire scene and model. This causes the recognition process to be time consuming. The object recognition methods based on global features firstly extract the target object from the entire scene and regard the 3D target object as a whole; then, the global features from the whole 3D target are applied for the object recognition. The common classical global features mainly contain GFPFH [42], VFH [43], CVFH [44] and ESF [45]. Although this kind of method is faster compared with the object recognition methods based on global features, it is sensitive to noise and occlusion. This is mainly because the noise and occlusion damage the extraction results of the target object from the complex scene. Moreover, due to the fact that the object recognition methods based on global features regard the target object as a whole, it is difficult to describe the details of 3D models and recognize objects with similar shapes. For the Ming and Qing official-style architecture, the differences are just some details between these roofs. This will damage the recognition rate.

In addition to extracting features directly from the point cloud for object recognition, some researchers decompose the point cloud data into basic shapes and convert these basic shapes into a topological graph to represent and recognize the target objects. Schnabel et al. [46] applies the random sample consensus (RANSAC) algorithm to decompose the point cloud data into basic shapes. Each basic shape is represented by an abstract point, and the adjacent relationship between them is represented by a topological graph. The object recognition is completed by graph matching. The similar methods in [47,48,49] are a few examples. The main difference of these methods is what kind of features are selected and extracted to construct the topological graph. For example, Cheng et al. [47] select various curvature information to construct the topological graph; Hao et al. [48] defines the connection types between different types of planes and analyzes the structure of common objects; Berner et al. [49] combines the outlines and shape to construct the topological graph. For the Chinese ancient architecture roof, the structure of the roof is the important feature. However, only relying the structure of the different Ming and Qing Dynasty official-style architectures is impossible. This is mainly because the different roof types may have the same structure relationship. For example, the structure of the overhanging gable roof and the flush gable roof are the same.

The object recognition method based on machine learning extracts and learns the features of samples, and uses the classifier to complete the classification and recognition of objects. Generally speaking, machine learning classifiers can be divided into two categories: traditional machine learning and deep learning. In the traditional machine learning method, the commonly used classifiers are: support vector machine [50], random forest [51], adaboost [52], jointboost [53], naive Bayes classifier [54] and maximum expectation algorithm [55]. Nowadays, the research of point cloud classification mainly focuses on the recognition of the different ground objects from complex scenes. Much of the literature shows that the most important impactors affecting recognition are the selected features. Hence, the extraction of the features presented in the Ming and Qing official buildings with different styles is very important in the process of the Ming and Qing official building style classification based on the point cloud.

Thanks to the performance improvement of computer hardware, some architectural heritage classification methods based on deep learning are proposed. One of the earliest and most famous deep learning architectures working directly on the point cloud is point net [56]. It is an end-to-end deep neural network, which can learn the features of classification, part segmentation and semantic segmentation by itself. Because the point net does not capture the local geometry, a development, namely PointNet++ [57] is proposed. Besides the PointNet, some researchers [58,59] applied convolution to gain understanding of point-based learning. In 2019, Wang et al. propose a method named DGCNN [60]. Instead of employing individual points, this method exploits local geometric structures by constructing a local neighborhood graph and applying convolution-like operations on the edges connecting neighboring pairs of points. Although the architectural heritage classification methods based on deep learning have obtained a good performance, no matter what the ML or DL, the training datasets are very important. Nowadays, some datasets such as ModelNet 40 [61], KITTI [62], Sydney Urban Objects dataset [63], Semantic3D [64], S3DIS [65] and ArCH [66] have been published, most of the current datasets collect data from urban environments, and there are still no published datasets focusing on immovable cultural assets with an adequate level of detail. This brings difficulties for Ming and Qing Dynasty official-style architecture roof types recognition.

1.3. Motivation and Contributions

In this work, we aim to distinguish the roof type from the 3D point cloud so that the information provided by the roof type can be used in the 3D reconstruction of Ming and Qing Dynasty official-style architecture. Based on a hierarchical semantic network, a classification method for the Ming and Qing Dynasty official-style architecture roof types is presented in this paper. With the help of this method, the roofs are classified into the correct categories. The main contributions of this paper are listed as follows:

• The features which distinguish the roof types are selected based on the “grammar book” and the corresponding feature extraction methods are proposed.
• Aiming at the structure of the ridges, the attributed relational graphs of the ridges from different types of the Ming and Qing Dynasty official-style architectures are constructed and recognized.
• A hierarchical semantic network for the Ming and Qing official-style architecture roof classification is proposed. In this framework, adaptive thresholds are estimated based on the construction rule of Qing Dynasty architecture, and the reliable thresholds are given in this paper.

Research in this paper is organized as follows. Section 2 analyzes and selects the identified features for the classification of the Ming and Qing Dynasty official-style architecture roof. Section 3 gives the corresponding features extraction methods. To recognize the structure of the ridges, graph matching relying on the attributed relational graph of the ridges is proposed in Section 4. In Section 5, a hierarchical semantic network for the classification approach of the Ming and Qing Dynasty official architecture roof is proposed. Section 6 shows the experimental results and analysis. Finally, a conclusion is conducted in Section 7.

2. Feature Selection

2.1. A Brief Introduction of the Roof Types

The roof of ancient Chinese architecture plays a particularly important role in the building facade, which makes the building produce a unique and strong visual effect and artistic appeal. For the Ming and Qing Dynasty official-style architecture, the roof types include basic, special and derived roof types. The basic roof types can be classified into five categories: hip roof, gable-and-hip roof, pyramidal roof, overhanging gable roof, and flush gable roof as is shown in

Table 1. The roof types of the Ming and Qing official-style architecture.

Roof Type	Illustrations	Example	Description
hip roof			Hip roofs with all sides sloping, are the classiest traditional roof style. There are a total of five ridges including a main ridges and four vertical ridges.
gable and hip roof			Gable and hip roofs, with two curving sides, are second in importance to hip roofs. They are nine ridges including a main ridges, four vertical ridges and four diagonal ridges.
overhanging gable roof			Overhanging gable roofs have two straight, overhanging slopes. They are five ridges including a main ridges and four vertical ridges.
flush gable roof			Flush gable roofs have a main ridge and raise sloping ridges on the gable walls. It is a very simple style with two slopes facing front and back.
pyramidal roof			Pyramidal roof has four slopes. The number of the slopes is equal to the number of vertical ridges which intersected at one point.
round ridge roof			Round ridge roof, with no main ridge, has two straight slopes. It is a variant of the gable and hip roof, overhanging gable roof and flush gable roof.

. The gable-and-hip roof, gable roof and overhanging gable roof have a corresponding round ridge roof; and the hip roof, gable-and-hip roof and pyramidal roof can be further divided into double-eave and single-eave as is shown in Figure 1. Different roof types reflect the different architectural hierarchies and the social status of the holder. For example, the hip roof can only be used for the royal buildings or Confucius halls. From high to low rank, the order of different roof types’ grades is: double-eave hip roof, double-eave gable-and-hip roof, double-eave pyramidal roof, hip roof, gable-and-hip roof, pyramidal roof, overhanging gable roof, round ridge roof, and flush gable roof.

Besides these basic roof types, there are some special and derived roof types for the Ming and Qing Dynasty official-style architecture. The common ones are: fan-shaped gable and hip roof with a round ridge, hip and flat roof, intersecting gable and hip roofs (Figure 2). These roofs are derived from different combinations of basic roofs. For example, the intersecting gable and hip roof is formed by the intersection of two gable-and-hip roofs. The representative intersecting gable and hip roofs building is the corner building of the Palace Museum in Beijing. Through various combinations of roofs, the shape and contour of the building become more abundant.

The roof types of the Ming and Qing Dynasty official-style architecture are rich and varied. It is difficult to list all the roof types. Compared with the special roof types and the derived roof types, the basic roof types are more common. Hence, in this paper, we mainly focus on the classification of the basic roof types.

2.2. Feature Analysis for the Ming and Qing Dynasty Official-Style Architecture Roof Classification

According to the description in Table 1, the roof of the Ming and Qing Dynasty official-style architecture is mainly composed of roofing and ridges. Among the different types of roofs, the number, topological structure of the ridges and the shape, number, and topological relationship of irregular surfaces are different. All these differences can be regarded as the identified features. Although the features from the ridges and roof surfaces are coupled with each other, the features from the ridges can provide more details and are more robust in the recognition of the Ming and Qing Dynasty official-style architecture compared with the features from roof surface. For example, both the hip roof and the pyramidal roof have four slopes. The difference between these two types of roofs is whether there is a main ridge. Similarly, the gable-and-hip roof with a round ridge does not have a main ridge, while the gable-and-hip roof has a main ridge.

However, only relying on the ridges to classify the roof of the Ming and Qing official-style architecture into correct categories is impossible. Several of the limitations are listed as follows:

• The structural relationship of the ridge vector graph from the flush gable roof and overhanging gable roof are almost the same.
• The ridges cannot provide the single-eave or multiple-eave information which is used to distinguish the single-eave or multiple-eave hip roof, pyramidal roof and gable and hip roof.

To overcome these limitations, other features from the Ming and Ming and Qing Dynasty official-style architectures should be considered. Based on these analyses, the selected features for the roof classification are listed as follows.

1.

The structure of the ridges—SoRs

The ridges from different roof types of the Ming and Qing Dynasty official-style architecture have different topological relationships. The structure of the ridges of different roof types can be seen in Figure 3. Based on SoRs, most roofs can be classified into the correct categories.

2.

The distance from the outline of the facades to outline of the roof—DfFtR

Figure 4a,b shows a flush gable roof building and an overhanging gable roof building, respectively. For the flush gable roof building, the outline of the facades is basically consistent with the outline of roof on the $\mathrm{XOY}$ plane as is shown in Figure 4c, while the outline of the facades is far away from the roof contour on both sides of the overhanging gable roof building as is shown in Figure 4d. From the perspective of the construction system for the Ming and Qing Dynasty official-style architecture, the distance from the outline of the facades to outline of the roof (DfFtR) of the overhanging gable roof building is also called cantilever length (悬挑). Obviously, DfFtR can be regarded as a salient feature for distinguishing the flush gable roof and overhanging gable roof.

3.

The density of the projective points—DoPP

The shape of a main ridge is nearly a cuboid as is shown in Figure 5a. Due to the fact that there are two facades on the front and rear sides of the main ridge, this results in the density of the projective points (DoPP) on the $\mathrm{XOY}$ plane being higher than that located in other areas of the roof surface as is shown in Figure 5b. Based on the DoPP, the main ridge can be recognized.

4.

The number of the roof eaves—NoREs

The number of the roof eaves (NoREs) is mainly used for distinguishing the single eave architecture and multiple eave architecture.

3. Feature Extraction

The input data are an entire architecture point cloud. Considering the calculation process of the selected features, the roof and the ridges should be extracted before generating the features. In this section, a roof extraction method based on the change of projective areas and a ridge extraction method using the section lines are illustrated at first. On the basis of the extracted roof and ridges, the features are generated in Section 3.3.

3.1. The Roof Extraction Method Based on the Change of Projective Areas

The shape of the Ming and Qing Dynasty official-style architecture roof is sloping or curved and the roof eaves extend out of the beam frame. This results in the projective area of the points becoming small from the roof eaves to the top of the roof on the $\mathrm{XOY}$ plane. Based on the change of projective areas, a roof extraction method is proposed. The details of this method are described as follows:

• Suppose that the original point cloud represented the Ming and Qing Dynasty official-style architecture (Figure 6a) is defined as $P=\left\{p_i\left(\begin{array}{ccc}{x}_i& y_i& z_i\end{array}\right)|i=1,2,\cdots ,N\right\}$, $N$ is the number of points in $P$. The point $p_h\left(\begin{array}{ccc}{x}_h& y_h& z_h\end{array}\right)$ and the point $p_l\left(\begin{array}{ccc}{x}_l& y_l& z_l\end{array}\right)$ belonging to the point cloud $P$ are the highest and lowest points along $z$ direction separately.
• Along the z direction, divide the point cloud $P$ into the several of subsets with interval $z_d$ as is shown in Figure 6b. The interval $z_d$ is set as $0.1 \mathrm{m}$ based on experience. The sampled points are defined as $S=\left\{s_j|j=1,2,\cdots ,M\right\}$, $M=\mathrm{int}\left(\frac{\left(z_h-z_l\right)}{z_d}+1\right)$. A point $p_i$ from the point cloud is categorized to the subset $s_j$ which should meet Equation (1).

$$\left\{\begin{array}{c}{p}_i\in s_j\\ j=\mathrm{int} \left(\frac{\left(z_i-z_l\right)}{z_d}+1\right)\end{array}\right.$$

(1)

• From 1 to M, wipe off the point set $s_j$ from the original point cloud in turn. After each point subset is removed, project the remaining points onto the $\mathrm{XOY}$ planar coordinates with a scale $d_s$. If the number of points falls in a grid beyond 0, this grid is marked as 255. Subsequently, a morphological close operator with a $7\times 7$ square structuring element is used to generate an initial binary image. Figure 6c,d shows the generated binary image created by the points which are higher the red points and the green points labeled as in Figure 6b separately. The number of pixels consisting of the binary image can be regarded as the projective areas on the $\mathrm{XOY}$ plane. The histogram composed of the areas is shown in Figure 6e.

Starting with the roof eave, the area becomes smaller with the increasing height. If the elevation value of a point is greater than that of the split roof eave, this point can be regarded as the roof point, otherwise the point is the roof point. The extracted roof can be seen in Figure 6f.

3.2. The Ridge Extraction Using Section Lines

Figure 7 shows the process of the ridge extraction. The details of this process are described as follows:

• Project the roof points onto the $\mathrm{XOY}$ plane, and divide the two-dimensional plane into grids according to the specific size $d_s$. The points $p_t\left(\begin{array}{ccc}{x}_t& y_{max}& z_t\end{array}\right)$, $p_b\left(\begin{array}{ccc}{x}_b& y_{min}& z_b\end{array}\right)$, $p_{lm}\left(\begin{array}{ccc}{x}_{min}& y_{lm}& z_{lm}\end{array}\right)$ and $p_{rm}\left(\begin{array}{ccc}{x}_{max}& y_{rm}& z_{rm}\end{array}\right)$ are the top point, bottom point, left-most point and the right-most point separately.
• Along the $Y$ axis, search the ridge points. Firstly, suppose that the points within the $i_{th}$ row which are parallel to the $X$ axis are defined as the point set $S_{xi}$. $S_{xi}=\left\{p_{xij}\left(\begin{array}{ccc}{x}_{xij}& y_{xij}& z_{xij}\end{array}\right)| j=1,2,\cdots ,M_{xi}\right\}$, $M_{xi}$ is the number of the points. The point $p_{xij}\left(\begin{array}{ccc}{x}_{xij}& y_{xij}& z_{xij}\end{array}\right)$ should meet Formula (2).

$$y_{min}+\left(i-1\right)d<y_{xij}\le y_{min}+id$$

(2)

Select the highest point $p_{ximaxz}\left(\begin{array}{ccc}{x}_{ximaxz}& y_{ximaxz}& z_{ximaxz}\end{array}\right)$ from the point set $S_{xi}$. If a point $p_{xij}$ meets Formula (3), this point is stored into the point set $p_f$.

$$z_{ximaxz}-z_{xij}\le z_t$$

(3)

In this formula, $z_{xij}$ is the elevation of the point, $z_t$ is the specified threshold. The extracted ridge points belonging to the $i_{th}$ row are shown in Figure 7b. Figure 7d shows the extracted ridge points within different rows.

• Similarly, continue step 2 along the $X$ axis and save the ridge points to the point set $p_f$. The extracted ridge points are shown in Figure 7f.

3.3. Feature Generation

Based on the roof extraction method and the ridge extraction method, the extraction process details of the features including NoREs, DfFtR, DoPP and SoRs are listed as follows.

• NoREs generation

As is shown in Figure 6e, the histogram can be divided into four stages. In the first stage, only the facade points are removed and the projective area remains unchanged; subsequently, the area becomes small with the decrease of points belong to the first roof eave; in the third stage, the facade points between the first roof eave and the second roof eave are wiped off gradually and the projective area keeps stable; lastly, with the increase of height, the projective area becomes 0. Obviously, the intervals where the projective area remains unchanged correspond to the facade points, and the intervals where the projective area decreases correspond to the roof points. The extracted facade points and roof points can be seen in Figure 6f. The number of the intervals where the projective area remains unchanged is NoREs.

2.

DfFtR generation

After roof extraction, project the façade points and roof points onto the $\mathrm{XOY}$ plane, and generate the outline of the facades and roof, separately. DfFtR is equal to the distance between two intersections which are created by a straight line defined by the center point and main direction of the architecture. These two intersections are on the side of the building facade.

3.

SoRs generation

Project the ridge points onto the $\mathrm{XOY}$ plane. If the number of the points located in a grid is beyond 1, the grid is labeled as 255. After these steps, the generated ridge lines still have a width of 2–3 pixels. Subsequently, a Skeletonization algorithm [67] is performed to thin the ridge lines. Finally, a straight-line detector based on the Freeman chain code [68] is used to generate the ridge lines.

4.

DoPP generation

The main ridge is parallel to the main direction of the architecture, and is located in the middle area of the architecture. Based on this, the ridge line which represents the main ridge is selected. Around the selected ridge line, create a buffer region. The number of points located in this buffer region divided by the area of this buffer region is the DoPP.

4. Graph Matching Relying on the Attributed Relational Graph of the Ridges

Different from NoREs, DfFtR and DoPP which have a specified threshold, the SoRs is the structural relationship of the ridges. Hence, the structure of the ridges should be converted into a topological graph and the graph matching algorithm is used to distinguish the roof types. Nowadays, the formal description methods of structure are various, including serial grammar, tree grammar, deep learning and so on. The attributed relational graph not only can describe the structural relationship of the graph, but also can introduce the property which describes some characteristics of the graph. In this section, we firstly construct the attributed relational graph of ridges; then, the subgraph isograms are applied to distinguish the roof types.

4.1. The Generation of the Attributed Relational Graph of the Ridges

Here, we define a quadruple to present the attributed relational graph as in Formula (4).

$$G=\left(V,E,R_V,R_E,G_V,G_E\right)$$

(4)

In Formula (4), $V=\left\{\begin{array}{ccc}{v}_1& v_2& \begin{array}{cc}\cdots & v_N\end{array}\end{array}\right\}$ is the finite data set of the vertices, $N$ is the number of vertices in the data set $V$; $E=\left\{\begin{array}{cccc}{e}_1& e_2& \cdots & e_M\end{array}\right\}$ is the finite set of the edges, such as $e=\left(\begin{array}{cc}{v}_i& v_j\end{array}\right)$, $1\le i, j\le N$,$i\ne j$, which represents the edge between the vertex $v_i$ and the vertex $v_j$, M is the number of edges in E, $R_V$ is the vertex property set, and $R_E$ is the edge property set. $G_V$ is a function that produces vertical properties from $V\to R_V$, and $G_E$ is a function that produces edge properties from $E\to R_E$.

In the process of describing ridge features using an attributed relational graph, the extracted ridge line is regarded as the vertex $V$ and the connection relationship between the ridges can be used as the edge $E$. $R_V$ and $R_E$ contain an attribute separately. The length $\alpha$ is the vertebrae properties in $R_V$ and the angle $\theta$ between the connected ridges is the edge property in $R_E$. Figure 8 shows the vector graph and the corresponding attributed relational graph of the ridges from the different types of Ming and Qing official architecture.

Table 2. The attributed relational graph template of the ridges from pyramidal roofs.

$V=\left\{v_1,v_2,v_3,v_4\right\}$

$E=\left\{e_1\begin{array}{cc}\langle v_1& v_2\rangle ,\end{array} e_2\begin{array}{cc}\langle v_2& v_3\rangle ,\end{array} e_3\begin{array}{cc}\langle v_1& v_3\rangle ,\end{array} e_4\begin{array}{cc}\langle v_1& v_4\rangle ,\end{array} e_5\begin{array}{cc}\langle v_2& v_4\rangle ,\end{array} e_6\begin{array}{cc}\langle v_3& v_4\rangle \end{array}\right\}$

$R_V=\left\{\alpha |{\alpha }_{vi},i=1,2,\cdots ,N\right\}$

$R_E=\left\{\theta |{\theta }_{ei},i=1,2,\cdots ,M\right\}$

$G_V=\left\{{\alpha }_i=l_i⁄{\displaystyle \sum _{j=0}^N}{l}_j\right\}$

$G_E=\left\{{\theta }_{ei}={\cos }^{-1}\left(e_i\right) \right\}$

shows the attributed relational graph template of the ridges from pyramidal roofs.

4.2. The Roof Type Reorganization Based on the Subgraph Isograms

The generated attributed relational graph from extracted the ridges should match the constructed attributed relational graph template so that the Ming and Qing Dynasty official-style architecture can be classified into the correct category. The graph isomorphism can solve the problem. The graph isomorphism is the function from the graph $G=\left(V,E,R_V,R_E,G_V,G_E\right)$ to the graph $G^{\prime }=\left(V^{\prime },E^{\prime },R_V{}^{\prime },R_E{}^{\prime },G_V{}^{\prime },G_E{}^{\prime }\right)$. This function $f: V\to V^{\prime }$ should satisfy the followed numbering conditions:

(1)

$\forall v\in V$, $\exists G_V\left(v\right)=G_V^{\prime }\left(f\left(v\right)\right)$;

(2)

$\forall e=\left(v_1,v_2\right)\in E$,$\exists e^{\prime }=\left(f\left(v_1\right),f\left(v_2\right)\right)\in E^{\prime }$ and $G_E\left(e\right)=G_E^{\prime }\left(e^{\prime }\right)$; $\forall e^{\prime }=\left(v_1{}^{\prime },v_2{}^{\prime }\right)\in E^{\prime }$,$\exists e=\left(f^{-1}\left(v_1^{\prime }\right),f^{-1}\left(v_2{}^{\prime }\right)\right)\in E$ and $G_E\left(e\right)=G_E^{\prime }\left(e^{\prime }\right)$.

When the structure of two graphs is the same, these two graphs must be an isomorphism. However, suffering from the extraction accuracy of the ridges, only a part of the graph is obtained in some cases. Hence, the isomorphism conditions need to be relaxed to allow one graph to be mapped to another part of the graph. This mapping is called subgraph isograms. The function $f:V\to V^{\prime }$ is the isomorphic function from the graph $G$ to the subgraph of $G^{\prime }$.

From the analysis above, the isomorphism of the graph is a special case of the subgraph isomorphism. In this paper, the Ullmann algorithm [69] is used to determine the isomorphism of subgraphs.

5. Our Proposed Method

5.1. The Workflow of Our Proposed Methods

The workflow of our proposed method is shown in Figure 9. The process of this workflow is separated into four stages. The details of each stage are described as follows:

• In the first stage, the roof extraction method is applied to segment the point cloud into the facade points and roof points. The feature NoREs is obtained. If $\mathrm{NoREs}=1$, the Ming and Qing official-style architecture roof is classified as a single-eave roof; otherwise, this roof is categorized as a multiple-eaves roof.
• Secondly, extract the ridge points from the roof points based on the ridge extraction method proposed in Section 3.2 and make use of the ridge points to generate the feature SoRs. Based on the method in Section 4, a single-eave roof can be classified into a hip roof, pyramidal roof, unclassified gable and hip roof or unclassified roof; and the multiple-eaves roof is classified into a hip roof, pyramidal roof or unclassified gable and hip roof.
• In the third step, calculate the DfFtR. If $DfFtR>Th{r}_{DfFtR}$, the unclassified single-eave roof is regarded as an unclassified overhanging gable roof; otherwise, the unclassified single-eave roof is regarded as an unclassified flush gable roof.
• Finally, calculate the DoPP based on the method in Section 3.3. If $DoPP>Th{r}_{Dopp}$, the unclassified flush gable roof, unclassified overhanging gable roof or unclassified gable and hip roof is grouped into the flush gable roof category, overhanging gable roof category or gable and hip roof category; otherwise, the unclassified roof is categorized as an overhanging gable round ridge roof, flush gable round ridge roof or gable and hip round ridge roof.

5.2. Threshold Determination

In our proposed method, $Th{r}_{DfFtR}$ and $Th{r}_{DoPP}$ are the thresholds in the suggested hierarchical semantic network for the Ming and Qing official-style architecture classification. These two thresholds are decided by the cantilever length (CL) and the height of the main ridge (HoMRR) separately.

From the perspective of the construction system, the Ming and Qing Dynasty official-style architecture can be further divided into wooden frame architecture with dougongs and wooden frame architecture without dougongs. The former mainly covers the categories of hip roof, gable-and-hip roof, pyramidal roof, overhanging gable roof and flush gable roof; the latter contains the pyramidal roof, overhanging gable roof, and flush gable roof. These two wooden frame architectures have different modules. The dimensions of the main components of the Ming and Qing official architecture should be a multiple of the modules. According to the Gongcheng Zuofa Zeli of the Qing Dynasty, both HoMRR and CL is equal to 12 * doukou under the doukou module system. Under the D module system, HoMRR should be equal to 2.2D and CL is equal to 3.3D as is shown in

Table 3. The rule table for the thresholds.

Index	The Modules of the Wooden Frame with Dougong	The Modules of the Wooden Frame without Dougong
eave column height	60 doukou	11 D
main ridge height	12 doukou	2.2 D
diameter of the draft	1.5 doukou	D/3
cantilever length	12 doukou	3.3 D

.

The doukou has 11 sizes or grades and the minimum dimensions of doukou is 1 cun (3.5 cm) as is show in Figure 10. However, the eave column diameter D does not have an exact value. To overcome this limitation, the corresponding sizes of the same components from the wooden frame architecture with or without dougongs is applied to obtain an approximate value of D. Based on this, D ranges from$3.6 \mathrm{doukou}$ to $5.5 \mathrm{doukou}$. Considering the different grades of Ming and Qing official-style buildings, HoMRR ranges from $1 \mathrm{cun}\times 8$ to $6 \mathrm{cun}\times 12$ and CL ranges from $1 \mathrm{cun}\times 3.3\times 3.6$ to $6 \mathrm{cun}\times 12$.

$Th{r}_{DfFtR}$ should be equal to CL. The higher the grade of the Ming and Qing official-style building is, the bigger CL is. Hence, $Th{r}_{DfFtR}$ should be higher than the minimum CL—$1 \mathrm{cun}\times 8$. For $Th{r}_{DoPP}$, suppose that the density of point cloud is $DEN$. In theory, $Th{r}_{DoPP}$ should be equal to $DEN\times \left(2\times HoMRR\times d_s+d_s\times d_s\right)/\left(d_s\times d_s\right)$ when $d_s$ is bigger than the width of the main ridge; otherwise, $Th{r}_{DoPP}$ is equal to $DEN\times \left(HoMRR\times d_s+d_s\times d_s\right)/\left(d_s\times d_s\right)$. In this paper, $d_s$ is user-defined. Here, we set $d_s$ as $HoMRR$. We can set this value to be the $Th{r}_{DoPP}$ which can be set as $2DEN$.

6. Performance Evaluation

6.1. Experimental Data Description

Many Ming and Qing official-style buildings have been established as major historical and cultural sites protected at the national level in China and are managed by corresponding institutions. When we capture the point cloud, the authorization from these institutions is necessary. This results in the acquisition of the point cloud becoming very difficult. In our experiments, three datasets were used to evaluate the performance of our proposed method. The first two datasets consisted of point clouds captured by the terrestrial laser scanning system or generated by UAV images and contained two roof types. To test more roof types, we introduced a third dataset composed of the point clouds derived from 3DsMAX models. The 3D information of these models is consistent with that of real buildings and is reliable test data through inspection and comparison by experts. Each dataset is described as follows:

• The first dataset contained the point cloud of the Gate of Supreme Harmony and the Hall of Complete Harmony labeled as rectangle 1 and 2 in Figure 11a. These point cloud was captured by the terrestrial laser scanning (TLS) system. Figure 12 shows the point cloud after registration in the commercial software package Leica Cyclone. The point cloud density of the Gate of Supreme Harmony and the Hall of Complete Harmony was 44,083 points/m² and 43,416 points/m², respectively.

• The second dataset was composed of the dense image matching (DIM) point cloud of BaoGuang Hall located in Qutan Temple, QingHai province, China as is shown in Figure 11b. 261 UAV images are collected by DJI Phantom4 which was composed of a FC6310R camera with a 13.2 × 8.8 mm² sensor size and a $2.41 \mathsf{\mu }\mathrm{m}$ pixel size. The flight path surrounded the building as is shown in Figure 13. The distance from the exposure points to this building varied from $20 \mathrm{m}$ to $85 \mathrm{m}$. Considering the $8.8 \mathrm{mm}$ focal length and the photographic distance, the ground sampling distance (GSD) for all cameras ranged from $0.5 \mathrm{cm}$ to $2 \mathrm{cm}$. Relying on the commercial software package Bentley, this DIM point cloud was generated. The density of the generated DIM point cloud was 59,737 $\mathrm{points}/{\mathrm{m}}^2$.

Figure 13. The flight path.

Figure 13. The flight path.

• The third dataset included 3DsMAX models of Meridian Gate, LiJing Xuan, Gate of Lasting Happiness labeled as rectangle 4, 5 and 6 in Figure 11a and a 3D model example from a website. To satisfy the data requirements, we converted these 3D models into 3D point cloud based on the commercial software CloudCompare. The density of the point cloud was 95 points/m².

There are six types of Ming and Qing Dynasty official-style architecture roofs: double-eave hip roof, double-eave gable and hip roof, flush gable roof, round ridge roof, overhanging gable roof and pyramidal roof. The roof type information of each test building is described in

Table 4. The details of each test building.

Architecture Name	Roof Type	Point Cloud	Illustration
Gate of Supreme Harmony	double-eave gable and hip roof
Hall of Complete Harmony	pyramidal roof
BaoGuang Hall	double-eave gable and hip roof
Meridian Gate	double-eave hip roof
LiJing Xuan	flush gable roof
Gate of Lasting Happiness	flush gable roof with round ridge
A 3D model example	overhanging gable roof

.

6.2. Experimental Results and Discussion

6.2.1. Experimental Results

In our experiment, the thresholds were set according to the description in Section 5.2.

Table 5. The experimental details and results.

	Gate of Supreme Harmony	Hall of Complete Harmony	BaoGuang Hall	Meridian Gate	LiJing Xuan	Gate of Lasting Happiness	A 3D Model Example
Original point cloud of test
The change of the areas on the X along the Z directions
Extracted roof
NoREs		null			null	null	null
Extracted ridge points
DfFtR	null	null	null	null
Main ridge		null		null
Roof type	double-eave gable and hip roof	pyramidal roof	double-eave gable and hip roof	double-eave hip roof	flush gable roof	flush gable roof with round ridge	overhanging flush gable roof

illustrates the classification details of each test Ming and Qing Dynasty official-style architecture roof from dataset 1, 2 and 3, respectively. The extraction process of each test architecture roof is listed as follows:

• NoREs extraction. As is shown in the second row of Table 5, for the Hall of Complete Harmony, LiJing Xuan, Gate of Lasting Happiness and the collected example, the projective areas kept stable at first; subsequently, the projective areas became smaller after the height was beyond the eaves. The NoREs from these architectures was 1 and the roofs of these architectures were grouped into the single-eave roofs. For the generated histogram of the Gate of Supreme Harmony, BaoGuang Hall and the Meridian Gate, there were two intervals where the projective area remained unchanged. The NoREs from the three architectures was 2. The roofs of the three architectures were categorized into the double-eave roofs. The extracted roof of each test architecture is shown in the third row of Table 5.
• SoRs extraction and reorganization. The extracted ridge points from each architecture can be seen in the fifth row of Table 5. The experimental results showed our proposed method could obtain the correct structure of the ridges of each architecture. Based on the extracted SoRs, the roof of the Meridian Gate was distinguished as a double-eave hip roof and the roof of the Hall of Complete Harmony was classified into the pyramidal roof category. The unclassified double-eave gable and hip roof type contained the roofs of Gate of Supreme Harmony and BaoGuang Hall. The roofs of other architectures were categorized as unclassified single-eave roofs.
• DfFtR detection. For the unclassified single-eave roofs, the DfFtR of the collected 3D model example was $0.32 \mathrm{m}$ and the outlines of the facades and roofs from LiJing Xuan and Gate of Lasting Happiness were almost the same as is shown in the sixth row of Table 5. The roof of the collected 3D model example was regarded as an unclassified overhanging gable roof and the roof of LiJing Xuan and Gate of Lasting Happiness were grouped as unclassified flush gable roofs.
• DoPP detection. As is shown in the seventh row of Table 5, except the Gate of Lasting Happiness, the density of points located in the areas around the main ridge from other unclassified roofs was higher than that in other areas. After this step, the Gate of Lasting Happiness was categorized as a flush gable roof with round ridge and other unclassified roofs were classified correctly.

The experimental results at each stage were consistent with the theoretical results and all the test architecture roofs were classified into the correct categories. This shows that our proposed method could get a good performance for the Ming and Qing Dynasty official-style architecture roof classification.

6.2.2. The Density of Point Cloud Sensitivity Analysis

The features extraction which relied heavily on the roof and ridge extraction methods had a significant effect on the experimental results. The density of the point cloud seriously influenced the roof and ridge extraction. To verify the robustness of our proposed methods, the sensitivity of the point cloud density should be analyzed. The point cloud of BaoGuang Hall was generated by UAV images, the point cloud of Gate of Lasting Happiness was created from the 3DsMAX and the point cloud of Hall of Complete Harmony was captured by terrestrial laser scanning technology, separately. Three point clouds came from different data sources. On the other hand, the roof types of BaoGuang Hall, the Hall of Complete Harmony and the Gate of Lasting Happiness were different. To ensure the robustness and accuracy of the analysis, we selected the point cloud of BaoGuang Hall, the Gate of Lasting Happiness and the Hall of Complete Harmony as the analysis objects and rarefied the point cloud density to 1/10 and 1/100 of the original density separately in this analysis process.

For the roof extraction, the shape of histogram generated by the projective areas along the Z direction and the extracted roof almost were the same with different point densities as is shown in

Table 6. The BaoGuang Hall roof recognition process based on point cloud with different density.

Density $\left(\mathbf{p}\mathbf{o}\mathbf{i}\mathbf{n}\mathbf{t}\mathbf{s}/{\mathbf{m}}^2\right)$	The Histogram of the Projective Areas	Roof	Ridges	Main Ridges
59,737
5974
597

,

Table 7. The Hall of Complete Harmony roof reorganization process based on point cloud with different density.

Density $(\mathbf{p}\mathbf{o}\mathbf{i}\mathbf{n}\mathbf{t}\mathbf{s}/{\mathbf{m}}^2)$	The Histogram of the Projective Areas	Roof	Ridges	Main Ridges
43,416				null
4342				null
434				null

and

Table 8. The Gate of Lasting Happiness roof reorganization process based on point cloud with different density.

Density $(\mathbf{p}\mathbf{o}\mathbf{i}\mathbf{n}\mathbf{t}\mathbf{s}/{\mathbf{m}}^2)$	The Histogram of the Projective Areas	Roof	Ridges	Main Ridges
95
9.5
1

. Theoretically, when the average distance between two nearest points was beyond the distance from the roof eave to the ground or distance between two roof eaves, the roof extraction would fail. However, this is an extreme case. In this circumstance, the density of point cloud would be too small and could not meet the requirement of 3D modeling. The results showed that the densities of the point cloud had little impact on the roof extraction.

In the ridge extraction stage, the ridges extracted from the Hall of Complete Harmony point cloud with different densities were the same as is shown in Table 7. For BaoGuang Hall, starting from the junction of the diagonal ridge and the vertical ridge, there was a short line segment when the density of point cloud was 1 percent of the original density as is shown in the fourth row and column of Table 6. In fact, a blank area between the diagonal ridge and the vertical ridge for gable and hip roof did exist as is shown the circle in Figure 7a. Due to the fact that the size of this blank area was small, it was completely filled after the morphological close operation with a $7\times 7$ operator in our proposed ridge extraction method when the density of the point cloud was $\mathrm{59,737}$ $\mathrm{points}/{\mathrm{m}}^2$ and$5974 \mathrm{points}/{\mathrm{m}}^2$, separately. When the density of the point cloud was lower than a specified value, the distance between the points located on the diagonal ridge or the vertical ridge becomes large and the distance between two grids which contained the projective points was beyond the size of the operator. This resulted in this blank area being preserved after morphological close operation. Hence, the short line segment was generated after the Skeletonization algorithm was performed. Similarly, the phenomenon that the vertical ridge of the Gate of Lasting Happiness was broken when the point cloud density was at minimum as in the fourth row and column of Table 8 was also caused for this reason. Although the extracted ridges had some differences when the densities of the point cloud were different, the generated SoRs was still a subgraph of the true ridge structure graph.

For the main ridge detection, under different point cloud density conditions, the main ridges of BaoGuang Hall were detected as is shown in the fourth column of Table 6. The distribution of the point cloud density on the$\mathrm{XOY}$ plane as is shown in the fourth column of Table 8 illustrated that Gate of Lasting Happiness did not have a main ridge. The experimental results were consistent with the true roof types. Notably, when the distance between points was greater than the height or width of the main ridge, our proposed method may not have detected the main ridge. This is mainly because the points located on the surface of the main ridge did not exist. This is an extreme case. In this paper, we suppose that the point cloud can represent the details of the Ming and Qing official-style architecture. Based on the above analysis, our proposed method is robust to the test point clouds with different densities.

Moreover, the gaps, noise and occlusion of point clouds also influenced the experimental results. In order to ensure that the point cloud data can successfully complete the 3D modeling, the quality of the collected point clouds should meet a certain criterion in practical engineering. Hence, we can ignore these factors in our experiments.

7. Conclusions and Future Work

This paper proposes a method for the classification of the Ming and Qing official-style architectures from 3D point cloud combining the ridges and other features. The highlights of this work are listed as follows:

• The features including NoREs, DfFtR, DoPP and SoRs are selected for the classification of the Ming and Qing official-style architecture roof and the corresponding feature extraction methods are proposed.
• The attributed relational graphs of the ridges from different roof types of the Ming and Qing official-style architecture are constructed and the Ullmann algorithm is applied to complete the initial roof type reorganization task based on SoRs.
• A hierarchical semantic network is proposed to distinguish the type of the Ming and Qing official-style architecture roof and the thresholds used in this semantic network are estimated based on the construction rules of the Ming and Qing official-style architecture. Based on the proposed method, all the selected Ming and Qing official architecture roofs are classified into the correct categories. The experimental results shows that our proposed method can achieve good performance and have robustness.

Additionally, it is worth noting that the classification of the Ming and Qing official-style architecture is not the terminal goal of our work. We hope to reconstruct a historical building information model (HBIM) of the Ming and Qing official-style architecture automatically especially in the geometric parameter acquisition of the model relying on the roof type. In previous work [70,71], how to obtain the single building and how to reconstruct the decorative component have been solved. The future work will include: (1) the Ming and Qing official-style architectural component segmentation and extraction; (2) parameterization of components; (3) the HBIM reconstruction mechanism; and (4) using various strategies to ensure the point cloud can be processed in memory.