Xiucheng Yang
email: xiucheng.yang@insa-strasbourg.fr

Pierre Grussenmeyer
email: pierre.grussenmeyer@insa-strasbourg.fr

Mathieu Koehl
email: mathieu.koehl@insa-strasbourg.fr

Hélène Macher
email: helene.macher@insa-strasbourg.fr

Arnadi Murtiyoso
email: arnadi.murtiyoso@insa-strasbourg.fr

Tania Landes
email: tania.landes@insa-strasbourg.fr

Review of built heritage modelling: Integration of HBIM and other information techniques

Keywords: built heritage, BIM, parametric objects, semantics, geometric model, knowledge

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Background

Heritage is our legacy from the past, what we live with today, and what we pass on to future generations. The documentation of built heritage involves the 3D modelling of the geometry and information management of semantic knowledge. Currently, lots of information techniques benefit for built heritage modelling, management and conservation, including 3D computer graphics, photogrammetry, laser scanning, Geographic Information System (GIS), ontology and recently developed Building Information Modelling (BIM).

The traditional concept of built heritage modelling entails either accurate geometric modelling or virtual reconstruction, such as the open-source platforms of MicMac 1 (photogrammetry), MeshLab 2 (3D triangular mesh) and 3DHOP (3D Heritage Online Presenter) [START_REF] Potenziani | 3DHOP: 3D Heritage Online Presenter[END_REF]. These 3D models, mainly from 3D computer graphics, photogrammetry and laser scanning, aspire to illustrate the shape of the heritage yet do not make access to additional information possible [START_REF] Meyer | A web information system for the management and the dissemination of Cultural Heritage data[END_REF]. In addition to highly accurate geometry, more considerable efforts are needed for the management of historical buildingss with more details (semantic, attribute, material, temporal and relationship) of the object and its sub-elements.

Heterogeneous knowledge is another essential asset for built heritage. A complete representation of architectural heritage also requires a large amount of knowledge related to its history, physical configuration and condition and a high number of social, political, economic, and cultural issues relating to the external environment [START_REF] Simeone | Bim and Knowledge Management for Building Heritage[END_REF]. Knowledge and information model, from ontology and GIS, play important roles in the knowledge management and analysis of the attribute, semantic and relationship information for the heritage documentation. But they are limited to describe the detailed geometric information, especially referring to the complex and irregular structures of heritage.

The recent developed Building Information Modelling (BIM) technique combines 3D modelling and information management. One of its modern application is heritage documentation and conservation [START_REF] Murphy | Historic Building Information Modelling -Adding intelligence to laser and image based surveys of European classical architecture[END_REF][START_REF] Yang | Complementarity of historic building information modelling and geographic information systems[END_REF][START_REF] Jordan-Palomar | Protocol to Manage Heritage-Building Interventions Using Heritage Building Information Modelling ( HBIM )[END_REF][START_REF] Reinoso-Gordo | Cultural Heritage conservation and sustainability based on surveying and modeling: The case of the 14th century building Corral del Carbón[END_REF][START_REF] Laefer | Toward automatic generation of 3D steel structures for building information modelling[END_REF] and has generated a new concept of Historic/Heritage Building Information Modelling (HBIM). Murphy et al. [START_REF] Murphy | Historic building information modelling (HBIM)[END_REF] firstly defined HBIM in 2009, which clearly expressed the prospects of the BIM technique in historic scenes. HBIM of a historical building can provide: (i) a complete survey and parametric modelling on the 1 MicMac: https://micmac.ensg.eu/ (Accessed on April 20, 2020) 2 MeshLab: http://www.meshlab.net/ (Accessed on April 20, 2020) geometry aspect; (ii) attribute, material and relationship information of the sub-elements; (iii) possible deformations and changes over time.

In brief, four kinds of 3D models (Figure 1) are discussed in this paper. (i) Geometric model reconstructs the real entity with a holistic shape. (ii) The semantic model recognises and endows the meaning of the elements of heritage. (iii) Parametric model defines and alters the shape of the elements by parameters [START_REF] Barazzetti | Parametric as-built model generation of complex shapes from point clouds[END_REF][START_REF] Antón | Accuracy evaluation of the semi-automatic 3D modeling for historical building information models[END_REF][START_REF] Nieto | Management of built heritage via HBIM Project: A case of study of flooring and tiling[END_REF]. (iv) Information model builds a database with the attached attribute, material, and relationship information about the entity and its components as well as the 3D geometry. Besides, a knowledge model is considered, which pays more attention to heterogeneous semantic details of the heritage instead of geometric aspect. Chane et al. [START_REF] Chane | Integration of 3D and multispectral data for cultural heritage applications: Survey and perspectives[END_REF] surveyed the imaging systems and their applications for the study of cultural heritage. Remondino [START_REF] Remondino | Heritage recording and 3D modeling with photogrammetry and 3D scanning[END_REF] reviewed the actual optical 3D measurement sensors and 3D modelling techniques, with their limitations and potentialities, requirements and specifications on heritage recording. Rüther et al. [START_REF] Rüther | Laser scanning in heritage documentation: The scanning pipeline and its challenges[END_REF] presented the pipeline and challenges of laser scanning in heritage documentation. Vanegas et al. [START_REF] Vanegas | Modeling the Appearance and Behavior of Urban Spaces[END_REF] provided an overview of methods spanning computer graphics and related fields involved in complex building modelling, rendering, visualization and simulation. Yang et al.

[5], Pocobelli et al. [START_REF] Pocobelli | BIM for heritage science: a review[END_REF], and Logothetis et al. [START_REF] Logothetis | Building Information Modelling for Cultural Heritage: A review[END_REF] described the potential and challenges of HBIM on historical buildings modelling, while Bruno and Roncella [START_REF] Bruno | HBIM for Conservation: A New Proposal for Information Modeling[END_REF] introduced the capability and workflow of HBIM on heritage conservation. However, there still does not exist a synthetic review of (i) what kinds of roles of these techniques on heritage documentation, and (ii) how to extend the capability of HBIM by integration to other techniques.

In this paper, we revealed the research trend in heritage documentation in a broader sense from the conventional 3D modelling and knowledge management to recently developed HBIM. Also, we present the primary roles of HBIM on heritage documentation and pay attention to how to extend the functions of HBIM and what kinds of problems exist when integrating it with other information techniques. Therefore, virtual reconstruction may resurrect the past once more, virtualize the current, and predict the future; yet the visual models are mostly 2.5D models.

3D modelling

Currently, advances in computer graphics allowed the parametric design of heritage composing of any irregular shaped object with high accuracy and very close to the truth by using modelling tools (such as Rhino 3D 3 , 3ds Max 4 , Maya 5 , and Blender 6 ). The parametric design consists of variable parameters, such as the values of all sizes, and invariable parameters, such as the relationships between the geometric elements.

The system can automatically maintain the constant settings when the variable parameters change. After predefinition of a set of programs, the revision of the built model just needs to modify several invariable parameters [START_REF] De Luca | Reverse engineering of architectural buildings based on a hybrid modeling approach[END_REF][START_REF] Barazzetti | Creation of Parametric BIM Objects from Point Clouds Using Nurbs[END_REF][START_REF] Banfi | Building Information Modeling for cultural heritage: The management of generative process for complex sistorical buildings[END_REF]. Also, 3D computer graphics used procedural modelling to model buildings with similar and repetitive structures by using a rule-based description in a rapid manner [START_REF] Vanegas | Modeling the Appearance and Behavior of Urban Spaces[END_REF][23] [START_REF] Zotti | Tangible heritage: Production of astrolabes on a laser engraver[END_REF]. Procedural modelling has succeeded in modern urban building modelling [START_REF] Wonka | Instant Architecture[END_REF] [START_REF] Schwarz | Advanced procedural modeling of architecture[END_REF], and some works have extended it to different styles of historic architectures [START_REF] Berndt | Generative 3D models: A Key to More Information within Less Bandwidth at Higher Quality[END_REF][28] [START_REF] Merrell | Model synthesis: A general procedural modeling algorithm[END_REF]. For example, a Generative Modelling Language (GML), 3 Rhino 3D: https://www.rhino3d.com/ (Accessed on April 20, 2020) 4 3ds Max: https://www.autodesk.com/products/3ds-max/ (Accessed on April 20, 2020) 5 Maya: https://www.autodesk.com/products/maya/ (Accessed on April 20, 2020) 6 Blender: https://www.blender.org/ (Accessed on April 20, 2020) allowing the generation of highly complex objects based on a set of formal construction rules, is developed and performed the procedural construction of Gothic church windows efficiently [START_REF] Havemann | Generative parametric design of gothic window tracery[END_REF].

Reality-based modelling

Until now, multi-platform-based imagery and laser scanning data are all used to geometrically model the heritage, which focuses on the outline structures and measurement information for the real object. It is no longer a problem to obtain an accurate 3D geometric model, either by a laser scanner or imagery dense matching approaches [START_REF] Grussenmeyer | Basics of range-based modelling techniques in cultural heritage 3D recording[END_REF]. Although aerial photogrammetry and Light Detection and Ranging (LiDAR) can be used to generate the Digital Surface Model (DSM) of heritage sites, it is difficult to meet the demands of heritage applications with regard to accuracy and detail [START_REF] Gimenez | Review: Reconstruction of 3D building information models from 2D scanned plans[END_REF] [START_REF] Höhle | Oblique Aerial Images and Their Use in Cultural Heritage Documentation[END_REF][START_REF] Yang | Building Faç ade Recognition Using Oblique Aerial Images[END_REF]. Built heritage modelling requires more complex and accurate geometric representations, so terrestrial data combined with low-altitude UAV data have frequently been adopted thus far [START_REF] Remondino | Image-based 3D modelling: A review[END_REF] [START_REF] Bernardini | The 3D model acquisition pipeline[END_REF][START_REF] Oreni | Survey turned into HBIM: the restoration and the work involved concerning the Basilica di Collemaggio after the earthquake (L'Aquila)[END_REF].

Photogrammetry and laser scanning have generated the accurate 3D point cloud and holistic mesh to record the shape of heritage. The current challenge is to extend these holistic geometries to semantic [START_REF] Gröger | CityGML -Interoperable semantic 3D city models[END_REF], parametric [START_REF] Xiong | Automatic creation of semantically rich 3D building models from laser scanner data[END_REF] and knowledge [START_REF] Drap | Ontology-Based Photogrammetry Survey for Medieval Archaeology: Toward a 3D Geographic Information System (GIS)[END_REF] models. Particularly, it is anticipated to realise semantic segmentation and labelling of sub-elements (from façades and roofs to columns, openings, etc.) and microstructures (sculptures and unique structures peculiar to heritage), and related fields, such as pattern recognition and machine learning, are rapidly developing. For example, many approaches have been proposed to label façade elements, typically by inverse procedural modelling [START_REF] Musialski | A survey of urban reconstruction[END_REF], RANSAC based segmentation [START_REF] Boulaassal | Automatic segmentation of building facades using Terrestrial Laser Data[END_REF], region growing algorithms [START_REF] Pu | Knowledge based reconstruction of building models from terrestrial laser scanning data[END_REF], Manhattan World assumption [START_REF] Vanegas | Modeling the Appearance and Behavior of Urban Spaces[END_REF], and grammar-based methods [START_REF] Ripperda | Application of a Formal Grammar to Facade Reconstruction in Semiautomatic and Automatic Environments[END_REF][START_REF] Becker | Grammar Supported Facade Reconstruction From Mobile Lidar Mapping[END_REF].

However, these automatic and semiautomatic segmentation works assumed buildings as a regular structure with crossing and repetitive pattern, which excluded the complex historical buildings.

Information and knowledge management

3D computer graphics, photogrammetry and laser scanning paid attention to the documentation of the geometric aspect of built heritage. On the contrary, GIS and ontology provide the enhanced relative database for the management and analysis of the semantic, attribute and relationships among the sub-elements composing heritage. GIS "lets us visualise, question, analyse and interpret data to understand relationships, patterns and trends" [START_REF]Available Online[END_REF], which focuses on managing the spatial relationship and semantic information [START_REF] Catulo | Simplified evaluation of seismic vulnerability of Lisbon Heritage City Centre based on a 3DGIS-based methodology[END_REF][START_REF] Liang | Semantic-based 3D information modelling and documentation of rockeries in Chinese classical gardens: A case study on the rockery at Huanxiu Shanzhuang, Suzhou, China[END_REF]. Ontology is used to formally represent knowledge and rules of a particular domain to facilitate computer processing, reasoning, knowledge sharing and re-use [START_REF] Kuster | The UDSA ontology: An ontology to support real time urban sustainability assessment[END_REF][START_REF] Corona | Coyote drums and jaguar altars: Ontologies of the living and the artificial among the K'iche' Maya[END_REF][START_REF] Simeone | Extending BIM approach to historical and archaeological heritage representation[END_REF].

Geographic Information System

GIS pays attention to managing, querying and analysing spatial, attribute, and relationship information among elements. As a tool for heritage site management, GIS can establish databases for archaeological sites and monuments at regional and national levels [START_REF] Richards | Recent trends in computer applications in Archaeology[END_REF] [START_REF] Schwerin | The mayaarch3d project: A 3D webgis for analyzing ancient architecture and landscapes[END_REF]. GIS provides tools to query the data for preservation purposes and to realise advanced analysis [START_REF] Cerutti | Architectural Heritage Semantic Data Managing and Sharing in GIS[END_REF]. Also, 3D WebGIS is widely adopted for 3D built heritage models to be visited on websites, which benefits the product management and popularity. In this manner, a user-centred system is obtained to extend the interoperability and exchange of data to reach more potential users [START_REF] Bianco | A database for the architectural heritage recovery between Italy and Switzerland[END_REF] [START_REF] Pisu | Cloud GIS and 3D modelling to enhance Sardinian Late Gothic architectural heritage[END_REF][START_REF] Acierno | Architectural heritage knowledge modelling: An ontology-based framework for conservation process[END_REF]. For example, a 3D WebGIS for archaeological research, called the MayaArch3D project [START_REF] Auer | Web-based Visualization and Query of semantically segmented multiresolution 3D Models in the Field of Cultural Heritage[END_REF], offers a system architecture for managing 3D models and their associated data so that researchers can view them on the Internet in real-time, analyse them, and compare them with other models.

GIS is not only a management and analysis tool but also a modelling tool that has been widely used to procedurally model large-scale heritage scenes [START_REF] Schwarz | Advanced procedural modeling of architecture[END_REF]. For example, ESRI CityEngine [START_REF]Esri CityEngine-Advanced 3D city design software[END_REF] is a tool that allows for the efficient modelling of 3D scenes at large scale and with arbitrary detail. The ancient Roman town, Pompeii, was automatically reconstructed utilizing CityEngine on the basis of ground plans (footprints) and drawings/sketches of selected building types [START_REF] Müller | Automatic reconstruction of Roman housing architecture[END_REF]. The famous Rome Reborn 2.0 [START_REF] Dylla | Rome reborn 2.0: A case study of virtual city reconstruction using procedural modeling techniques[END_REF] reconstructed approximately 7000 domestic buildings using grammar-based procedural modelling [START_REF] Dore | Semi-Automatic Modelling of Building Façades With Shape Grammars Using Historic Building Information Modelling[END_REF] and CityEngine based on manual creation of typical monuments and buildings with computer graphics techniques.

Ontology

GIS is characterized by managing spatial-related information. By comparison, an ontology provides a complete representation of built heritage related to its history, physical configuration and condition and a high number of social, political, economic, and cultural issues relating to the external environment [START_REF] Simeone | Extending BIM approach to historical and archaeological heritage representation[END_REF] [START_REF] Simeone | Bim and Knowledge Management for Building Heritage[END_REF].

It is increasingly used to overcome the barriers to heterogeneous semantic data sharing and integration about cultural heritage. Specifically, several advantages of ontology semantics in built heritage include (i) Homogenous representation. The main advantage of using ontologies is the ability of homogenous representation and management of all the knowledge related to heritage in a unified semantic network, including concepts, relationships, functions, rules and constraints [START_REF] Acierno | Architectural heritage knowledge modelling: An ontology-based framework for conservation process[END_REF]. (ii) Sharable and readable in a scientific community. Ontology semantics can share consensual knowledge agreed by a scientific community to increase the interoperability in individuals and organizations [START_REF] Messaoudi | An ontological model for the reality-based 3D annotation of heritage building conservation state[END_REF] [START_REF] Quattrini | Conservation-oriented HBIM. The bimexplorer web tool[END_REF]. (iii) Understandable by a computer. The rules defined by an ontology allow checking the data and inferencing new knowledge [START_REF] Messaoudi | An ontological model for the reality-based 3D annotation of heritage building conservation state[END_REF][57] [START_REF] Kim | Integration of ifc objects and facility management work information using Semantic Web[END_REF].

In the cultural heritage context, International Committee for Documentation Conceptual Reference Model (CIDOC CRM) is the main ontological reference model [START_REF] Doerr | The CIDOC Conceptual Reference Module: An Ontological Approach to Semantic Interoperability of Metadata[END_REF]. This ontology was initially towards the establishment of standard description and maintenance of museum artefacts. It became the ISO standard 7 in 2006 to allow the formal and highly specific representation of information about historical buildings. To date, CIDOC CRM facilitates integration and interchange of heterogeneous cultural heritage information and support operations of reasoning and inference. While the CIDOC CRM model was developed mainly to manage the cataloguing of cultural heritage documentation, some other domain-specific ontologies have been progressively introduced to represent other aspects of the heritage conservation process [START_REF] Acierno | Architectural heritage knowledge modelling: An ontology-based framework for conservation process[END_REF][START_REF] Zalamea Patino | Merging and expanding existing ontologies to cover the Built Cultural Heritage domain[END_REF]. Geneva City Geographic Markup Language ontology (Geneva CityGML) [START_REF] Kolbe | Representing and Exchanging 3D City Models with CityGML[END_REF] is an ontological equivalent of the CityGML standard and allow to create 3D models of historical buildingss. The Monument Damage Information System (MONDIS) supports the documentation and analysis of damaged historical structures, their diagnosis, and possible interventions [START_REF] Cacciotti | Monument damage information system (mondis): An ontological approach to cultural heritage documentation[END_REF].

BIM and HBIM

BIM was originally used for the management of the life-cycle construction process in the architectural industry. As-built BIM contributes to the representation of physical and functional characteristics of existing buildings by reality-based data acquisition and reverse engineering [START_REF] Arayici | Towards building information modelling for existing structures[END_REF]. The aim of introducing the BIM/as-built BIM technique into heritage modelling underlines the strict relation between object modelling and information involving various attributes, temporal change and spatial constraints [START_REF] Oreni | Hbim for Conservation and Management of Built Heritage: Towards a Library of Vaults and Wooden Bean Floors[END_REF] [START_REF] Brumana | Strategy for integrated surveying techniques finalized to interpretive models in a Byzantine Church, Mesopotam, Albania[END_REF].

HBIM is suitable for the recreation of a historical building from the existing description data (such as historical documents, bibliographic references, photographs, drawings, etc.), especially for the past/disappeared heritage. As-built HBIM is increasingly used to restore, model and manage existing valuable constructions and historic structures by using reality-based recording data (typically 3D point cloud from laser scanning and photogrammetry).

According to the description documents (for HBIM) and reality-based recording data (for as-built HBIM), a library of parametric and semantic elements (such as wall, roof, stair, door and window) can be created and then used to establish the heritage [START_REF] Fernández | BIM aplicado al Patrimonio Cultural. Documento 14. Guía de usuarios BIM. Building SMART Spain Chapter[END_REF][START_REF] Antonopoulou | BIM for Heritage[END_REF][START_REF] Parisi | The value of knowledge through H-BIM models: historic documentation with a semantic approach[END_REF][START_REF] Almaimani | BIM-Driven Design Approach for Islamic Historical Buildings[END_REF][START_REF] Abd Elwahab | Towards a parametric plug-in for conservation of built heritage[END_REF]. The model is not only a virtual representation of the construction. It is a vital part of the project, where the different elements of the building become advanced 7 CIDOC CRM: http://www.cidoc-crm.org/ (Accessed on April 20, 2020) objects with parametric intelligent, including the quantitative and qualitative description and strict relationship information. As a typical parametric design application, HBIM realises the linkage of the information storage and modification, in which the elements can be modified by altering parameters and the relations to other objects remain unchanged.

Currently, the HBIM modelling workflow tends to combine both descriptive and reality-based data to document the built heritage [START_REF] Murphy | Historic Building Information Modelling -Adding intelligence to laser and image based surveys of European classical architecture[END_REF][START_REF] Bagnolo | HBIM for archaeological sites: from SFM based survey to algorithmic modeling[END_REF][START_REF] León-Robles | Heritage Building Information Modeling (H-BIM) Applied to A Stone Bridge[END_REF]. BIM software is firstly used to construct the parametric elements regarding the historical buildings based on documentation. The application of inverse engineering, which maps the parametric elements to the reality-based data, makes it possible to refine the parameters and obtain a current-state diagnosis [START_REF] Ludwig | The advantages of parametric modeling for the reconstruction of historic buildings. The Example of the in war destroyed Church of St. Catherine (Katharinenkirche) in Nuremberg[END_REF] [START_REF] Fai | Establishing an Appropriate Level of Detail (LoD) for a Building Information Model (BIM) -West Block[END_REF]. Therefore, an HBIM project needs to organise the protocol created simultaneously or interactively by analysing documentation-and reality-based data. That is, how to fit the standard shapes and BIM components into the 3D point cloud are essential processing that could be performed via manual interpretation (currently) and 3D structure similarity measurement [START_REF] Zhang | A fast method for measuring the similarity between 3d model and 3d point cloud[END_REF].

Brief Summary

As in the above description, each technique has an advantage in built heritage modelling and covers the heritage documentation from data sources to different levels of models (Figure 2). 3D computer graphics focus on accurate description using parametric design, large-scale areas using procedural modelling and virtual reconstruction with enhanced visual effect. Reality-based shape information is obtained by photogrammetry and laser scanning with non-contact tools, which is accordant with the concept of heritage conservation. Automated or semi-automated algorithms are expected to realize semantic segmentation of the holistic geometry. GIS is used to manage the attribute and spatial information, and webGIS provides an available web access tool for 3D models. Ontology deals with the heterogeneous knowledge describing the heritage. Recently developed BIM techniques provide a more powerful environment that combines 3D geometric modelling and information management.

Literature Analysis

On the basis of the Web of Science TM Core Collection, we indexed the publications with combinations of key terms and analysed the tendency of each technique. To limit the broad scope to 3D built heritage modelling, structured queries on Web of Science using combinations of key terms and their synonyms (heritage/"historic* building*" AND 3D) were conducted, yielding a total of 2848 records until the year of 2019. Then, to analyse the transformation from geometric modelling to semantic-and information-enriched heritage modelling, typical publications utilizing BIM techniques are marked.

Due to the applied query method, the review excludes research currently underway that is not available in the mentioned databases, journals or conferences that are not included in the Web of Science TM Core Collection, or studies that have not been published in English yet.

What types of techniques are popular within certain time span

We firstly take related key terms (Table 1) to monitor the publications on heritage documentation using different techniques. Two periods are considered, including all years in the database (from 1975 to 2019) and 2010s. The number of publications can reflect the focuses of the researchers in the field of heritage documentation. The result shows heritage documentation draws increasing attention, considering that 87% of the works are published in the 2010s.

A rate (Eq. 1) is calculated by the ratio of publication numbers during the 2010s among the total publications, which are dedicated to mirroring the change of the hotspot issues. That is, the higher the value is, the more attention has been paid in recent days. As shown in Figure 3, built heritage modelling (with the rate 87%) has been attracting emphasis in the 2010s, which serves as the baseline to find the increasingly popular and potential techniques in the field of heritage documentation. Through this study, it becomes clear that photogrammetry is the most popular method in heritage modelling. The reasons may be that photogrammetry can provide not only images for documentation purposes but also accurate geometry models with low costs in time and money. Imagery is also an interesting supplement for laser scanning, and machine learning in computer vision provides new opportunities for image-based semantic modelling.

More and more works in heritage modelling have begun to use laser scanning, computer science, GIS and especially BIM techniques. Laser scanning is becoming cheaper and portable, which broadens its applications. With the easy acquisition of accurate geometry by photogrammetry and laser scanning, the semantic segmentation has become a critical issue. BIM techniques can create the parametric element library, restore the graphics geometry information and non-graphic attribute and relationship information, and manage the 3D temporal model. These advantages make it go further in the necessity of semantic modelling, although the time-consuming manual operation is needed, especially when preserving complex and irregular structures. GIS can manage and analyse the multi-dimensional information in the web access, which is accordant with the heritage conservation and popularity demands. Once BIM and GIS deal well with complex structures, semantic or information modelling of built heritage will be solved. However, there will still be a long road for semi-automatic or automatic parameterization of the elements. Currently, objects in certain scenes can be recognised and labelled with machine learning and pattern recognition approaches. It is highly anticipated to realise the semantic segmentation of 3D heritage geometric models or 2D imagery, although such an application needs a large amount of training data and addresses simple scenes.

The role of ontologies on built heritage documentation has not yet obtained due attention. The early research paid more attention on the management of artefacts [START_REF] Acierno | Architectural heritage knowledge modelling: An ontology-based framework for conservation process[END_REF], similarly to the CIDOC CRM that was originally developed for the standard description and maintenance of museum artefacts. Ontology-based knowledge model for architectural heritage is growing in recent years, especially with the integration of geometric modelling, BIM and GIS.

It seems that computer graphics is developing relatively slowly in the heritage modelling field. The reasons may be that virtual reconstruction has been relatively mature compared to semantic information, and business corporations spend more effort than academic fields on better visual effects.

Roles of BIM techniques in built heritage

We selected about 60 typical articles related to built heritage modelling utilizing BIM techniques (Table 2). Via the research statistics, current BIM in heritage modelling has focused on semantic modelling by historical materials with the BIM concept or reality-based data with the as-built concept. Documentation materials, including 2D drawings [START_REF] Apollonio | 3D Modeling and Data Enrichment in Digital Reconstruction of Architectural Heritage[END_REF], 2D floor plans [START_REF] Lee | Virtual reconstruction and interactive applications for Korean traditional architectures[END_REF], archival documents [START_REF] Del Giudice | BIM for cultural heritage[END_REF] and architectural pattern books [START_REF] Cina | Structural calculations by horizontal and vertical interoperability for the redevelopment of existing buildings[END_REF], are used to re-create the heritage by parameterizing the new elements in the BIM library.

Yet some firstly build a virtual or geometric model using computer graphics software and just improve it to a parametric model using BIM software [START_REF] Oreni | Hbim for Conservation and Management of Built Heritage: Towards a Library of Vaults and Wooden Bean Floors[END_REF][START_REF] Cursi | A semantic web approach for built heritage representation[END_REF]. Laser scanning has been widely used to reconstruct existing heritage and import it into BIM software to build the elements as a reference. Most works utilizing combined data from documentation and photogrammetry just follow the common HBIM process, mapping the parametric elements to the point cloud [START_REF] Dore | Semi-Automatic Modelling of Building Façades With Shape Grammars Using Historic Building Information Modelling[END_REF][START_REF] Dore | Structural Simulations and Conservation Analysis -Historic Building Information Model (HBIM)[END_REF], although several works obtained two separate models with separate procedures under the BIM and as-built BIM concept [START_REF] Oreni | Hbim for Conservation and Management of Built Heritage: Towards a Library of Vaults and Wooden Bean Floors[END_REF][START_REF] Li | Application of BIM in the protection of historical buildings based on Riegel historic value theory[END_REF]. The ideal model deduced from historic architectonic patterns and real geometry can be compared between the past and current [START_REF] He | A gis-based cultural heritage study framework on continuous scales: A case study on 19th century military industrial heritage[END_REF].

Some have also explored specific analytical applications and heritage management on the basis of BIM parametric models. Meanwhile, some works have proposed approaches dedicated to heritage modelling applications while just conducting them in modern buildings at the present stage. Many works have realised the semi-automatic segmentation of the point cloud and built the semantic model in the BIM platform [START_REF] Macher | From Point Clouds to Building Information Models: 3D Semi-Automatic Reconstruction of Indoors of Existing Buildings[END_REF], while current results are conducted in regular buildings and expected the further development of heritage buildings. The irregular and complex shapes of heritage yield more difficulties, especially for semi-automatic element extraction.

There are also three reviews to explore the potential of BIM in heritage modelling. [START_REF] Hichri | Review of the "As-Built Bim" Approaches[END_REF] focused on the 3D modelling of heritage building by point cloud and analysed "as-built" BIM characterization including element shape, relations and attributes; [START_REF] Logothetis | Building Information Modelling for Cultural Heritage: A review[END_REF] mainly summarised the current BIM platforms and illustrated the common HBIM process for historical buildingss; and [START_REF] Pocobelli | BIM for heritage science: a review[END_REF] reviewed the parametric modelling of heritage structures in BIM platform and the potential of HBIM for cultural heritage conservation and management.

Discussion on the integration of HBIM and other techniques

BIM provides the possibility to combine the geometric, semantic and parametric modelling and attribute, temporal and relationship information management to the uniform platform. However, it still exists lots of limitations, especially when it comes to heritage documentation. It is highly anticipated to extend BIM functions on heritage-related issues by integrating traditional information techniques.

Generate complex element by the combination with computer graphics

It is usually a manually time-consuming concept in BIM software to parametrically model the historical elements, and they are limited in their ability to create complicated irregular geometry. That is, the time complexity and geometric accuracy for parameterizing the complex and irregular historic structure elements are the main challenges.

Thus far, there exist mainly two ways to parametrically model the irregular elements not included in the BIM library. The first one is the manual modelling of the elements in a BIM software platform directly (e.g.Revit 8 and ArchiCAD 9 ). The main drawback is its low efficiency and how hard it is to ensure geometric accuracy. The other is the resort to programming, including the Geometric Description Language (GDL) of ArchiCAD library parts and Autodesk Dynamo visual programming for Revit. The parametric GDL and Dynamo elements could be completely described as 2D symbols, 3D geometric models, attribute specifications and additional spatial relationships in the BIM platform.

Both the computer graphics and the original BIM concept for built heritage modelling have formed documentation-based parametric modelling using historical materials, such as historical documents, bibliographic references, photographs and drawings. BIM goes further than computer graphics to improve the virtual reconstruction of an information-rich model by semanticizing the elements.

Currently, BIM is increasingly paying attention to irregularly shaped modelling by introducing computer graphics modelling techniques. Some works build the irregular structures with the help of computer graphics software, such as Rhino 3D [START_REF] Garagnani | Parametric accuracy: building information modeling process applied to the cultural heritage preservation[END_REF][START_REF] Oreni | Beyond Crude 3D Models : From Point Clouds to Historical Building Information Modeling via NURBS[END_REF] or Sketchup [START_REF] Lin | Improvingtraditional building repair construction quality using historic building information modeling concept[END_REF], which could address any irregularly shaped object compared to BIM platforms [START_REF] Barazzetti | Cloud-to-BIM-to-FEM: Structural simulation with accurate historic BIM from laser scans[END_REF]. For example, Rhino 3D software is able to generate accurate and free-formed models of complex and irregular geometries, especially using NURBS to represent surfaces.

The current tight combination of graphics geometry modelling and BIM semantic modelling, such as the Rhino 3D-Grasshopper-ArchiCAD connection developed by ArchiCAD and Rhino 3D, empowers architects to start and modify their designs in any of the three design environments (Rhino 3D, Grasshopper or ArchiCAD). Also, Rhino 3D builds BIM semantic models and has been expanded to Rhino BIM. The design of Rhino 3DBIM is 3D-centric, where the design is modelled and captured in a 3D database compared to the conventional BIM software.

Automate as-built HBIM based on point cloud processing approaches

Built heritage geometric modelling has often been conducted using reality-based data [START_REF] Grussenmeyer | Comparison methods of terrestrial laser scanning, photogrammetry and tachometry data for recording of cultural heritage buildings[END_REF][START_REF] Karachaliou | UAV for mapping historic buildings: from 3D modelling to BIM[END_REF][START_REF] Martínez-Carricondo | Combination of nadiral and oblique UAV photogrammetry and HBIM for the virtual reconstruction of cultural heritage. Case study of Cortijo del Fraile in Níjar, Almería (Spain)[END_REF]. BIM goes further than the traditional geometric model with parametric and semantic elements. As a 3D information modelling environment, BIM software is increasingly supportive of 3D point cloud representing the entity surface, which is called as a scan-to-BIM process. By importing the point cloud into the BIM software, the solid building components can be created using the existing BIM IFC classes and self-defined structures. 8 Revit https://www.autodesk.com/products/revit (Accessed on April 20, 2020) 9 ArchiCAD: https://www.graphisoft.com/archicad/ (Accessed on April 20, 2020)

Unfortunately, the scan-to-BIM process remains mostly a manual process [START_REF] Fai | Establishing an Appropriate Level of Detail (LoD) for a Building Information Model (BIM) -West Block[END_REF][START_REF] Garagnani | Parametric accuracy: building information modeling process applied to the cultural heritage preservation[END_REF][START_REF] Maietti | Digital Cultural Heritage: Semantic Enrichment and Modelling in BIM Environment[END_REF], and a key challenge today is thus to automate the process leading to as-built BIM from a point cloud. To reduce the manual work in such scan-to-BIM process, many commercial tools and algorithms exist. Many works utilised the commercial scan-to-BIM plugins in Autodesk Revit platform, including ClearEdge3D

Edgewise, IMAGINit Scan to BIM, Pointsense and Leica CloudWorx. Some tried to automate the scan-to-BIM process by self-developed algorithms [START_REF] Macher | From Point Clouds to Building Information Models: 3D Semi-Automatic Reconstruction of Indoors of Existing Buildings[END_REF][START_REF] Thomson | Automatic geometry generation from point clouds for BIM[END_REF][START_REF] Macher | Automating the verification of heritage building information models created from point cloud data[END_REF]. Although they show excellent progress for the automation reconstruction of as-built BIM from point clouds, those works only deal with planar walls and floors and some other regular structures (i.e., rectangular openings and cylinder columns).

That is, most commercial BIM packages and scan-to-BIM algorithms were mainly developed for modern buildings with regular shapes. Historical buildingss are composed of complicated and irregular architectural elements, which makes it problematic to segment and parameterise the elements. For the irregular structures widely existing on built heritage, the current scan-to-HBIM process is time-consuming manual modelling. Although existing BIM platforms are supportive of the point cloud, the parametric modelling tools are still limited because most of BIM software is not 3D-centric modelling environment and cannot directly parametrically design the shape on the reference of the point cloud in 3D space. It is usually a manually time-consuming concept in BIM software to parametrically model the historical elements, and they are limited in their ability to create complicated irregular geometry. That is, the time complexity and geometric accuracy for parameterizing the complex and irregular historic structure elements are the main challenges.

Besides, there have been lots of point cloud processing libraries and algorithms available in the field of reality-based modelling. These approaches pay attention on the semantic segmentation of the holistic geometric models, and they need additional platforms to manage the semantic and relationship information.

In contrast, the relationship and parametric description is the core for (H)BIM technique. The scan-to-(H)BIM obtain the semantic, parametric, attribute and relationship during the modelling process.

The scan-to-(H)BIM is still a new topic and its integration with point cloud processing algorithms can help to reduce the human involvement in the modelling process. But this kind of integration relies on the progress of the BIM platforms to support advanced point cloud processing algorithms.

Information management by BIM-GIS integration

BIM provides high editing functionalities for parametric modelling and achieves semantic models.

But BIM is limited to storing semantic and attribute aspects thus far since it lacks query (attribute and spatial) functions. GIS has a priority in information management (attribute, semantic, and relationship information analysis and management).

The current trend concerns the convergence of both concepts on lots of fields, such as the AEC industry [START_REF] Song | Trends and Opportunities of BIM-GIS Integration in the Architecture, Engineering and Construction Industry: A Review from a Spatio-Temporal Statistical Perspective[END_REF], facility management [START_REF] Mirarchi | Supporting Facility Management Processes through End-Users' Integration and Coordinated BIM-GIS Technologies[END_REF], urban architectural [START_REF] Krijnen | Processing BIM and GIS Models in Practice: Experiences and Recommendations from a GeoBIM Project in The Netherlands[END_REF], and so on. The separate standards, for instance, the two most popular standards of Industrial Foundation Class (IFC) for BIM and CityGML for GIS, seem to be widely accepted for exchanging semantic 3D information and geographical for BIM and GIS [START_REF] Deng | Mapping between BIM and 3D GIS in different levels of detail using schema mediation and instance comparison[END_REF]. During the integration process, some significant details are lost due to the extraction and simplification of data from one system to another [START_REF] Gröger | CityGML -Interoperable semantic 3D city models[END_REF][START_REF] Song | Trends and Opportunities of BIM-GIS Integration in the Architecture, Engineering and Construction Industry: A Review from a Spatio-Temporal Statistical Perspective[END_REF].

When it comes to the historical buildingss composed of complex elements, the BIM-GIS integration is also the trend. It faces the new challenge because both BIM and GIS are originally developed for modern buildings. A typically two-stage outline involving HBIM for 3D modelling and GIS for further management and analysis has been defined by Dore and Murphy [START_REF] Dore | Integration of HBIM and 3D GIS for Digital Heritage Modelling Integration of HBIM and 3D GIS for Digital Heritage Modelling[END_REF]. But it is critical to minimise information loss in the conversion process [START_REF] Saygi | Evaluation of GIS and BIM roles for the information management of historical buildings[END_REF][START_REF] El-Mekawy | A Unified Building Model for 3D Urban GIS[END_REF]. Many studies have been performed to minimise the information loss in integrating BIM and GIS. The common process is to extend the models from BIM to GIS, and a CityGML extension called GeoBIM to get semantic IFC data into the GIS context is developing well [START_REF] De Laat | Integration of BIM and GIS: The development of the CityGML GeoBIM extension[END_REF]. BuildingSMART IFC is also extending itself for GIS project application, such as the new entity for spatial zones, geographic elements and external spaces. A united system integrating GIS and BIM concepts without information loss is not foreseen.

It benefits information consistency among the different BIM software, GIS software and other modelling software if the representation of the complexity of the irregular shapes of the elements tends to be simplified. Unfortunately, when the simplification causes the loss of the details of the architectural surface, it goes against the aim of heritage conservation. The complexity of many elements and the lack of unified criteria make it difficult to simultaneously minimise information loss and optimise geometric architectural conservation during the exchange process between the two types of information systems.

Heterogeneous knowledge management by the connection with ontology

HBIM provides a modelling environment where geometrical information and some intangible data are structured in a coherent database, and GIS provides a management environment where geometrical, attribute and relationship can be queried and analysed. But they still leave out of the model a large amount of knowledge, not directly relatable to heritage's components but still needed to fully represent the object [START_REF] Simeone | Extending BIM approach to historical and archaeological heritage representation[END_REF].

Thus, ontology plays increasingly important roles in connection with HBIM to comprehensively represent the historical buildings [START_REF] Pauwels | Integrating building information modelling and semantic web technologies for the management of built heritage information[END_REF][START_REF] Rodrigues Gonçalves | Digital Cultural Heritage: Semantic Enrichment and Modelling in BIM Environment[END_REF][START_REF] Iadanza | Semantic web technologies meet BIM for accessing and understanding cultural heritage[END_REF]:

(i) Knowledge modelling: Ontology-based knowledge model is combined with HBIM to enrich the semantic information. Ontology serves as a data collector to represent a large amount of semantics possibly independent from the built heritage (for example historical context, social information, environmental resources, other heritage information, etc.) [START_REF] Yang | HBIM Modeling from the Surface Mesh and Its Extended Capability of Knowledge Representation[END_REF].

(ii) Information extraction and querying: The combination of ontology can extend the HBIM model with the capabilities of information extraction and querying. The typical workflow is to use Autodesk Revit as HBIM platform and Protégé for editing ontologies and then to export the project database and query the information by Revit DBLink [START_REF] Quattrini | Knowledge-based data enrichment for HBIM: Exploring high-quality models using the semantic-web[END_REF]. Besides, based on ontology-based feature modelling, ifcXML is utilised to conduct query processing to extract information relevant to construction practitioners from a given HBIM [START_REF] Nepal | Ontology-Based Feature Modeling for Construction Information Extraction from a Building Information Model[END_REF].

(iii) Connect and transfer the information between HBIM and GIS: Ontological framework can be built for dealing with the interoperability issue between BIM hierarchy structure and GIS relational database.

HBIM model and GIS model are first mapped to ontology respectively and then connected and queried using ontology rules [START_REF] Karan | BIM and GIS Integration and Interoperability Based on Semantic Web Technology[END_REF][START_REF] Hor | Bim-Gis Integrated Geospatial Information Model Using Semantic Web and Rdf Graphs[END_REF].

(iv) Help to create the parametric model: This kind of combination could be the basis for automated HBIM parametric design, although no work has thus far paid attention to the ontology-based BIM modelling.

Lin [START_REF] Lin | The STG pattern-application of a "Semantic-Topological-Geometric" information conversion pattern to knowledge-based modeling in architectural conceptual design[END_REF] mentioned the possibility to applied ontology to computer graphics parametric design which can automate the modelling work to some degree.

To fully represent and comprehend a historical/archaeological artefact, an integration with BIM and ontology can provide both object-oriented information (typically found in BIM software) and semantic information (typically found in ontological modelling systems). Currently, ontology semantics can be connected to BIM by direct ad-hoc development in BIM platform [START_REF] Simeone | Extending BIM approach to historical and archaeological heritage representation[END_REF], migrating IFC files to RDF/OWL [START_REF] Pauwels | Enhancing the ifcOWL ontology with an alternative representation for geometric data[END_REF] and developing a unified platform [START_REF] Acierno | Architectural heritage knowledge modelling: An ontology-based framework for conservation process[END_REF].

Brief summary

HBIM environment increasingly facilitates the interoperability of data sets and data exchange with reality-based modelling, computer graphics, ontology and GIS. Shown as Figure 4, the integration can extend HBIM capabilities, such as accurately parametric modelling of irregular structures by computer graphics, enhanced semantic segmentation based on reality-based modelling approaches, enhanced management and analysis of semantic knowledge and spatial information, and structural analysis for conservation.

But the integration and transformation among different environments still face various challenges, including how to address the irregular and complex structures in the scan-to-HBIM and HBIM-GIS integration, how to realize the interactive parametric design between computer graphics and HBIM, and how to avoid the information loss during the model transfer.

Conclusions

This paper reviewed studies on heritage documentation using geometric modelling (3D computer graphics and reality-based modelling), knowledge management (GIS and ontology) and recently developed BIM technique. The monitoring of publications indicated that heritage documentation had drawn increasing attention in recent years, considering that 87 percent of papers were published in the 2010s.

Also, the publication in the field of heritage documentation displayed that laser scanning, GIS and especially BIM techniques became increasingly important, photogrammetry was always the most popular method, and ontologies and 3D computer graphics were traditional ways.

This paper also reviewed the typical roles of HBIM, firstly proposed in 2009, in recent years to reconstruct, manage and conserve the built heritage. HBIM provided a unified platform for semantic/parametric modelling, 2D/3D visualization, and spatial/attribute database and management of the built heritage. The integration of existing techniques can extend the capability of HBIM on heritage documentation and management, such as computer graphics for parametric modelling of irregular structures, reality-based modelling for automated semantic segmentation, GIS for spatial information

Figure 1 .

 1 Figure 1. A column described in different types of 3D models, including geometric, semantic, parametric and information model

  (a) Geometric model of holistic entity (b) Semantic model w ith segmented column (d) Information model w ith attached properties (c) Parametric model w ith rule definition

Figure 2 .

 2 Figure 2. The overall scheme of built heritage modelling with combined techniques

= publication numbers during the 2010s all publication numbers ( 1 ) 1 . 409 Figure 3 .

 114093 Figure 3. Publication rate during recent years. The vertical dash line indicates the publication rate in the field of built heritage modelling. Generally, the higher rate of the specific technique means the potential, while the lower rates suggest the traditional or not well-known techniques.

Figure 4 .

 4 Figure 4. Merits (in green colour) and challenges (in red colour) in the integration of HBIM and other techniques.

  1.2.1. 3D computer graphics3D computer graphics is the earliest to model heritage and is normally dedicated to virtual reconstruction. Virtual reconstruction commonly refers to a virtual replica of the heritage object (disappeared, damaged or existing), in which the data source may come from digitalised historical materials recording the heritage, reality-based photographs and measurable drawings representing the surface.

Table 2 .

 2 The roles of BIM in built heritage documentation

	Roles of HBIM	References
	Documentation-bas
	ed (Red)
	3D
	Laser scanning
	parametric
	based (Blue)
	modelling
	Photogrammetry
	based (Green)
	Analysis, management, &
		[6,7,19,51,71,86,90,93-100]
	conservation
	Expected to HBIM	[101-105]
	Review	[17,18,92]

analysis and ontology for heterogeneous semantics management. However, the integration still faces challenges to enhance interoperability and reduce information loss among different platforms. A unified HBIM platform for better heritage documentation can be generated when addressing these challenges.