Joao Santos
email: santosjmo@gmail.com

Veronique Cerezo

Khedoudja Soudani

Sara Bressi

A comparative life cycle assessment of hot mixes asphalt containing bituminous binder modified with waste and virgin polymers

Keywords: Life cycle assessment, Waste nitrile rubber (NBR), EthyleneVinyl-Acetate (EVA), Modified bituminous binder

This paper presents the results of a life cycle assessment undertaken to compare the potential environmental impacts associated with the use of asphalt surface mixtures produced with polymer modified bitumen with those of a conventional asphalt surface mixture. Seven types of hot mix asphalt mixtures to be used in the surface course are compared, among which one produced with virgin materials and conventional binder, which is used as a reference, and six alternative mixtures containing a bituminous binder modified with different percentages of waste nitrile rubber from shoe sole and Ethylene-Vinyl-Acetate polymer. The ILCD impact assessment method at midpoint level was adopted to assess the environmental performance of the wearing course of a French pavement structure over a 30-year project analysis period considering the following pavement life cycle phases: (1) extraction of raw materials, modification of the bituminous binder and mixtures production; (2) transportation of materials, and; (3) construction and maintenance and rehabilitation.

Introduction

Recently, increased traffic levels, heavier loads, and extreme weather conditions have urged road authorities to develop new, or advance existing solutions, in order to improve the resistance of the road pavements to the adverse effects of mechanical and environmental loading.

Concomitantly, as greenhouse gas (GHG) and their effect on the climate are increasingly in the spotlight with respect to policy, legislation and general public's concern, the pavement industry and scientific community have been challenged to improve conventional asphalt mixtures materials by developing more sustainable technologies.

Asphalt binder modification by means of polymer addition, either virgin or recycled and used individually or in a blend mode, has proved to have the potential to enhance the bituminous properties, most typically those related to high temperature performance characteristics, thereby postponing the effects of permanent deformation and fatigue damage [START_REF] Airey | Fundamental binder and practical mixture evaluation of polymer modified bituminous materials[END_REF][START_REF] Kumar | Strength characteristics of polymer modified mixes[END_REF]. However, depending on their nature, the manufacturing of these polymers can also increase the environmental impacts of the asphalt binder in the mixture [START_REF] Van Dam | Towards sustainable pavement systems-a reference document[END_REF].

Therefore, it is of paramount importance to objectively quantify the extent to which the potential environmental benefits associated with the decreasing amount of materials needed over the pavement life cycle as a consequence of the increased service life of pavements is not offset by the environmental impacts originated by their production.

To the best of the authors' knowledge, the only study existing in the literature assessing the environmental impacts resulting from using polymers to modify asphalt binder was performed by [START_REF] Butt | Considering the benefits of asphalt modification using a new technical life cycle assessment framework[END_REF]. However, neither the waste nitrile rubber (NBR) nor the Ethylene-Vinyl-Acetate (EVA) polymers were among those studied.

Given the circumstances above stated, the study presented in this paper aims to investigate the extent to which the use of polymer modified bitumen (PMB) in asphalt mixtures applied in the wearing course of flexible road pavements is beneficial from an environmental point of view.

Nomenclature

Methodology

A comparative process-based life cycle assessment (LCA) study was performed according to the ISO 14040 series [START_REF]International Standard ISO 14040: environmental management -life cycle assessment: principles and framework[END_REF][START_REF]environmental management -life cycle assessment: requirements and guidelines[END_REF] and the University of California Pavement Research Center [START_REF] Harvey | Pavement life cycle assessment workshop: discussion summary and guidelines[END_REF] to compare the potential environmental impacts of different asphalt mixtures adopted in the construction, maintenance and rehabilitation (M&R) of a road pavement section during its life cycle.

The stages adopted in this study include goal and scope definition, inventory analysis, impact assessment, and interpretation.

Goal and scope definition

Goal

The main goal of this study is to compare the potential life cycle environmental impacts arising from the use of asphalt surface mixtures containing different percentages and types of PMB with those of a conventional asphalt surface mixture in the construction and M&R of wearing courses of flexible road pavements (baseline scenario).

Furthermore, in order to assess the robustness of the results to changes in the methodological assumptions, a scenario analysis was performed by considering two alternative scenarios. In the first one, hereafter named AS1, it is assumed that the use of any type of PMB leads to an increase in the durability of the pavement equal to 20%, or in other words 2 years. In the second one, hereafter named AS2, it is assumed that the asphalt plant is run by natural gas rather than by heavy fuel oil (HFO). This is a plausible scenario that stands the best chance of becoming the actual and near-term future practice, thereby replacing the baseline scenario, as new asphalt plants are increasingly switching to natural gas because of its general lower price and cleaner-burning properties.

The comparative findings of this study are intended to be used by highway agencies and pavement practitioners to make more assertive judgments on the pros and contras associated with the use of binder modifiers agents for enhancing the performance of road pavements throughout their life cycle.

System description and boundaries

The system boundaries define the unit processes considered in the LCA study and were drawn to cover the following pavement life cycle phases, modeled through individual but interconnected modules: (1) extraction of materials and mixtures production, consisting of the acquisition and processing of raw materials, and the mixing process of asphalt mixtures in plant; (2) construction and M&R, including all construction and M&R procedures and related construction equipment usage; (3) transportation of materials, accounting for the transportation of materials to and from the construction site and between intermediate facilities; and (4) end-of-life (EOL), which models the destination of the pavement structure after the project analysis period (PAP).

The upstream emissions and resources consumption associated with the production of the energy sources used to power the different processes and construction equipment were also included in the system boundaries. On the other hand, construction equipment, road-related safety and signaling equipment (including road marking), road accessories (fences, road lighting software, etc.), construction and M&R of the remaining layers of the pavement structure and the earthworks required to build the platform over which the pavement foundation will be built were not included in the system boundaries. The environmental impacts resulting from the construction and M&R of the remaining layers were disregarded because their geometry and type of mixture is the same regardless of the type of the mixture considered in the wearing course.

Functional unit

The functional unit is the central core of any LCA and forms the basis for comparisons between different systems with the same utility for the same function. In the pavement domain, this means a unit of pavement that can safely and efficiently carry the same traffic over the same PAP. Then, it is defined by their geometry, service life and level of traffic supported.

The functional unit of the case study presented in this paper is a typical French highway section of 1-km length, composed of two independent roadways, each with 2 lanes with an individual width of 3.5m. The PAP is 30 years, starting in 2015. The initial two-way average annual daily traffic (AADT) was considered to be equal to 6500 vehicles/day, of which 33% are heavy duty vehicles (HDV) equality divided between rigid HDV and articulated HDV.

The geometric characteristics of the wearing course considered are presented in Figure 1. As for pavement maintenance, a pavement M&R strategy derived from French practice was considered in the baseline scenario [START_REF] Laurent | Evaluation économique des chaussées en béton et classiques sur le réseau routier national français[END_REF]. It is based on the assumption that asphalt mixtures using conventional binder and PMB will perform likewise. The features of the M&R activities undertaken in the baseline scenario, as well as the application timing are displayed in Figure 1. In the AS1 the increase in the durability of the pavement equal to 20% means that the first and second M&R activities will take place in the years 2026 and 2036. Therefore, there will be no need of applying the 3 rd M&R activity, as scheduled in the baseline scenario. In the AS2 a pavement M&R strategy equal to that of the baseline scenario is adopted.

Table 1 presents the formulations of the PMB adopted in the alternative asphalt surface mixtures applied in the wearing courses, as determined by [START_REF] Soudani | Contribution à l'Etude du Comportement Rhéologique des Bitumes Modifiés par le Déchet Polymérique NBR -Influence sur les Propriétés Mécaniques et d'Adhérences des Enrobés[END_REF]. The design of all mixtures consists of 94.52% aggregates and 5.48% bituminous binder by weigh of asphalt mixture.

Data sources and data quality requirements

The inventory data required to perform a LCA study are classified into two categories: primary and secondary data. Primary data are those specific to the production processes for the product or service studied in the LCA. In turn, secondary data represent generic or average data for the product or service subject to analysis [START_REF]International reference life cycle data system (ILCD) handbook -general guide for life cycle assessment -detailed guidance[END_REF].

In this study the data sources were selected in order to be as much time, geographical and technological representative as possible. That means that the most recent and truthful data representing French processes and conditions were used as inputs and outputs when modelling the processes covered by the several sub-components integrating the system boundaries. Specifically, the primary data include mainly: (1) the composition of the mixtures; (2) the annual fuel consumption, production and life period of asphalt mixing plants; [START_REF] Van Dam | Towards sustainable pavement systems-a reference document[END_REF] transportation distances; and (3) construction vehicles fleet composition.

Regarding the secondary data, they are mainly related to the inventory analysis of (1) raw materials, (2) fuels, and (3) construction, transportation, and on-road vehicles operation, and were obtained from existing publicly available reports and the ecoinvent database version 3.2, but modified whenever possible and suitable to best approximate French conditions by using French energy inputs/mixes.

Life cycle inventory

The life cycle inventory (LCI) stage consists of the real data collection and modelling of the system. In addition to the data sources, it relies on the several models selected for modelling the processes analyzed by the several considered sub-systems that make up the whole system.

Materials extraction and mixtures production phase

This phase addresses the environmental burdens arising from the acquisition and processing of the materials applied during the initial construction and future M&R of a road pavement segment. This includes all materials manufacturing processes, from extraction of raw materials to their transformation into a pavement input material (material extraction sub-phase), ending with the mixture production at a mixing plant (materials production sub-phase). The latter subphase accounts for the environmental burdens associated with the operation of the (1) mixing plant (i.e., dryer, hot screen, mixers, etc.), (2) wheel loader during the movement of aggregates from the stockpiles to the feed bins and (3) electronic group of the asphalt plant setup.

Materials extraction sub-phase

The aggregates required to produce the asphalt mixtures were modelled as gravel and the LCI data associated with their production were obtained from the unit process "gravel, crushed | gravel production, crushed" of the ecoinvent database. Also taken from the aforementioned database was the LCI data corresponding to the production of asphalt binder ("pitch | petroleum refinery operation") and EVA polymer ("ethylene vinyl acetate copolymer | ethylene vinyl acetate copolymer production"). The EVA polymer is assumed to be posteriorly transported to the asphalt plant facility where is located a PMB production plant.

As for the NBR processing (sole shoe milling), a representative production rate of a Comas FD1-A milling shoe sole machine equal to 90 kg milled shoe sole/hour was considered when determining the electricity consumption requirements (200 kWh/ton milled shoe sole) [12].

Furthermore, from the perspective of the LCA methodology, the consideration of NBR as pavement material was made on the basis of the "cut-off" approach [START_REF] Schrijvers | Developing a systematic framework for consistent allocation in LCA[END_REF]. The production of the several types of PMB (Table 1) is assumed to be performed according to the average conditions allowed by an IKA DISPAX-REACTOR® DR 2000/50 PMB production plant [14]. They corresponds to a production rate of approximately 26 ton PMB/hour, to which it is associated an electricity consumption of 7.1 kWh/ton of PMB and an energy consumption for thermal oil heating of approximately 98 MJ/ton of PMB.

The LCI data related to the production and distribution of those energy resources representing the French conditions were posteriorly taken from the ecoinvent database.

Mixtures production sub-phase

This sub-phase addresses the LCI of the asphalt mixtures production process. In this case study it was assumed that all asphalt mixtures were produced through a conventional heavy fuel oil-fired batch mixing plant. The energy required for producing 1 ton of asphalt mixture and storing the binder in the asphalt plant were respectively 255 and 40 MJ. The fuel consumed by the wheel loader was 0.194 liters and the electricity consumed by the electric group of the asphalt plant was 5 MJ per ton of asphalt mixture produced. The values presented above correspond to the average French practices. Finally, the LCI data corresponding to the process "heat production, heavy fuel oil, at industrial furnace 1MW | heat, district or industrial, other than natural gas | cut-off, U" existing in the ecoinvent database was taken as reference when modelling the operation of the burner existing in the asphalt plant during the production of the mixtures.

Construction and M&R phase

In the construction and M&R phase, the environmental burdens are due to the combustion-related emissions from the construction machinery operation. The consumption-related emissions associated with the operation of each construction equipment were determined by combining the LCI data corresponding to the ecoinvent database process "machine operation, diesel, >= 74.57 kW, high load factor | machine operation, diesel, >= 74.57 kW, high load factor" with the typical productivity of each operation involved in pavement construction and maintenance activities. Furthermore, it was considered that the paving operations were equal for all the mixtures analyzed.

Transportation of materials phase

The environmental impacts resulting from the transportation of materials are due to the emissions released during the combustion process of the transportation vehicles when performing two-way trips from the site where the extraction of the raw materials takes place to the asphalt plant and from the asphalt plant to the work site. All materials and mixtures were assumed to be hauled by HDV, and a modified version of the ecoinvent database process "transport, freight, lorry >32 metric ton, EURO4 | transport, freight, lorry >32 metric ton, EURO4 | cut-off, U" was used to determine the environmental burdens associated with the transportation of materials movements. The original process was modified in order to disregard the inventory corresponding to the construction of the road infrastructure that is considered by the original process. The transportation distances considered for each material and mixtures used in this case study are representative of the French conditions and are shown in Table 2.

EOL phase

When a road pavement reaches the end of the PAP, it can be given two main destinations: (1) remain in place; or (2) be removed. In this case study, it is assumed that the pavement remains in place and undergoes the M&R activity illustrated in Figure 1. The environmental burdens assigned to this phase are due to the materials extraction and mixtures production and the combustion-related emissions from the use of the construction equipment and transportation HDV.

Life cycle impact assessment

The life cycle impact assessment (LCIA) stage of the standardized LCA methodology comprises several steps, namely, classification, characterization, normalization, group and weighting [START_REF]International Standard ISO 14040: environmental management -life cycle assessment: principles and framework[END_REF]. Among these steps, only classification and characterization were undertaken in this study. The calculation of the impact category indicator scores was performed at midpoint level by applying the ILCD 1.0.8 2016 method [START_REF]International reference life cycle data system (ILCD) handbook -recommendations for life cycle assessment in the European context -detailed guidance[END_REF]. Finally, the OpenLCA software version 1.5.0 was used for modelling the processes analyzed in this case study [START_REF] Greendelta | Open LCA[END_REF].

Results

Baseline scenario

Table 3 summarizes the LCIA results for the baseline and alternatives scenarios in which the conventional bituminous binder (no modified) is used in the production of the asphalt surface mixtures. Figure 2 shows the relative variation of the impacts scores for each asphalt surface mixture produced with PMB in relation to those associated with the conventional asphalt surface mixture. The relative values should be understood as follows: positive relative numbers mean that the use of PMB improves the LCIA results in relation to those associated with the conventional asphalt surface mixture. In turn, negative numbers represent a worsening of the environmental profile. The analysis of the results presented in this figure shows that overall the use of EVA polymer as a modifier agent leads to a deterioration of the life cycle environmental profile of the pavement structure in relation to that corresponding to the use of conventional (no modified) binder. The only exception to this general trend is observed for the impact categories ionizing radiation, ozone layer depletion and land use. Given those results, it is not surprising that the greatest negative relative variation of the impacts scores are observed when the binder is modified by means of the use of 5% of EVA polymer by weight of binder. In these circumstances, the greatest relative variation in absolute value (approximately 12%) is registered in the impact category freshwater eutrophication.

In turn, the use of NBR polymer as a modifier agent leads to an improvement of the life cycle environmental profile of the pavement structure in relation to that corresponding to the use of conventional (no modified) binder. That improvement is observed for all impact categories and can be as high as approximately 4% (ozone layer depletion) whereas the lowest and marginal improvement is observed in the impact category land use (approximately 0.1%). Notwithstanding the environmental benefits obtained with the use of the NBR polymer, when it is combined with the EVA polymer in a formulation in which the total amount of polymer is equal to 2%, it is not enough to offset the negative effects resulting from the use of the EVA polymer. However, when the total amount of polymer totals 5% and the percentages of NBR and EVA polymers are equal to 75% and 25% of that value, in 9 out of 15 impact categories the positive environmental effects resulting from the use of NBR polymer cancels out the negative environmental effects associated with the use of EVA polymer. The greatest relative improvement is once again observed in the impact category ozone layer depletion (approximately 4%), whereas the most tenuous is registered in the impact category respiratory effects.

Alternative scenarios

Figures 3 and4 show respectively for AS1 and AS2 the relative variation of the impacts scores for each alternative asphalt surface mixture in relation to those corresponding to the use of the conventional asphalt surface mixture in the baseline scenario.

The analysis of Figure 3 reveals that if the use of PMB in asphalt mixtures has the capacity to improve pavement performance according to the conditions considered in this case study, then considerable environmental benefits can be obtained regardless of the type of PMB used in the asphalt mixture applied in the wearing course of the pavement structure and subsequent M&R activities. The most prominent reduction in the impacts scores are obtained when the total percentage of polymer in the PMB amounts to 5%, and particularly when that percentage is fully made up of NBR polymer. The greatest life cycle impacts saving is equal to approximately 21% (impact category ozone layer depletion in the asphalt mixture adopting PMB 5%(100NBR%;0%EVA)) while the lowest life cycle impacts saving is equal to approximately 9% (impact category freshwater eutrophication in the asphalt mixture adopting PMB 5%(0NBR%;100%EVA)). When analyzing the extent to which the alternative asphalt plant fuel affects the variation of the scores of a given impact category across the alternatives asphalt surface mixtures, Figure 4 shows that overall shifting from HFO to natural gas offers considerable potential and uniform environmental benefits across the six alternative asphalt surface mixtures. The most expressive improvements in the environmental profiles are observed in the impact category respiratory effects (approximately 30%) followed by the impact category acidification (approximately 28%). The only exception to the global improvement in the environmental profiles resulting from changing the type of fuel consumed by the asphalt plant is observed for the impact category freshwater eutrophication, which is expected to see its scores increased by roughly 4%.

Summary and conclusions

In this paper, the results of a process-based LCA of the construction and M&R of the wearing course of a French road pavement section using PMB asphalt surface mixtures, were presented and compared with those in which a conventional asphalt surface mixture (no modified binder) is alternatively applied. Furthermore, two alternative modelling scenarios were considered. In the baseline and second alternative scenarios the conventional asphalt surface mixture (using no modified binder) and the six alternative asphalt surface mixtures (using PMB) were assumed to perform likewise throughout the pavement life cycle. In turn, in the first alternative scenario, the six alternative asphalt surface mixtures were assumed to extend the service life of the wearing course, comparatively to that provided by conventional asphalt surface mixture.

The life cycle of the road pavement sections was divided into four main phases: (1) materials extraction and mixtures production; (2) construction and M&R; (3) transportation of materials; and (4) EOL. The LCI of inputs and outputs associated with the processes considered by the several pavement life cycle phases was determined by combining primary data, representing the current French practices and conditions, with secondary data taken primarily from the ecoinvent database version 3.2. The ILCD 1.0.8 2016 impact assessment method was adopted to characterize the environmental performance of the road pavement section.

The LCIA results of the case study showed that in the baseline scenario the use of EVA polymer as a modifier agent leads to a deterioration of the life cycle environmental profile of the pavement structure in relation to that corresponding to the use of conventional (no modified) binder. This result contrast with that in which the NBR is the binder modifier agents, as it was found to improve the life cycle environmental performance of the road pavement section.

However, if the use of PMB asphalt mixtures succeed in improving pavement performance as proven in literature, thereby reducing the frequency of the application of M&R activities, then considerable environmental benefits can be obtained regardless of the type of PMB used in the asphalt mixture implemented in the wearing course of the pavement structure and subsequent M&R activities.

Fig. 1 .

 1 Fig. 1. Geometric characteristics of the flexible pavement structure and M&R strategy adopted in the baseline scenario. Key: AC-asphalt concrete.

Table 1 .

 1 Formulation of the several PMB considered in the case study.

					PMB ID
		1	2	3	4	5	6
	Total % of polymer (by weight of binder)	2	2	5	5	5	5
	% of NBR (by weight of polymer)	50	75	50	75	100	0
	% of EVA (by weight of polymer)	50	25	50	25	0	100

Table 2 .

 2 Transportation distances considered in the case study.

	Type of material	One-way trip distance (km)
	Aggregates	20
	Binder	100
	Asphalt mixtures 20
	EVA	100
	NBR	100

Table 3 .

 3 LCIA results referring to the case in which the bituminous binder used in the production of the asphalt surface mixtures is not modified.

	Impact category	Reference unit	Baseline scenario	Alternative scenario AS1 AS2
	CC	kg CO2-Eq	453306	371038	406746
	Ac	mol H+-Eq	4455	3647	3186
	FEco	CTUh.m3.yr	2465819	2018343	2375463
	FEu	kg P-Eq	63	52	66
	EQ-IR	mol N-Eq	8.3E-1	6.8E-1	7.7E-1
	MEu	kg N-Eq	520	425	440
	TEu	mol N-Eq	5768	4721	4884
	CE	CTUh	2.1E-2	1.7E-2	2.1E-2
	HH-IR	kg U235-Eq	136657	111858	128349
	NoCE	CTUh	9.2E-2	7.5E-2	8.7E-2
	OLD	kg CFC-11-Eq	3.2E-1	2.6E-1	3.1E-1
	POC	kg ethylene-Eq	2138	1750	1842
	RE	kg PM2.5-Eq	378	309	262
	LU	kg Soil Organic Carbon	5277339	4319656	4907831
	MFR	kg Sb-Eq	29	23	29

  Relative variation of the LCIA results obtained in the AS1 in relation to those corresponding to the use of conventional asphalt surface mixture (no PMB) in the baseline scenario. Key: Polymer%(NBR%;EVA%). Fig.4. Relative variation of the LCIA results obtained in the AS2 in relation to those corresponding to the use of conventional asphalt surface mixture (no PMB) in the baseline scenario. Key: Polymer%(NBR%;EVA%).

	2%(50%;50%)		2%(75%;25%)	5%(50%;50%)
	5%(75%;25%)		5%(100%;0%)	5%(0%;100%)
	LU	MFR	-5% 0% 5%	CC	Ac	FEco	0% 5%(50%;50%) 5%(0%;100%)	MFR	Ac 2%(50%;50%) 25% CC 5%(75%;25%)	2%(75%;25%) 5%(100%;0%)
	POC RE	-15% -10%			EQ-IR FEu	RE	LU		5% 10% 15% 20%	FEu FEco
	OLD					MEu	POC			0%	EQ-IR
	NoCE				TEu
		HH-IR	CE			OLD			MEu
								NoCE		TEu
	Fig. 2. Relative variation of the LCIA results in the baseline scenario for each			HH-IR	CE
	alternative asphalt surface mixture. Key: Polymer%(NBR%;EVA%).
							0% 10% 20% 30% 40% 2%(50%;50%) CC Ac 5%(75%;25%) Fig. 3. -10% POC RE LU MFR 0% 5%(50%;50%) 5%(0%;100%)	FEco FEu EQ-IR 2%(75%;25%) 5%(100%;0%)
							OLD			MEu
								NoCE		TEu
									HH-IR	CE


Yilin Zhang

Marie Poux-Berthe

Chris Wells

Karolina Koc-Michalska

Karl Rohe

François Hollande

Nicolas Sarkozy

good, all, fair, nothing, must, Jean-Luc Mélenchon

all, fair, Luc Mélenchon

DISCOVERING POLITICAL TOPICS IN FACEBOOK DISCUSSION THREADS WITH GRAPH CONTEXTUALIZATION

Keywords: UMP, dwarf, liar, aggravating, euros, quinquennium. Anti-Sarkozy UMP, liar, dwarf, budgetary, aggravating, euros, thief socialist, concord, captain, assistantship, reelected, flamby, strong, gentleman, censored, concord, socialist, assistantship, reelected, captain, strong, flamby, censored, Koran, Allah, angel, religion, pig, Islam, pork, mosque, arab. Islam, religion, and immigration Koran, angel, religion, Allah, pig, Islam, mosque, pork JLM, resistance, troll, FDG, forehead, bric, human, fought, dictatorship JLM, resistance, troll, FDG, revolutionnair, bric, forehead. Fig 8. Cluster topics when h = 0 14 dwarf, liar, aggravating, euros, quinquennium, modest residential, ancestry, chic, IRS, balance sheet, pent, loss making socialist, concord, assistantship, captain, reelected, flamby, strong, gentleman, ecologic, green, sincerity, madam, antisemitic, admired, supported, standing Koran, angel, pig, Allah, religion, Islam, pork, mosque, arab. Islam, religion, and immigration Koran, angel, Allah, religion, pig, Islam, pork, mosque, arab JLM, troll, FDG, forehead, human, bric, fought, revolutionary, fraternity JLM, resistance, troll, FDG, bric, revolutionary, forehead, fought, human, fraternity. Citizen-clusters Post-clusters François Hollande, Nicolas Sarkozy, fail, president, live, incompetent, May, charisma, arrogant, dwarf, goodbye, liar concord, flamby, sir, president, captain, bravo, charisma, assistantship, arrogant, goodbye, strong, debate, failed, incompetent residential, descent, child, clinical, chic, aristocrat, inhabit, land, employment employment, euro, child, residential, pedigree, chic, clinic, pent, inhabit, land. Islam, religion, and immigration Koran, Allah, religion, Islam, angel, pig, pork, Muslim, Arab, racist Koran, religion, Allah, angel, Islam, pig, Muslim, mosque, Arab, Lyon, racist, church JLM, comrade, resistance, FDG, front, liberal, revolutionary, human, capital, ecologic JLM, comrade, resistance, FDG, troll, revolutionary, front, human, struggle, liberal, fight clusters by pairGraphText Citizen-Clusters Cluster 1 fail, president, live, incompetent, May, charisma, arrogant, dwarf, goodbye, liar Cluster 2 residential, descent, child, clinical, chic, aristocrat, inhabit, land, employment Cluster 3 Koran, Allah, religion, Islam, angel, pig, pork, Muslim, Arab, racist Cluster JLM, comrade, resistance, FDG, front, Jean-Luc Mélenchon, liberal, revolutionary, human, capital, ecologic Post-Clusters Cluster 1 concord, flamby, sir, president, captain, bravo, charisma, assistantship, arrogant, goodbye, strong, debate, failed, incompetent Cluster 2 employment, euro, child, residential, pedigree, chic, clinic, pent, inhabit, land Cluster 3 Koran, religion, Allah, angel, Islam, pig, Muslim, mosque, Arab, Lyon, racist, church Cluster 4 JLM, comrade, resistance, FDG, troll, Jean-Luc Mélenchon, revolutionary, front, human, struggle, liberal, fight Citizen-Clusters Cluster 1 identity, trick, fascist, opposite, reducer, Allah, flamby, top, continuous, incarnate, commercial, mission Cluster 2 baptist, professor, suburb, king, happiness, aristocrat, sincerity, school, regime, residential, exist, erasable, place Cluster 3 dismissal, fraud, multiple, lump, aggravating, unfair, review, gift, parliamentary, budget, referendum Cluster Parisian, Russian, discriminant, defense, land, vineyard, flag, revel, pedigree, captain, conceivable, Post-Clusters Cluster 1 continued, resistance, great, passion, channel, bravo, debate, fight, difficult, goodbye, great, beat, stand, hope Cluster 2 troll, military, comrade, raid, concord, max, Philippe Poutou, tomorrow, soldier, killer, victim, hateful, bulletin, Jewish, fraud Cluster 3 Allah, Israel, altarpiece, foul, angel, dozen, list, cuckoo, municipal Cluster 4 lucid, Allah, African, boat, clandestine, sister, successful, realist, old, movie, angel, tear, promise, president, live, François Hollande, bravo, France courage, all, good, strong, debate Cluster 2 residential, land, pent, pedigree, clinical, inhabit, chic, functional, school, conceivable, childhood Cluster 3 Nicolas Sarkozy, fair, good, other, must, polish, can, say, nothing, François Bayrou, generation Cluster 4 fair, good, Jean-Luc residential, clinic, stock market, live, childhood, build, school, free, assembly, departure, functional, bourgeois, depend Cluster 3 Muslim, religion, Islam, speak, racist, insult, Arab, Koran, evil, fear, angel Cluster 4 European, financial, public, undertaken, billion, public, budget, advice, bank, balance sheet, service, euros, jobs Post-Topics Cluster 1, François Hollande, social, debate, may, president, victory, change, rich, augment, tax, poor, million Cluster 2 president, franc, sir, live, bravo, all, aim, pay, win, good, want, FDG Cluster 3 Islam, religion, racist, evil, immigrant, Arab, insult, Koran, know, from, Jewish, racism Cluster 4 more, good, France, Jean- can, politic, generation

We propose a graph contextualization method, pairGraphText, to study political engagement on Facebook during the 2012 French presidential election. It is a spectral algorithm that contextualizes graph data with text data for online discussion thread. In particular, we examine the Facebook posts of the eight leading candidates and the comments beneath these posts. We find evidence of both (i) candidate-centered structure, where citizens primarily comment on the wall of one candidate and (ii) issue-centered structure (i.e. on political topics), where citizens' attention and expression is primarily directed towards a specific set of issues (e.g. economics, immigration, etc). To identify issue-centered structure, we develop pairGraphText, to analyze a network with high-dimensional features on the interactions (i.e. text). This technique scales to hundreds of thousands of nodes and thousands of unique words. In the Facebook data, spectral clustering without the contextualizing text information finds a mixture of (i) candidate and (ii) issue clusters. The contextualized information with text data helps to separate these two structures. We conclude by showing that the novel methodology is consistent under a statistical model.

1. Introduction. Social networking sites (SNSs) such as Facebook and Twitter now make up a major part of Internet communications [START_REF] Ellison | Social network sites: Definition, history, and scholarship[END_REF], [START_REF] Kaplan | Users of the world, unite! the challenges and opportunities of social media[END_REF]), including political communication. By providing platforms for citizens to publicly communicate with each other and with politicians, SNSs may increase the accessibility of candidates and political dialog [START_REF] Wellman | Does the internet increase, decrease, or supplement social capital? social networks, participation, and community commitment[END_REF] and motivate political engagement within the public [START_REF] Williams | Social networks in political campaigns: Facebook and the congressional elections of 2006 and 2008[END_REF], [START_REF] Williams | Explaining facebook support in the 2008 congressional election cycle[END_REF], [START_REF] Hebshi | The rohe of online social networking in the 2008 democratic presidential primary campains[END_REF], [START_REF] Kushin | Getting political on social network sites: Exploring online political discourse on facebook[END_REF]). They also appear to facilitate the spread of false or offensive information, and a variety of forms of actors to reach micro-targeted publics with a high degree of efficiency [START_REF] Kreiss | Technology firms shape political communication: The work of microsoft, facebook, twitter, and google with campaigns during the 2016 us presidential cycle[END_REF]. Since the 2008 US election particularly [START_REF] Wattal | Web 2.0 and politics: the 2008 us presidential election and an e-politics research agenda[END_REF], SNSs have been playing a significant role in advertising and interactions during the presidential elections.

Drawing meaning from the massive text corpora of political discussion threads on SNSs has been a major project of scholars working in text mining [START_REF] Pang | Opinion mining and sentiment analysis[END_REF], Stieglitz and Dang-Xuan (2013), [START_REF] Grimmer | Text as data: The promise and pitfalls of automatic content analysis methods for political texts[END_REF]) and sentiment analysis in recent years. One popular text mining approach is the probabilistic topic models based on latent Dirichlet allocation (LDA) [START_REF] Blei | Latent dirichlet allocation[END_REF], [START_REF] Blei | Probabilistic topic models[END_REF], [START_REF] Chang | Relational topic models for document networks[END_REF]), which have been extensively used in social science [START_REF] Ramage | Comprehensive functional analysis of mycobacterium tuberculosis toxin-antitoxin systems: implications for pathogenesis, stress responses, and evolution[END_REF]. Sentiment analysis is another approach to analyze text. It focuses on understanding emotions in the text. [START_REF] Wang | A system for real-time twitter sentiment analysis of 2012 us presidential election cycle[END_REF] provides a system for real-time sentiment analysis on Twitter during the 2012 US election. For instance, using sentiment analysis and regression, Stieglitz and Dang-Xuan (2012) finds that political tweets on Twitter that contain stronger emotions receive more public interactions. There are also studies of how political sentiment on SNSs reflect the offline political landscape [START_REF] Tumasjan | Election forecasts with twitter: How 140 characters reflect the political landscape[END_REF], and how it can affect political elections [START_REF] Choy | A sentiment analysis of singapore presidential election 2011 using twitter data with census correction[END_REF].

Apart from the topic or sentiment information, patterns of political discussion on SNSs are also of great theoretical and empirical interests to scholars of communication and political science. Such platforms have long been heralded for their potential to foster a "public sphere" in which ordinary citizens can recognize one another and hear reasons both for and against their own points of view [START_REF] Papacharissi | The virtual sphere the internet as a public sphere[END_REF]). More recent analyses of online political discourse are less optimistic, identifying instead vitriol, "trolling", and larger patterns of partisan polarization. As a result, a great deal of research investigates the extent to which online actors are connected to political opponents [START_REF] Adamic | The political blogosphere and the 2004 us election: divided they blog[END_REF], [START_REF] Colleoni | Echo chamber or public sphere? predicting political orientation and measuring political homophily in twitter using big data[END_REF], [START_REF] Bakshy | Exposure to ideologically diverse news and opinion on facebook[END_REF])

Another approach to understand structure of political discussions is social network analysis, which aims to identify influential political actors and communities in the discussions [START_REF] Stieglitz | Political communication and influence through microblogging-an empirical analysis of sentiment in twitter messages and retweet behavior[END_REF]) and to study properties of the communities [START_REF] Robertson | Off the wall political discourse: Facebook use in the 2008 us presidential election[END_REF]) Gonzalez-Bailon et al. ( 2010)). One popular community detection approach is spectral clustering (Von [START_REF] Luxburg | A tutorial on spectral clustering[END_REF], which is fast, easy to implement, and consistent in block models for network [START_REF] Holland | Stochastic blockmodels: First steps[END_REF], [START_REF] Airoldi | Mixed membership stochastic blockmodels[END_REF], [START_REF] Qin | Regularized spectral clustering under the degree-corrected stochastic blockmodel[END_REF]).

In this paper, we combine text mining and community detection to investigate the multiple dimensions of citizens' interactions with political content coming from political actors. In our data, which come from the 2012 French election, citizens commented on presidential candidate's Facebook posts. This creates a communication network between two types of units: (i) citizens and (ii) candidate-posts, as the eight presidential campaigns each has posts on Facebook, and citizens comment on the posts. This paper studies the structure of the resulting discussion threads.

The activities of the citizens are characterized by (i) which of the candidate-posts they comment on and (ii) the text of their comments. We are interested in two broad types of patterns in these activities: (i) candidate-centered structure, where citizens primarily comment on the wall of one candidate; and (ii) issue-centered structure, in which citizens' attention and expression is directed towards a specific set of issues (e.g. economics, immigration, etc). To search for such patterns, we cluster the citizens based on their activities. In each cluster, we examine whether the activities of the citizens focus on particular candidates (i.e. candidate-centered)(Section 2.2) or whether the activities focus on certain political issues (i.e. issue-centered)(Section 4). This distinction reflects the possibility that the Facebook conversation might be organized more along lines of partisanship (candidate-centered), as opposed to matters of concern to "issue publics" (issue-centered) [START_REF] Kim | Issue publics in the new information environment: Selectivity, domain specificity, and extremity[END_REF]).

There has been significant progress on both topic modeling for text [START_REF] Blei | Probabilistic topic models[END_REF] and community detection for social networks [START_REF] Airoldi | Mixed membership stochastic blockmodels[END_REF]). Recently, there has been significant interest in clustering networks for which we have additional information on the citizens in networks [START_REF] Chang | Hierarchical relational models for document networks[END_REF]; [START_REF] Binkiewicz | Covariate-assisted spectral clustering[END_REF]). In this paper, we extend these ideas to the setting of discussion threads. Our network is two-way or bi-partite, in which the two types of units, citizens and candidate-posts, are linked by commenting in a discussion thread. Below, we refer to the links showing which citizens commented on which candidate-posts as the network or the graph. We refer to both the text in candidate-posts and the text in citizen-comments as the text. The duality between citizens and candidate-posts also appears in the text; candidates say things differently from citizens.

A key difficulty in analyzing this process, and the key methodological innovation of this paper, is to combine these disparate sources of data, the graph information and the two types of text information (citizen-words and thread-words), in a meaningful way. We develop a graph contextualization technique, pairGraphText, to leverage high dimensional node covariates into spectral clustering. We extend and specialize the techniques of [START_REF] Binkiewicz | Covariate-assisted spectral clustering[END_REF] to deal with both (i) the asymmetrical nature of the network between citizens and candidate-posts, and (ii) the high dimensional and sparse nature of the text. With noticeable themes, four sub-populations and four sub-groups of the candidate-posts are uncovered by our method. We interpret the clusters by a word-content strategy: For each cluster, we (i) identify keywords, and then (ii) read through central conversations containing the keywords. Our graph contextualization method, pairGraphText, is adaptable to symmetric or directed graphs, unipartite or bipartite, assortative or dis-assortative, weight or unweighted. It scales to hundreds of thousands of nodes and thousands of covariates (e.g. words). pairGraphText uses a sparsity penalty to select the key covariates that align with the graph. After combining the covariates with the graph, we use spectral clustering to compute a partition of the nodes. Finally, we provide diagnostics to identify key covariates to interpret the different clusters. Theorem 5.2 shows that our method is consistent under the Node-Contextualized Stochastic co-Blockmodel. Section 4 uses pairGraphText to identify the issue centered structure in the Facebook discussion threads. In Section 6, we compare pairGraphText to a state-of-the-art topic modeling method, relational topic model (RTM) [START_REF] Chang | Relational topic models for document networks[END_REF], by both the Facebook discussion threads and simulations. We show that RTM focuses more on the text data, while pairGraphText focuses more on the graph data.

This paper is organized as follows. In Section 2, we briefly describe the 2012 French presidential election, the discussion threads on Facebook, and the result of regularized spectral clustering without any contextualizing information. In Section 3, we introduce the graph contextualization technique, pairGraphText, which leverages node covariates in spectral clustering. In Section 4, we identify the issue-centered structure of the discussion threads using pairGraphText. The statistical consistency of our method is provided under the Node Contextualized Stochastic co-Blockmodel in Section 5. In Section 6, we discuss different choices for weights of words, and we compare pairGraphText with a state-of-the-art topic modeling method. Section 7 concludes with a discussion of our method.

2. Background and key summaries of the data. France's presidential elections proceed in two stages. On April 22 2012, the first round of voting narrowed the field of candidates from ten to two; the second round, between François Hollande and Nicolas Sarkozy, took place on May 6. In these analyses, we focus on the eight candidates who received at least 1% of the votes in the 1st round of the election. These eight candidates-François Hollande, Nicolas Sarkozy, Marine Le Pen, Jean-Luc Mélenchon, François Bayrou, Eva Joly, Nicolas Dupont-Aignan, and Philippe Poutou-made a total of 3239 posts on Facebook. In response, 92,226 Facebook users, which we call citizens, made 594,685 comments on the candidate-posts. 1There are two main structures that we aim to detect and study in the conversation: (i) candidatecentered structure, where citizens primarily comment on the wall of one candidate; and (ii) issuecentered structure, in which citizens' attention and expression is directed towards a specific set of issues (e.g. economics, immigration, etc).

The communication network.

To study the structure of the conversations, we construct a weighted bi-partite network between citizens and candidate-posts (see Figure 1) from the discussion threads.

92,226 Citizens

3239 Candidate-Posts

Fig 1.
The Communication Network is a bi-partite graph between citizens and candidate-posts. Each edge weight corresponds to the number of times that a citizen comments on a candidate-post.

A citizen is linked to a candidate-post if and only if the citizen comments on the candidate-post. The weight of this link is the number of times the citizen comments on the candidate-post. To represent this network, we construct the weighted adjacency matrix A ∈ R 92,226×3239 with (2.1)

A ij = # of times of citizen i comments on candidate-post j.

Denote the degree of a citizen i, d i = j A ij , as the number of comments by citizen i. Denote the degree of a candidate-post j, d j = i A ij , as the number of comments underneath the candidatepost. Figure 2(a) shows the proportion of citizens who have at least d comments, as a function of d. Figure 2(b) gives the same result for the post-degrees. q q q q q q q q q q q q q q q q q q q 10 -5 10 -4 10 -3 10 -2 10 -1 10 0 10 0 10 1 10 2 10 3

Citizen-Degrees

Upper Tail

Upper Tail of Citizen-Degrees (a) q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 10 -3 10 -2.5

2.2.

Citizens' attention-ratio towards candidates. Let ζ ij be the number of times that citizen i comments under candidate j's wall. For each citizen i, we denote their attention-ratio as

AttentionRatio(i) = max ζ i d i .
When the attention-ratio is one, it indicates the citizen only comment on one candidate-wall, while smaller attention-ratio indicates the citizen comments across different candidate-walls. We say that citizen i focuses on candidate j if ζ ij ≥ ζ i for any candidate . The citizens that have tied favorites are randomly assigned to one of their favorite candidates. Then, the citizens are naturally partitioned into eight clusters based on the candidates they focus on. Figure 3 Figure 4 gives a balloon plot of Ψ C for the partition created by where citizens focus their attention. It also shows a clear candidate-centered structure: Each candidate has a corresponding citizencluster that mainly comment on their posts. Combined with the size of each citizen-cluster, it shows leading candidates attract larger clusters of citizens. See supplementary material for more evidence for candidate-centered structure. q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q Poutou Dupont Joly However, such strong candidate-centered structure, where citizens primarily comment on the wall of one candidate, does not lead to the conclusion that citizens devote their attention to candidates rather than issues. It might be an "illusion" from the "magnifying" effect of Facebook [START_REF] Webster | The marketplace of attention: How audiences take shape in a digital age[END_REF]). One possibility is many citizens may only follow one candidate on Facebook, so they can only see posts from one candidate. Even if they are interested in topics that are discussed by many candidates, they are likely to comment only on the candidate's posts that they follow. In this case, even a slight more interest in one candidate can be magnified by Facebook to a strong candidatecentered structure. To understand whether the citizens' attention is only directed by candidates, we dig more deeply into the discussion threads in the following sections.

Bayrou

Importantly, the partition of citizens in Figure 4, which is created by where citizens focus their attention, uses the additional information of which of the eight candidates writes each post. In other words, this partition of the rows of A ∈ R 92,226×3239 uses a partition of the 3239 columns of A which is defined by which candidate writes the post. The next sections will define two additional partitions of the citizens. Neither of these partitions will use the information of which candidate writes the post. The summary Ψ C will be computed with these new partitions to help interpret whether they are discovering candidate-centered structure. One possible reason for the discrepancy between the attention-based partition and the partition from di-sim is that there may be some additional structure and di-sim is finding a mixture of the candidate-centered structure with that additional structure. pairGraphText, which we will introduce in the following sections, confirms that there is also an issue-centered structure in the network by incorporating text information.

Graph Contextualization with pairGraphText.

As shown in Section 2, there are at least two good clusterings of the nodes (by attention-ratio or by di-sim). Given the potentially large number of plausible clusterings of the nodes, the overarching aim of graph contextualization is to find a co-clustering of A (i.e. clustering both its rows and columns) such that these clusters align with a partition in the contextualizing information.

To quantify and utilize the contextualizing information, Section 3.1 describes how we preprocess the text in the discussion threads. Section 3.2 defines the document-term matrices to represent the text used by citizens and candidate-posts. Section 3.3 introduces the pairGraphText algorithm.

3.1.

Preprocessing the text. To preprocess the text, we represent the text in document-terms, remove numbers, symbols (e.g. %, @, etc), and stop words (e.g. le, la, en, au, etc.) and transfer words into their roots by stemming. For example, maintenaient, maintenait, maintenant, maintenir are transferred into their root mainten.

Document-term matrices (node covariate matrices).

From the cleaned text, we retain two different sets of words: "citizen-words" which are contained by at least 0.1% of the comments, and "thread-words" which are contained in at least 0.1% of the contents in threads (i.e. posts and comments). In this data, over 99% of citizen-words and thread-words are overlapped, such as franc, vot, plus, etc. There are also thread-words that are not in citizen-words, such as confrontaient, relancait, etc.

To contextualize the citizens with the words that they write, define X ∈ R N C ×M C , where N C is the number of citizens and M C = 2020 is the number of citizen-words. For citizen i and citizen-word j, X ij = # of comments from citizen i that contain citizen-word j.

Representing the candidate-posts is not as simple. Candidate-posts provide platforms for conversations, but usually it is the comments underneath it that generate conversations. This phenomenon is colloquially referred to as "thread highjacking," where the discussion thread (beneath a candidatepost) is used to discuss something other than what is discussed in the candidate-post. In particular, many of the candidate-posts direct their followers to interviews that happen in traditional media. Thus, to properly contextualize the thread, one must include the text that citizens are responding to, which is not necessarily the candidate-post. To represent the text that citizens are responding to when they post a comment in a thread, we use matrix Y ∈ R N P ×M P , where N P = 3239 is the number of candidate-posts and M P = 2021 is the number of thread-words. For candidate-post i and thread-word j, Y ij =1{candidate-post i contains thread-word j}+ # of comments underneath candidate-post i that contain thread-word j.

We refer to X and Y document-term matrices and consider them as node covariate matrices that contain the text information about both types of nodes (citizens and candidate-posts). The rows index the nodes (citizens or candidate-posts) and columns index the dictionaries (citizen-words or thread-words). Our setting allows citizen-covariates and post-covariates to differ in both type and number. In general, there could be various types of covariates. Note that categorical covariates should be re-expressed with dummy variables. In practice, node covariate matrices X and Y should be centered and scaled by column before analysis.

pairGraphText.

pairGraphText is a refinement of Covariate Assisted Spectral Clustering (CASC) [START_REF] Binkiewicz | Covariate-assisted spectral clustering[END_REF]. In CASC, the graph is uni-partite. Denote X ∈ R N ×M as the node covariate matrix and L ∈ R N ×N as the regularized graph Laplacian To generalize CASC, pairGraphText refines the matrix C in several ways. This refinement will first be expressed in terms of a uni-partite graph where X = Y . Replace C = XX T with C W = XW X T for some matrix W . Note that when W is identity matrix, C W = C. By imposing matrix W , pairGraphText addresses the following limitations of CASC.

• For any matrix H, denote its ith row as H i• and its jth column as

H •j . Note that C W = ij W ij X •i X T
•j . So, when W ij is nonzero for i = j, it creates an "interaction" between X •i and X •j , i.e. ith and jth covariates. Such interactions are not included in C = XX T = j X •j X T

•j . • In C, there is not a natural way of excluding covariates, i.e. discarding columns of X. However, in many settings, several covariates could be unaligned with the graph and they should be excluded from the similarity matrix. C W can select covariates by setting some elements (or rows/columns) of W to zero. • C presumes that two nodes are more likely to be connected when they have similar covariates.

But in some situations, this is not true. For example, in a dating network, relationships are more prevalent among men and women than two people of the same gender. In C W , if W ii is negative, then two nodes are closer in the similarity matrix C W if they have different values for the ith covariate. • The symmetric matrix C only allows for symmetric contributions of covariates, which may not be the case for directed graphs. This can be addressed by allowing W to be asymmetric. • Finally, CASC was not designed for bi-partite networks. In a bipartite graph, the rows of A might have different contextualizing measurements than the columns of A. In the Facebook data, these measurements correspond to the matrices X and Y . Because they have a different number of measurements, the multiplication XY T is not defined for the Facebook data. However, the multiplication XW Y T is well defined for a rectangular W . This removes the need for a one-to-one correspondence between the columns of X and Y ; they could contain entirely different types of measurements.

We propose estimating a matrix W to address the issues above. Define the call-response matrix

(3.2) W = X T LY,
which measures the correlation between thread-words and citizen-words along the graph. For example, if discussion threads containing the word franc have comments from citizens that are likely to say vot, then citizen-word vot is highly correlated with a thread-word franc along the graph.

To illustrate W = X T LY , examine a single element x T Ly, where x ∈ R 92,226 is a column of X corresponding to word vot and y ∈ R 3239 is a column of Y corresponding to word franc. So, x i is the number of times that citizen i uses vot and y j is the number of times that franc appears in the thread for candidate-post j. If x is centered and independent of L and y, then x is an uninformative covariate, and E[x T Ly] = E(E(x T |L, y)Ly) = 0. Conversely, if for centered x and y,

x T Ly = i,j:A ij =1 x i y j [D C ] ii [D P ] jj
is large (positive or negative), it suggests that linked nodes in L have (positively or negatively) correlated values of x and y. Figure 6 gives a small part of the call-response matrix.

There are thousands of words in the discussion threads. To select the highly correlated words along the graph, we define a hard-threshold function on W ,

(3.3) [T ω (W )] sr = W sr , if W sr > ω 0, o.w.
In practice, we can set the threshold ω as the 1 -α quantile of |W ij |'s.

C T = ij [T ω (W )] ij X •i Y T •j
shows closeness of citizens and candidate-posts based on their usage of words in the network. [C T ] ij is large when citizen i and candidate-post j use many highly correlated pairs of words. The threshold function T ω (• ) helps select pairs of words, and imposes sparsity when W is high-dimensional. Therefore, pairGraphText applies di-sim to the similarity matrix:

(3.5) S = L + hC T .
This similarity matrix combines both the graph information, represented by L, and the text information, represented by C T = XT ω (W )Y T , with a tuning parameter h to balance between these two parts.

4. Issue-centered structure. We identify topics that attract public's attention in the Facebook discussion threads using pairGraphText. We scale the document-term matrices by both rows

Algorithm 1 pairGraphText

Input: adjacency matrix A ∈ R N P ×N C , node covariate matrices X ∈ R N P ×M P and Y ∈ R N C ×M C , number of citizen-clusters KC , number of post-clusters KP , weight h, and the significance level α.

1. Compute the regularized graph Laplacian L from A as in (3.1). Center X and Y by column. (In practice, scaling X and Y by rows and columns or using weighted X and Y might also be beneficial. See more details in Section 6.2. )

2. Compute W = X T LY . Set ω to be the 1 -α quantile of |Wij|'s.

3. Compute the similarity matrix for pairGraphText as

S = L + hXTω(W )Y T .
4. Compute the top K left and right singular vectors UC ∈ R N C ×K , UP ∈ R N P ×K corresponding to the K largest singular values of S, where K = min{KC , KP }.

Form matrices U *

C ∈ R N C ×K and U * P ∈ R N P ×K such that for any i ∈ {1, . . . , NC (NP )},

(3.6) [U * C ]i• = [UC ]i• [UC ]i• 2 and [U * P ]i• = [UP ]i• [UP ]i• 2 .
6. Cluster the rows of U * C into KC clusters with k-means. If the ith row of U * C falls in the kth cluster, assign citizen i to citizen-cluster k.

7. Cluster the candidate-posts by performing step 6 on the matrix U * P with KP clusters.

and columns. 4 From the scree plot of the singular values of S (see Figure 3 in supplementary material), we decide to study the top K = 4 clusters due to the large gap after the fourth singular value.

To study how the text in discussion threads affects the partition of citizens and candidate-posts, we show the clustering results in three cases: (i) when we use no text, i.e. the tuning parameter h in Equation (3.5) is h = 0, 5 (ii) when we incorporate text, i.e. h = 0.035, 6 and (iii) when we only use the text assisted part (defined in (3.4)), i.e. h = ∞. Section 4.1 shows that with more text incorporated (i.e. with larger h), the clusters become less candidate-centered. Section 4.2 introduces a word-content strategy to extract topics of clusters. Section 4.3 describes the cluster topics and supports Section 4.1 by showing that clusters with larger h are more heavily focused on the contextualizing information. Ψ P shows how post-clusters distribute on candidate-walls. This is similar to Ψ C defined in (2.2), which shows how citizen-clusters interact with candidate-walls. Figure 7 displays Ψ P and Ψ C in 4 We replace Xij and Yij by Xij/ i Xij j Xij and Yij/ i Yij j Yij. 5 When h = 0, pairGraphText is equivalent to di-sim. 6 In case (ii), h can be any real positive value. We choose h = 0.035 since it shows clusters with major differences from both cases when h = 0 and when h = ∞. Recall the similarity matrix S = L + hCT (see (3.5)). For identification of h = 0.035, we scale the text-assisted part CT to have the same second singular value with L. Then, h means how much we weigh the text-assisted part in pairGraphText. h = 0.035 means that we weigh the text-assisted part 0.035 times of the graph information. balloon plots in the three cases. When we use no text, i.e. h = 0, there appears some candidatecentered structure in both citizen-clusters and post-clusters. As we incorporate text, in the case when h = 0.035, each post-cluster spreads across multiple candidates. With even more text incorporated, in the case h = ∞, neither of the post-clusters nor citizen-clusters are candidate-centered. In the following subsections, we identify the cluster topics using key words, comments and posts. q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q h = 0 h = 0.035 h = ∞

Average Number of Posts

(a) q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q h = 0 h = 0.035 h = ∞

4.2.

A word-content strategy to identify cluster topics. To identify the cluster topics, we first identify keywords for each cluster, which we will define in Section 4.2.1. These keywords give the first impression of the cluster topics.

However, it is insufficient to examine the words in isolation, because the same word is often used differently by different subsets of the population. For example, religion is often used by citizens talking about the religion of peace and it is also often used by atheists criticizing its appearance in the public sphere. Thus, to identify the cluster topics, besides identifying keywords, we also need to read through the conversations that contain these keywords. We focus on the central conversations in each cluster, which we will define in Section 4.2.2.

We call this strategy word-content strategy, where for each cluster, we (i) identify the keywords and (ii) read through the central conversations that contain the keywords in the cluster. 4.2.1. Identify the keywords. We identify the keywords in each cluster by setting "scores". For any k ∈ {1, . . . , 4} and j ∈ {1, . . . , M C }, define the score of citizen-word j in citizen-cluster k as

Φ kj = i∈k X ij i∈k Xij , where Xij = j X ij i X ij X ij ,
and i ∈ k denotes the citizen i belongs to cluster k. We similarly define the scores of thread-words in post-clusters based on the document-term matrix of candidate-posts Y . These scores are also discussed in [START_REF] Witten | Classification and clustering of sequencing data using a poisson model[END_REF], where they are derived by maximum likelihood on a Poisson model. We define the keywords in a cluster to be the words with the largest scores in the cluster. We show keywords of each cluster in Section 4.3.

Identifying central conversations.

We identify the central conversations by diagnostics from k-means clustering. Recall the pairGraphText algorithm partitions citizens by applying kmeans on the N C rows of matrix U * C ∈ R N C ×4 (defined in (3.6)) which correspond to the N C citizens. For any citizen i, we denote their cluster-centrality as

ρ i = [U * C ] T i• [µ * C ] i ,
where [µ * C ] i is the cluster centroid of citizen i from k-means on rows of U * C . There are four different cluster centroids. For each cluster, the central citizens are the citizens in the cluster with the largest cluster-centrality, i.e. those that align best with the cluster centroid. We similarly define the central posts for post-clusters. For a citizen-cluster, the central conversations are the comments from the central citizens; for a post-cluster, the central conversations are the discussion threads (including posts and comments) initiated by the central posts.

We read through the central conversations that contain the keywords in each cluster. This wordcontent strategy helps us identify topics that attract citizens' attention. We will show these topics in Section 4.3. 4.3. Topics of clusters. We extract topics of the clusters by the word-content strategy in three cases, h = 0, h = 0.035, and h = ∞. Figure 8, 9 and 10 show the cluster topics with the keywords and a brief description of the central conversations in each cluster. In these figures, the links indicate major interactions7 between citizen-clusters and post-clusters, with the link widths proportional to elements of matrix Ψ ∈ R 4×4 , where for any a, b ∈ {1, . . . , 4}, Ψ ab = # of comments from citizens in citizen-cluster a under candidate-posts from post-cluster b (# of citizens in citizen-cluster a) × (# of candidate-posts in post-cluster b) .

This is similar to matrices Ψ C defined in (2.2) and Ψ P defined in (4.1), which show how clusters (for citizens or candidate-posts) distribute on the eight candidate-walls. Ψ shows how the citizen-clusters interact with the post-clusters. When h = 0 (see Figure 8), clusters focus on candidates or the radical discussions. As we incorporate the text, in the case when h = 0.035 (see Figure 9), the citizen-clusters are similar to those when h = 0, but there appears a post-cluster about ecology. As we incorporate more text, in the case when h = ∞ (see Figure 10), we identify more topics, such as economic and crises. There also appear a cluster for both citizens and candidate-posts with many copy-paste comments.

More data analysis results are in a Shiny App available at https://yilinzhang.shinyapps.io/ FrenchElection.

Incorporating the text makes the central conversations more vivid representations of the clusters, allowing for a more precise interpretation of the topic. During the 2012 French election, the citizens devoted their attention and expression in (i) the debates and fights among different candidates, (ii) radical discussions on Islam, religion, and immigration, and (iii) other topics including ecology, economy, and crises. Definition 5.1. Let Z C ∈ {0, 1} N C ×K C and Z P ∈ {0, 1} N P ×K P , such that there is only one 1 in each row and at least one 1 in each column.

Let B ∈ [0, 1] K C ×K P be of rank K = min{K C , K P }. Let E C ∈ R K C ×M C
and E P ∈ R K P ×M P . Under the NC-ScBM, the adjacency matrix A ∈ {0, 1} N C ×N P contains independent Bernoulli random variables with (1) A = E[A] = Z C BZ P , and the node covariate matrices X ∈ R N C ×M C and Y ∈ R N P ×M P contain independent sub-gaussian elements with

(2) X = E[X] = Z C E C and Y = E[Y ] = Z P E P .
Recall the similarity matrix for pairGraphText defined in Equation (3.5), S = L+hXT ω (W )Y T .

We define the population similarity matrix as (5.1)

S = L + hX WY T , where L = D -1/2 C AD -1/2 P and W = X T LY, where diagonal matrices [D C ] ii = j A ij + τ c and [D P ] jj = i A ij + τ p . Let U C and U C ∈ R N C ×K
(U P and U P ∈ R N P ×K ) contain the top K left(right) singular vectors of S and S.

The basic outline of the proof for statistical consistency is: Under some conditions, 1. the element-wise difference between T ω (W ) and W is bounded by ω in probability; 2. the similarity matrix S converges to S in probability; 3. the singular vectors U C and U P converge to U C and U P within some rotations in probability; 4. the mis-clustering rates for citizens and candidate-posts goes to zero in probability.

The definition of mis-clustered is the same as in Rohe et al. ( 2016) and is given in Section 3.2 in supplementary material. The complete proof is given in Section 3.3 in supplementary material. Denote • as the spectral norm and • F as the Frobenius norm. For any matrix H, we define

sym(H) = 0 H H T 0 and H 2 = max( max i H i• 2 , max j H •j 2 ). Denote • φ 2 as the sub-
gaussian norm, such that for any random variable ξ, there is ξ φ 2 = sup t≥1 t -1/2 (E|ξ| t ) 1/t . To simplify notation, we denote N as the number of nodes and M as the number of covariates, though N C and N P , M C and M P can be different.

Theorem 5.2. Suppose A, X and Y , are the adjacency matrix and the node covariate matrices sampled from the NC-ScBM. Let λ 1 ≥ λ 2 ≥ • • • λ K > 0 be the K non-zero singular values of S. Let M C and M P be the mis-clustered citizens and the mis-clustered candidate-posts. Denote q c and q p as the largest sizes of citizen-clusters and post-clusters. Define δ = min(min

i [D C ] ii , min j [D P ] jj ) and γ = max( X 2 , Y 2 , X 2 , Y 2 ). Define ξ = max(σ 2 L F √ ln M , σ 2 L ln M, γ 2 δ √ ln M
), where L is the regularized graph Laplacian defined in Equation (3.1) and σ = max

(max ij X ij - X ij φ 2 , max ij Y ij -Y ij φ 2 ).
For any ∈ (0, 1), assume

(1) δ > 3 ln(2N ) + 3 ln(8/ ), (2) ξ = o(ω), and

(3) h ≤ min( a γ 2 sym(W) , a γ 2 ω
), where a = 3 ln(16N/ ) δ .

Then, with probability at least 1 -, for large enough N , the mis-clustering rates

|M C | N ≤ c 0 q c K ln(16N/ ) N λ 2 K δ and |M P | N ≤ c 0 q p K ln(16N/ ) N λ 2 K δ ,
for some constant c 0 .

Remark. Assumption (1) indicates the sparsity of the graph. Assumption (2) and (3) are conditions on parameters ω and h for consistency. Note the largest sizes of clusters q c and q p are O(N ). Suppose λ K is lower bounded by some constant c 1 > 0, which indicates the "signal" of each of the K blocks is strong enough to be detected. Then, when δ grows faster than ln N , we have mis-clustering rates goes to zero as N → ∞.

6. Comparison analysis. Section 6.1 shows the importance of the call-response matrix W . Section 6.2 discusses different scaling and weighting choices for document-term matrices. In Section 6.3, we compare pairGraphText with state-of-the-art topic modeling approach, relational topic model (RTM) [START_REF] Chang | Relational topic models for document networks[END_REF], on both the Facebook discussion threads (Section 6.3.1) and on the simulated data (Section 6.3.2). 6.1. Importance of the call-response matrix W . Recall the call-response matrix W (defined in (3.2)), which shows the correlation between citizen-words and thread-words on the communication network. It induces weights on different pairs of citizen-words and thread-words; word-pairs with higher correlation on the network are weighted more. In this section, we show the importance of the weights induced by the matrix W . We compare pairGraphText with the all-one pairGraphText, which replaces matrix W by the "all-one" matrix, J ∈ R M C ×M P , where J ij = 1, for all i, j. For comparison, we set the tunning parameter in (3.5) as h = ∞. Table 1 and 2 show the keywords of each cluster by pairGraphText and by all-one pairGraphText.

Without weights on the word-paris, the all-one pairGraphText fails to extract some topics in the citizen-clusters, such as Islam and the debates among top candidates, which are clear in the citizen-clusters by pairGraphText. Moreover, some words appear in multiple clusters by the all-one pairGraphText. For example, the word Allah appears in both post-cluster 3 and 4 in Table 2. This makes it harder to distinguish different topics between different clusters. 6.2. Different choices for document-term matrices. Recall the document-term matrices X and Y (defined in Section 3.2). These matrices don't consider lengths of comments and posts or popularities of words. We can address this issue by either (1) scaling the document-term matrices by rows and columns as in Section 4, i.e. replacing X ij and Y ij by

X ij / i X ij j X ij and Y ij / i Y ij j Y ij ,
or (2) using the weighted document-term matrices. One standard weighting method is TF-IDF (term frequency-inverse document frequency), which is commonly used in information retrieval and text mining [START_REF] Salton | A vector space model for automatic indexing[END_REF]; [START_REF] Joachims | A probabilistic analysis of the rocchio algorithm with tfidf for text categorization[END_REF]; [START_REF] Sivic | Video google: A text retrieval approach to object matching in videos[END_REF]; [START_REF] Ramos | Using tf-idf to determine word relevance in document queries[END_REF]). TF-IDF weights words based on both the document length and the word popularity. For each word i and document j, the TF-IDF is # of occurences of word i in document j # of words in document j × log 2 # of documents # of documents that contain word i .

In our data, documents are the posts and comments in the discussion threads. For the weighted document-term matrix of citizens Y , we first calculate the TF-IDF matrix of comments, and then add up those comments from the same citizen. The weighted document-term matrix X is the TF-IDF matrix of posts. We compare plain (unscaled and unweighted), scaled, and TF-IDF weighted document-term matrices on the Facebook discussion threads in Section 3 in the supplementary material.

6.3.

Comparison with relational topic model. This section compares pairGraphText and Relational Topic Model(RTM) on both Facebook discussion threads (Section 6.3.1) and simulated data (Section 6.3.2).

6.3.1.

Comparison with Relational Topic Model on the Facebook Discussion Threads. Relational topic model (RTM) [START_REF] Chang | Relational topic models for document networks[END_REF]) is a popular approach to extract topics from documents with a network structure (e.g. citation network). RTM is designed for uni-partite networks, where there is only one type of nodes. To apply RTM on the bi-partite network with both candidate-posts and citizens, we consider two approaches, (1) symmetrized network (Table 3) and (2) co-occurrence network (Table 4).

We define the symmetrized network as a network with posts and citizens, disregarding the different types of nodes. Recall the adjacency matrix A ∈ R 92,226×3239 (defined in 2.1), the adjacency matrix for the symmetrized network is sym(A) = 0 A A T 0 . Table 3 shows the keywords of the four topics by RTM on the symmetrized network. Words such as Nicolas Sarkozy appears in both Cluster 1 and 3, making it hard to distinguish between different topics. The racial topic (Islam, religion, and immigration), which is clear by pairGraphText, is not that clear in Table 3. We define the co-occurrence network of posts as a network of posts, where the link width is large when the two posts share many citizens who comment frequently on both posts. Similarly, we define the co-occurrence network of citizens as a network of citizens, where the link width is large when the two citizens comment a lot on many same posts. We define the adjacency matrix of the co-occurrence network for posts as A T A and the adjacency matrix of the co-occurrence network for citizens as AA T .

Table 4 shows the keywords of the four topics by RTM on the co-occurrence networks. Similarly to pairGraphText, RTM also extracts topics like Islam and religion, debates among top candidates, and economic issues.

6.3.2.

Comparison with relational topic model on simulated data. In this section, we use simulation examples to compare pairGraphText and RTM based upon both statistical accuracy and computational running time.

We simulate documents with links and text, then use pairGraphText and RTM to cluster these documents. There are two sources of data, (1) links between documents, i.e. graph, and (2) text in the documents, i.e. text. We compare pairGraphText and RTM in three cases: (1) when both the graph and text contain block information (both signals), (2) when only the graph contains block information (graph signals), and (3) when only text contains block information (text signals). For each of the three cases, we simulate varying levels of signal strength. (See more details on how we define signals in the next paragraph). For each signal level, we simulate 100 random data sets. Each data set consists of 1000 documents and 1000 words in total, with around 200 words and 20 links per document. In this step, we simulate the documents with links and words under a block model with two blocks, each with around 500 documents and around 500 words. (See more details for the block model in the next paragraph). On each data set, we run pairGraphText and RTM to partition the 1000 documents into two clusters. For RTM, we define its estimated cluster label for each document i as max k # of words in document i belongs to block k.

We simulate all the adjacency matrices (graphs) and the document-term matrices (text) under a Degree Corrected Stochastic Blockmodel [START_REF] Karrer | Stochastic blockmodels and community structure in networks[END_REF]. Denote z(i) as the block label of any document i and z text (w) as the block label of any word w. Under this model, two documents i and j are linked with each other with probability θ i θ j B z(i)z(j) , and document i contains word w with probability θ i θ text w B text z(i)ztext(w) , where θ i , θ j , θ text w are degree parameters. The element B uv shows the expected number of links between blocks u, v, and the element B text uv shows the expected number of appearances for words in block v in documents in block u. We define B ∝ 0.1 0. For pairGraphText, we set weight h so that the first singular values of the graph Laplacian L and the text assisted part hC T are equal. We choose the threshold ω to be the 95% quantile of non-zero |W ij |'s. We set the number of random starts the k-means steps (Step 6 and 7 in Algorithm 1) as 10 4 . For RTM, we use the function rtm.collapsed.gibbs.sampler in the R package lda. We set the scalar value of the Dirichlet hyperparameter for topic proportions α = 0.001, the scalar value of the Dirichlet hyperparamater for topic multinomials η = 0.1, the numeric of regression coefficients expressing the relationship between each topic and the probability of link β = (0.5, 0.5), and the number of sweeps of Gibbs sampling over the entire corpus to make as num.iterations = 1e4. We set α to be small since we aim to cluster each document to one topic instead of multiple topics. We set the num.iterations large enough so that the likelihood from each document converges.

Figure 11 RTM enables us to predict keywords and citations for new documents [START_REF] Chang | Relational topic models for document networks[END_REF]). However, to cluster massive documents into different topics, pairGraphText is a better choice.

See Section 4 in the supplementary material for more simulations comparing pairGraphText with multiple methods including CASC and spectral clustering. qqqq q q q q qqqqqqqqqqqqqqqqqq qqqqqqqqq q q q q q qqqqqqqqqqqq qqqqqq q q q qq qqqqqq qqqqqqqqq 7. Discussion. This paper searches for (i) candidate-centered structure and (ii) issue-centered structure in the political discussions on Facebook surrounding the 2012 French election. The candidate-centered structure is relatively easy to detect since we have the labels of each post belongs to which candidate. But the search for issue-centered structure is more challenging, because we have no such labels of citizens or any labels of issues. To identify topics in the discussions, we use both the graph and the text. pairGraphText synthesizes the graph and the text, and it adresses the noisy and high-dimensional problem for text by thresholding. Using pairGraphText, we identify topics that attract people's attention, including Islam, religion, immigration, ecology, economy, and crises. During the interpretation of clusters, we propose the word-content strategy to extract the cluster topics, and our Shiny App https://yilinzhang.shinyapps.io/FrenchElection plays a signicant role in the interdisciplinary collaboration between statisticians and social scientists. Our codes and data sets are available on Github https://github.com/yzhang672/AOAS. We also provide an R package pairGraphText to implement our method on Github https://github.com/ yzhang672/pairGraphText. [START_REF] Chang | Hierarchical relational models for document networks[END_REF] proposed the relational topic model (RTM), a hierarchical probabilistic model for networks with node covariates. They modeled topic assignments for documents using latent Dirichlet allocation (LDA) [START_REF] Blei | Latent dirichlet allocation[END_REF]). Instead of studying networks of documents or posts, we study the bi-partite network between candidate-posts and citizens. Also, our method is unsupervised, more computationally efficient, and generally more accurate compared with RTM. RTM enables us to predict keywords and citations for new documents. However, to cluster documents into different topics, pairGraphText is a better choice than RTM.

pairGraphText is useful for applications outside of discussion threads. It is applicable to any network with node covariates. pairGraphText enhances the homogeneity of covariates within clusters. This boosts the signal of the clusters and helps with interpretation.

Fig 2 .

 2 Fig 2. Upper Tail of Degrees. Figure (a) shows the upper tail of citizen-degrees. 90% of the citizens write fewer than 10 comments, a small number of citizens write thousands of comments. Figure (b) shows the upper tail of post-degrees by candidate. The top three candidates: Hollande, Sarkozy, and Le Pen (on right), have the largest degrees.

  Categorizing the citizens based upon where they focus their attention produces a partition. For any partition of citizens, P : {1, . . . , N C } → {1, . . . , K C } where N C = 92, 226 is the number of citizens and K C is the number of citizen-clusters, define matrix Ψ C ∈ R K C ×8 such that for any a ∈ {1, . . . , K C } and b ∈ {1, . . . , 8}, (2.2) [Ψ C ] a,b = # of comments from citizens in cluster a under posts on bth candidate-wall (# of citizens in cluster a) × (# of posts on bth candidate-wall) .

Fig 4 .

 4 Fig 4. Citizen-Clusters Figure (a) shows interactions between the citizen-clusters and candidate-walls. The sizes of the balloons are the elements of ΨC (defined in (2.2)).Figure (b) shows the number of citizens in each cluster. In Figure (b), we label each citizen-cluster by the corresponding candidate . For example, the first citizen-cluster is Hollande-centered from Figure (a), so we label it as 1 (Hollande) in Figure (b).

2. 3 .

 3 Studying the graph using di-sim. Despite the overwhelming evidence for strong candidatecentered clusters in Figure 4, the spectral algorithm di-sim (Rohe et al. (2016)) finds a different partition of the citizens. Di-sim partitions both citizens and candidate-posts by applying a spectral clustering algorithm. It applies the singular value decomposition to a normalized version of the adjacency matrix A (defined in (2.1)) 2 . By applying k-means to the top left and right singular vectors, di-sim partitions the citizens and posts to different clusters. 3 Figure 5 displays the matrix Ψ C (defined in (2.2)) for the partition of citizens created by di-sim. Only the top three candidates have clusters that focus on them: Hollande and Sarkozy each has two clusters and Le Pen has one cluster that focuses on her. Other citizen-clusters (6,7,8) spread across multiple candidates.

Fig 5 .

 5 Fig 5. The Citizen-Clusters by DI-SIM. Similar to Figure 4(a), this figure shows the balloon plot of ΨC corresponding to the citizen-clusters by di-sim.

,

  where D C and D P are diagonal matrices with [D C ] ii = j A ij + τ c and [D P ] jj = i A ij + τ p , where τ c (τ p ) is set to be the average row (column) degree. When the uni-partite graph is undirected, D C = D P . CASC adds the covariate assisted part C = XX T to the regularized graph Laplacian and performs spectral clustering on the following similarity matrix S casc (h) = L + hC.

Fig 6 .

 6 Fig 6. Part of the Call-Response Matrix before and after Thresholding Some pairs of words are relatively more highly correlated, like nicolassarkozy and francoishollande, jeanlucmelenchon and jeanlucmelenchon, vot and franc, etc. After thresholding, only the relatively highly correlated pairs of words are left, making the call-response matrix much more sparse.

4. 1 .

 1 The clusters from pairGraphText with larger h are less candidate-centered. For each partition of candidate-posts, P : {1, ..., N P } → {1, ..., 4}, we define the matrix Ψ P ∈ R 4×8 such that for any a ∈ {1, ..., 4} and b ∈ {1, ..., 8}, (4.1) [Ψ P ] ab = # of posts in cluster a from candidate b's wall (# of posts in cluster a) × (# of posts from candidate b's wall).

Fig 7 .

 7 Fig 7. Clusters and Candidate-Walls Figure (a) and (b) display ΨP and ΨC in ballloon plots for the three cases.

Fig 9 .

 9 Fig 9. Cluster patterns when h = 0.035

  (a) compares the mis-clustering rate of pairGraphText and RTM. Without text signals (the middle plot), RTM fails to recover block labels even with large graph signals, but pairGraphText recovers block labels with the graph signal over 10. Without graph signals (the right plot), pairGraphText can only recover 90% of block labels, but RTM recovers all block labels, when the text signal is over 0.4. With both graph signals and text signals (the left plot), both methods perform better than the two cases when only one type of signals exists, and both methods can recover block labels with large enough signals. RTM generalizes the text-based topic modeling method, LDA (Blei et al., 2003), to integrate links (graph); it depends more on text and uses links to improve. On the other hand, pairGraphText generalizes the link-based spectral clustering to integrate text; it depends more on links and uses text to improve. From the Figure 11(a), RTM fails to recover block labels without text signals, but pairGraphText can still recover most block labels without graph signals. Figure 11(b) also shows that pairGraphText is much faster than RTM.

Fig 11 .

 11 Fig 11. Comparison between pairGraphText and RTM

Attention-Ratio Density Attention-Ratio of Citizens (with at least 10 Comments)

  shows the histogram of attention-ratio for all citizens with d i ≥ 10. Most of the mass of this histogram is close to one, indicating that most citizens primarily comment on one candidate-wall. This gives the first impression of candidate-centered structure.

		All			Hollande			Sarkozy			Le Pen
	6
	4
	2
	0
		Mélenchon			Bayrou			Joly			Dupont
	6
	4
	2
	0
	0.25	0.50	0.75	1.00	0.25	0.50	0.75	1.00	0.25	0.50	0.75	1.00	0.25	0.50	0.75	1.00
	Fig 3. Distribution of Citizens' Attention-Ratio. In this figure, we focus on citizens who have at least 10
	comments. The first plot displays the histogram of attention-ratio for all citizens. The rest eight plots are for the eight
	citizen-clusters based on the candidates they focus on. We don't display the citizens who focus on Poutou, because he
	attracts very few comments.

The data was gathered by sotrender.com. They collect all posts from the official Facebook profiles of the top eight candidates and all the comments beneath them. Citizens who commented on the candidate-posts are distinguished by identification numbers, which are corresponding to the urls of their Facebook profiles. sotrender.com does not control for citizens being human users (non-bots) or being unique users (e.g. without establishing artificial accounts in order to comment on candidate-posts).

This normalized version of the adjacency matrix A is the regularized graph Laplacian which we will define in details in (3.1)

Di-sim is similar to the step

-7 in Algorithm 1. It applies the singular value decomposition on the graph Laplacian instead.

We only display the links that correspond to the three or four largest elements of Ψ in each case.

Acknowledgements. We thank Jonathan Chang from Physera and David Blei from Columbia University for their advice in implementing the Relational Topic Models. We thank Emma Krauska and Fan Chen from University of Wisconsin-Madison for their discussions to name pairGraphText. We thank the Editor, Associate Editor, and reviewer who provide helpful comments on the manuscript.

SUPPLEMENTARY MATERIAL

Supplementary Materials for Discovering Political Topics in Facebook Discussion threads with Graph Contextualization (http://arxiv.org/src/1708.06872/anc/; .pdf). This supplementary consists of three parts. Part 1 provides more evidence for the candidate-centered structure. Part 2 explains our choice of the number of clusters K when searching for the issue-centered structure. Part 3 discusses different choices for document-term matrices. Part 4 provides more simulations comparing pairGraphText with RTM and other methods including CASC and spectral clustering. Part 5 provides theoretical justifications for pairGraphText.

The authors gratefully acknowledge support from NSF grant DMS-1612456 and ARO grant W911NF-15-1-0423. † The authors gratefully acknowledge support from Audencia Foundation Research grant.