Kevin Berlemont
email: kevin.berlemont@lps.ens.fr

Jean-Pierre Nadal

CONFIDENCE-CONTROLLED HEBBIAN LEARNING EFFICIENTLY EXTRACTS CATEGORY MEMBERSHIP FROM STIMULI ENCODED IN VIEW OF A CATEGORIZATION TASK

Confidence-controlled Hebbian learning efficiently extracts category membership from stimuli encoded in view of a categorization task

Kevin Berlemont, Jean-Pierre

As categorization occurs constantly in everyday life, the ability to learn new categories is essential to achieve accurate behaviors [START_REF] Ashby | Human category learning[END_REF]. In laboratory experiments, associations between the continuum of sensory stimuli and the discrete categories are learned through trial-and-error protocols [START_REF] Law | Reinforcement learning can account for associative and perceptual learning on a visual-decision task[END_REF]Gold, 2009, Engel et al., 2015]. Within the framework of reinforcement learning [START_REF] Sutton | Reinforcement learning: An introduction[END_REF], different learning algorithms have been proposed to account for such behavioral learning. As a bio-physically plausible Hebbian learning rule modulated by a reward signal [START_REF] Schultz | A neural substrate of prediction and reward[END_REF][START_REF] Loewenstein | Operant matching is a generic outcome of synaptic plasticity based on the covariance between reward and neural activity[END_REF][START_REF] Gerstner | Eligibility traces and plasticity on behavioral time scales: experimental support of neohebbian three-factor learning rules[END_REF], Reward-modulated Hebbian learning (RMHL) [START_REF] Legenstein | A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback[END_REF][START_REF] Legenstein | A reward-modulated hebbian learning rule can explain experimentally observed network reorganization in a brain control task[END_REF]. has been proposed and used in various contexts. However, such rule requires to keep track of the rewards associated to the previous trials to generate the prediction error. RMHL has in particular be used in models of categorical perception to adapt the weights between the encoding and the decision layers [START_REF] Engel | Choice-correlated activity fluctuations underlie learning of neuronal category representation[END_REF][START_REF] Min | A neural circuit mechanism of categorical perception: top-down signaling in the primate cortex[END_REF].

Experimental [START_REF] Goldstone | Influences of categorization on perceptual discrimination[END_REF][START_REF] Sigala | Visual categorization shapes feature selectivity in the primate temporal cortex[END_REF][START_REF] Harnad | Categorical perception[END_REF][START_REF] Xin | Sensory-to-category transformation via dynamic reorganization of ensemble structures in mouse auditory cortex[END_REF] and theoretical analysis [Bonnasse-Gahot andNadal, 2008, 2012] show that, for a task-optimized coding layer, tuning curves of coding neurons are sharper near than away from the category boundaries. These findings account for various effects characterizing categorical perception [START_REF] Harnad | Categorical perception[END_REF]. The impact of the optimization of the coding layer on the efficiency of the above mentioned Hebbian type learning has not been specifically addressed.

For what concerns the modeling of behaviour in categorization tasks, authors most frequently make use of Drift-to-Bound decision models (DDM) [START_REF] Ratcliff | A theory of memory retrieval[END_REF][START_REF] Bogacz | The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks[END_REF]. However, for a biophysical modeling taking into account both the coding and decision stages, one considers dynamical neural networks. Attractor neural decision networks account well for various behavioral data such as reaction times, accuracy and confidence [START_REF] Wei | Confidence estimation as a stochastic process in a neurodynamical system of decision making[END_REF]Wang, 2015, Berlemont et al., 2020].

The idea that confidence plays a role in learning has been proposed [START_REF] Summerfield | Expectation in perceptual decision making: neural and computational mechanisms[END_REF][START_REF] Meyniel | Brain networks for confidence weighting and hierarchical inference during probabilistic learning[END_REF][START_REF] Meyniel | Brain dynamics for confidence-weighted learning[END_REF]. Within the framework of drift-diffusion models and taking a Bayesian viewpoint, [START_REF] Drugowitsch | Learning optimal decisions with confidence[END_REF] have shown that the optimal learning rate for categorization tasks should depend on the confidence in one's decision, where confidence is defined as the probability of having answered correctly. Within the attractor neural network framework, confidence in one's decision is well accounted for by the difference, at the time of decision, between the neural activities of the decision-specific neural pools [START_REF] Wei | Confidence estimation as a stochastic process in a neurodynamical system of decision making[END_REF]Wang, 2015, Berlemont et al., 2020].

In this work, we consider this neural basis of confidence and, making use of the general framework of reward-based Hebbian learning [Hebb, 1949, Miller and[START_REF] Miller | The role of constraints in hebbian learning[END_REF], we investigate how confidence can improve learning. We consider a network composed of two layers: the coding layer and the decision layer. We compare two possible tunings of the coding neurons: uniformly distributed or optimized in view of the categorization task. En passant, we derive an analytical formula giving the optimal tuning curve distribution in the present context.

We find that reward-modulated Hebbian learning is not able to successfully make use of an optimized coding layer. In contrast, when the learning is controlled by confidence, a network with coding neurons optimized for the categorization task leads to better performance than a network with uniform tuning curves. Moreover, confidence-controlled Hebbian learning accounts for near optimal performances. Our findings show that confidence, as measured at the neural level, allows to successfully learn a categorization task with a local Hebbian rule, without keeping track of prediction errors.

RESULTS

NEURAL CIRCUIT MODEL AND LEARNING RULES

We consider a neural circuit trained to perform a categorization task, more precisely a two-alternative-forced choice (2AFC) task. Stimuli are sampled among two categories (such as motion directions for example) and sequentially presented to the network. The categories overlap, that is the category membership of a stimulus is more or less ambiguous depending on how far it is from the class boundary in the stimulus space. The behavioral task is to learn to distinguish the category of the stimuli through trial and error. The network is composed of two layers (Fig. 1.A), a stimulus encoding layer feeding a decision-making attractor network. Coding neurons are modeled as Poisson neurons with stimulus-specific bell-shaped tuning curves (Fig. 1.E). We restrict to a one dimensional stimulus space, a direction that discriminates the two categories. The activity of the coding layer is pooled from the decision layer. The decision layer is composed of an attractor network of two populations (C 1 and C 2 ) that compete with each other (Fig. 1.A). A decision is reached when the activity of one population crosses a threshold θ. This two-units model is a mean-field approximation of a large network of spiking neurons [START_REF] Wang | Probabilistic decision making by slow reverberation in cortical circuits[END_REF].

In the following, we compare two different networks: one where the tuning curves in the coding layer are uniformly distributed along the stimulus axis with identical widths w i = w (in the following referred to as the uniform network), and one where the distribution of tuning curves is optimized with respect to the categorization task (in the following referred to as the task-optimized network. In the latter case, tuning curves are sharper near the boundary between categories [START_REF] Bonnasse-Gahot | Neural coding of categories: information efficiency and optimal population codes[END_REF], the coding layer allocating more resources where the task is more difficult (greater ambiguity of the stimulus) -see Fig. 1.E.

For a given coding layer (optimized or not), we consider the learning occurring between the coding and decision layers through the adaptation of the synaptic connections. To analyse in detail the learning between the coding and decision layers, we do not consider the simultaneous adaptation of the tuning curves in the coding layer.

At each trial, the strength W j i of the synaptic connection between a coding neuron i and the winning population (C j ) of the trial is updated. We consider three reward-based Hebbian learning rules -which can be seen as examples of 'NeoHebbian Three-Factor Learning Rules' [START_REF] Gerstner | Eligibility traces and plasticity on behavioral time scales: experimental support of neohebbian three-factor learning rules[END_REF]. As reference, the first rule is a pure Hebbian reinforcement learning rule (hereafter PHL), equivalent to a Hebbian supervised learning rule:

W j i = W j i + δ r j r i R (1)
Here and in the following, R is the reward of the trial (1 if the decision was correct, -1 otherwise), r i the firing rate of the presynaptic neuron, r j the one of the post-synaptic unit, and we denote by δ the learning rate. The second rule is the Reward Modulated Hebbian Learning rule (RMHL),

W j i = W j i + δ r j r i (R -R x ) (2)
where R x is the moving average of the reward obtained on a short recent time window for the particular stimulus x on that trial. Note that R -R x , if non-zero, has the sign of R. Finally, the confidence-controlled reward-based Hebbian learning rule (CCHL) that we introduce in the present paper is defined by:

W j i = W j i + δ q r j r i R (3)
where q is the confidence control parameter defined by: q = 0 for a trial with high confidence 1 for a trial with low confidence (4)

That is, there is no updating of the weights for high confidence trials, and the learning rule is the pure reward-based Hebbian one in case of low confidence.

Following [START_REF] Wei | Confidence estimation as a stochastic process in a neurodynamical system of decision making[END_REF]Wang, 2015, Berlemont et al., 2019], confidence is here modelled as a function of the difference ∆r between the neural activities of the two populations of the decision layer at the time of decision. Jaramillo et al [START_REF] Jaramillo | Engagement of pulvino-cortical feedforward and feedback pathways in cognitive computations[END_REF] have proposed a pulvinar-cortical circuit model that can extract this quantity. In a previous work, we have shown that this difference in neural activity provides a neural basis for behavioral confidence in human [Berlemont et al., 2019]. We consider that a trial corresponds to a high (resp. low) confidence trial when the difference in activity ∆r at the time of the decision is greater (resp. lower) than a threshold level z.

More details about the model can be found in the Material and Methods section and in the Supplementary Information.

LEARNING: REWARD-MODULATED HEBBIAN LEARNING

First, we study the simplest case of pure reward-based Hebbian learning. Before learning, the synaptic connections are initialized with random values leading to performance at chance level. Fig. 1.C represents the weights of the coding neurons towards the neural population C 1 during learning.

In Fig. 2.A we compare the performances achieved by the network for different sizes, for the two types of coding layers (uniform and task-optimized tuning curves) after 2000 trials. The large number of trials is chosen (here and for most results shown in this paper) so that the weights stabilize, allowing to obtain well-defined average performances for all values of parameters. However, it is important to note that the network does learn to perform the behavioral task with a rather good success rate, and this after only a small number of trials as can be seen in Fig. 2.C and D.

In panels 2.E and 2.F, we show the difference in accuracy between a task-optimized network and an uniform one in the plane stimulus ambiguity/network size: Red (resp. Blue) regions correspond to higher (resp. lower) performances for the task-optimized coding layer with respect to the uniform one. We show the results for the PHL (Fig. 2.E) and the RMHL (Fig. 2.F), at medium overlap of the categories.

Surprisingly, the performance appears to be generally better for the non-optimized network. First, when the distribution of the tuning curves is uniform, the performances do not vary significantly with N , whereas, for a task-optimized network, the accuracy decreases with the number of coding neurons. Secondly, when the number of neurons increases, a network with a non-optimized coding layer performs better that a task-optimized network. It is only for small values of N near the category boundary that the performances are better with an optimized coding layer (2.E and 2.F). The region where uniform coding performs better increases with α. However, the difference in accuracy between the two networks decreases if one keeps increasing α (see Supplementary Fig. 2 for an example). This is explained by the fact that, when the categories widths are wider, the overlap between the tuning curves is larger, and the optimal code becomes more similar to an uniform one.

These apparent paradoxical behaviors all result from the same reason. In the case of the task-optimized coding layer, the tuning curves are sharp at the boundary between categories in order to maximize the Fisher information. This means that a neuron close to the boundary will only emit spikes for stimuli close to the center of its tuning curve. Thus, the strength of the associated synaptic connections will less often be updated than for a neuron far away from the boundary, whose firing rate is higher in average. Finally, the weights tend to decrease to zero close to the category boundary, in effect 'fighting' against the optimal tuning of coding cells (Fisher information should be large at such location). Hence, during learning the network progressively loses information as finely tuned coding neurons tend to be associated with a synaptic connection of decreasing strength.

LEARNING: CONTROL BY CONFIDENCE

In order to counterbalance the loss of information near the category boundary, the idea would be to only update the synaptic connections when it is useful to do so for the network, that is when it would gain information in doing so. In other words, once the behavioral task begins to be acquired, one would like to finely tune the learning by only considering the hard trials, hence the ones near the categories boundary [START_REF] Krauth | Learning algorithms with optimal stability in neural networks[END_REF][START_REF] Guyon | Discovering informative patterns and data cleaning[END_REF][START_REF] Alemi | A three-threshold learning rule approaches the maximal capacity of recurrent neural networks[END_REF].

This can be achieved through the control of learning by the confidence. Let us first see how confidence, as measured by Red (resp. Blue) corresponds to a positive (resp. negative) difference, meaning that the network with an optimized coding layer has better (resp. worse) accuracy that the one with a uniform coding layer. The x-axis gives the stimulus x (category boundary at x = 0.5), and the y-axis to the number of neurons N . The width of the categories (centered at 0 and 1) is α = 0.2.

the difference ∆r between the activities of the decision units at the time of decision, is being built during the learning process. Fig. 8.F shows the mean level of confidence of the network, defined as the mean value of ∆r, with respect to the stimulus x that is presented. One can see that confidence builds up at the early stage of the learning process. One can state that the notion of confidence is present in the network from the very beginning of the learning process. As expected, the confidence level decreases as the stimulus is chosen closer to the categories boundary. As the number of learning steps increases, convergence in confidence level is fast far from the boundary, and slow near the boundary. As the task is learned more accurately, the representation of confidence within the attractor network becomes sharper at the boundary.

We thus expect the CCHL to make use of this sharpening representation of confidence during learning to improve the performances of the network near the categories boundary.

In Fig. 8.A and B we consider the accuracy near the boundary (stimulus x = 0.45) for the confidence-controlled learning rule as compared to the pure Hebbian one. The important parameter in the CCHL scheme is the confidence threshold z. This parameter determines the fraction of trials that will be considered as low or high confidence trials. The x-axis in Fig. 8.A and B represents the fraction of low confidence trials as one varies the threshold z. We observe that, for low values of N , the confidence control does not have any impact on the performances. This is easily understood: due to the low amount of resources, the network can not have really sharp tuning curves at the boundary and is less affected by the decay of weights near the boundary. However, as soon as one increases the number of neurons N , confidence control has an impact on the performances. The tendency of all the curves is the same: an increase of performance when the confidence threshold decreases, until it reaches a maximum at a confidence threshold corresponding to ∼ 25% of low-confidence trials. Finally, as one keeps decreasing the threshold z, the performances decrease: learning becomes inefficient as there are too few stimuli leading to a change in weight. Finally, we find that the value of confidence threshold z leading to the best performances is stable with respect to the number of trials (Supplementary Fig. 1), with a plateau around ∼ 25% of low-confidence trials. These results imply in particular that the control by confidence can be kept constant during learning and does not need to adapt to the current state of learning.

Figure 8.C and D represent the Fisher information (F code ) of the coding layer combined with the synaptic weights (see Material and Methods), through the learning process. We note that the impact of confidence depends strongly on the coding layer. For an uniform coding, the Fisher information is almost unchanged after learning, at low or high N . For an optimized coding layer one can observe a strong modification of the Fisher information during learning. The confidence control leads to an increase of F code near the boundary by giving more weights to the neurons closer to the categories boundary. CCHL thus efficiently makes use of the optimized coding layer to increase the performances where the task is more difficult.

CONFIDENCE-CONTROL LEARNING EFFICIENTLY EXTRACTS INFORMATION FROM THE TASK-OPTIMIZED CODING

LAYER

We now compare the performances for the optimized and the uniform coding layer for the confidence-controlled scheme.

We present the results in Fig. 4, the analogous of Fig. 2, panels E and F, showing the differences in accuracy between a network with an optimized coding layer, and one with an uniform coding layer, in the plane stimulus ambiguity/network size, and for two values of the categories width, corresponding to medium and large overlaps of the categories (see Supplementary Fig. 3 for the late stage learning).

We note that with the optimized coding layer the performances are better, independently of the number of neurons N . Indeed, by focusing on trials with low confidence, the network tends to decrease the synaptic weights far from the boundary and increase the ones close to the boundary (Supplementary Fig. 4) With higher weights close to the category boundary, the decision network takes advantage of the sharp tuning curves in the optimized coding layer to obtain a better accuracy.

In Supplementary Fig. 5, we compare the performances of the confidence-controlled learning rule with the ones of the RMHL. In agreement with the results on the Fisher information, we find that confidence-controlled Hebbian learning outperforms this learning scheme which requires to store the mean reward obtained so far for this stimulus.

CONFIDENCE-CONTROLLED HEBBIAN RULE ALMOST ACHIEVE OPTIMAL PERFORMANCES

We ask now how the performances obtained with the confidence-controlled rule compare to the optimal ones that such network could achieve on average over the stimulus space, regardless of the learning rule. The way we define and compute the optimal performances is given in the Material and Methods section. Fig. 5 presents the difference between the optimal performances and the ones from the confidence-controlled Hebbian learning rule, at different stages of the learning process for a category width α = 0.2, and for both the non-optimized and the optimized coding layer. We first note that near the boundary, the network with optimized coding layer has better performances than the optimal model. However, the global performances are lower because it performs worse when the stimuli are a bit further from the boundary. Surprisingly, if we increase the number of trials (Fig. 5.C and D), the performances of the neural circuit model with a uniform coding layer do not increase. For the optimized coding layer, the accuracy increases and the network performances becomes very close to the optimal ones, especially at small network sizes. In Supplementary Fig. 6 we show that these qualitative results are observed at different values of the categories width.

DISCUSSION

In this work, we proposed a learning mechanism for categorization tasks that do not rely on a prediction error signal.

Comparing with both a standard reward-based Hebbian rule and the reward-modulated Hebbian rule, we showed that a networks with task-optimized and uniform coding layer. Red (resp. Blue) corresponds to a positive (resp. negative) difference, meaning that the network with an optimized coding layer has better (resp. worse) accuracy that the one with a uniform coding layer. The x-axis gives the stimulus x (category boundary at x = 0.5), and the y-axis to the number of neurons N . The width of the categories (centered at 0 and 1) is α = 0.2 for (A) and α = 0.3 for (B). Red (resp. Blue) corresponds to a positive (resp. negative) difference, meaning that the performances for the neural network with confidence-controlled learning are less (resp. higher) than the optimum performances for this neural architecture. (A) corresponds to the case where the coding layer is uniform, and (B) when it is optimized. (C) and (D): same as (A) and (B) respectively, but for 10000 learning trials. All panels: categories of width α = 0.2.

Stimulus Stimulus Stimulus Stimulus

Stimulus Stimulus Stimulus Stimulus

scheme controlled by confidence increases significantly the performance of the neural circuit model for the categorization task, and actually achieves almost optimal performances.

The learning considered here is the one of the weights between a stimulus encoding layer and an attractor decision network. We investigated the impact on learning of having a task-specific encoding as compared to the case of an uniform coding of the stimulus. Our work demonstrates that reward-modulated Hebbian learning fails to extract the relevant information from an optimized coding layer, whereas the confidence-controlled scheme is on the contrary, particularly efficient in such cases. In addition, in contrast to RMHL, the proposed confidence-controlled rule does not need to store past average rewards.

The adaptation of the tuning curves within coding layers has been studied experimentally [START_REF] Fritz | Adaptive, behaviorally gated, persistent encoding of task-relevant auditory information in ferret frontal cortex[END_REF][START_REF] Xin | Sensory-to-category transformation via dynamic reorganization of ensemble structures in mouse auditory cortex[END_REF]. Authors have shown that task-specific neural representations develop across different cortical areas [START_REF] Cromer | Representation of multiple, independent categories in the primate prefrontal cortex[END_REF][START_REF] Fitzgerald | Generalized associative representations in parietal cortex[END_REF] during the learning of a perceptual task. These representations are accompanied by a modification in the tuning properties of the neurons. In monkeys trained to classify directions of random dot motion into two arbitrary categories [START_REF] Freedman | Experience-dependent representation of visual categories in parietal cortex[END_REF], tuning changes are observed for neurons in the lateral intraparietal (LIP) area. In trained monkeys, individual neurons tend to show smaller differences in firing rate within categories, and larger differences in firing rate between categories [START_REF] Freedman | Experience-dependent representation of visual categories in parietal cortex[END_REF], whereas in naive animals LIP neurons represent directions uniformly with bell-shaped tuning functions [START_REF] Fanini | Direction selectivity of neurons in the macaque lateral intraparietal area[END_REF]. Similar reorganization has also been observed in mice for an auditory categorization task [START_REF] Xin | Sensory-to-category transformation via dynamic reorganization of ensemble structures in mouse auditory cortex[END_REF]. Similar adaptation may also occur in an unsupervised way in the absence of a categorization task [START_REF] Köver | Perceptual and neuronal boundary learned from higher-order stimulus probabilities[END_REF].

The adaptation of the coding layer has also been studied computationally [START_REF] Bonnasse-Gahot | Neural coding of categories: information efficiency and optimal population codes[END_REF][START_REF] Engel | Choice-correlated activity fluctuations underlie learning of neuronal category representation[END_REF][START_REF] Tajima | Population code dynamics in categorical perception[END_REF], 2017[START_REF] Min | A neural circuit mechanism of categorical perception: top-down signaling in the primate cortex[END_REF]. Whereas most models of decision-making consider an uniform coding of the stimulus before the decision part [START_REF] Beck | Probabilistic population codes for bayesian decision making[END_REF][START_REF] Drugowitsch | Learning optimal decisions with confidence[END_REF], few models analyse the nature of a stimulus coding layer optimized in view of a categorization task [START_REF] Bonnasse-Gahot | Neural coding of categories: information efficiency and optimal population codes[END_REF]. In agreement with the experimental findings, theoretical predictions with a two layers feedforward architecture give that tuning curves in the coding layer are sharper at the vicinity of a decision boundary. In addition, within a purely statistical framework, a gradient-based supervised learning rule allows to learn the tuning curves parameters [START_REF] Bonnasse-Gahot | Neural coding of categories: information efficiency and optimal population codes[END_REF].

Authors have proposed an alternative type of model, assuming a top-down modulation of the decision layer onto the coding layer [START_REF] Tajima | Population code dynamics in categorical perception[END_REF][START_REF] Tajima | Task-dependent recurrent dynamics in visual cortex[END_REF]. To obtain the adaptation of the tuning curves, authors use a top-down modulation under the form of a reward-modulated Hebbian learning [START_REF] Engel | Choice-correlated activity fluctuations underlie learning of neuronal category representation[END_REF][START_REF] Min | A neural circuit mechanism of categorical perception: top-down signaling in the primate cortex[END_REF].

It will be interesting to see how, for the neural network architecture studied in the present work, both the tuning curves parameters and the weights from the coding layer to the decision attractor network can be efficiently learned altogether with confidence-controlled Hebbian learning rules.

Learning is accompanied by a sense of confidence about the different predictions [START_REF] Nassar | An approximately bayesian delta-rule model explains the dynamics of belief updating in a changing environment[END_REF]. This sense of confidence plays a functional role in learning [Nassar et al., 2010, Meyniel and[START_REF] Meyniel | Brain networks for confidence weighting and hierarchical inference during probabilistic learning[END_REF] as it sets the balance between predictions and new information. Many studies report the existence of surprise signals in the brain, i.e a strong signal in presence of unexpected stimulus [Hillyard et al., 1971, Summerfield and[START_REF] Summerfield | Expectation in perceptual decision making: neural and computational mechanisms[END_REF]. Theoreticians have studied different models of surprise-based learning, in the absence of reward (see [START_REF] Gerstner | Eligibility traces and plasticity on behavioral time scales: experimental support of neohebbian three-factor learning rules[END_REF] for a review). More recently, making use of fMRI [START_REF] Meyniel | Brain networks for confidence weighting and hierarchical inference during probabilistic learning[END_REF], EEG [START_REF] Jepma | Catecholaminergic regulation of learning rate in a dynamic environment[END_REF][START_REF] Nassar | Statistical context dictates the relationship between feedback-related eeg signals and learning[END_REF] or MEG [START_REF] Meyniel | Brain dynamics for confidence-weighted learning[END_REF] techniques, it has been shown that this surprise signal is controlled by confidence. Confidence has been shown to grade the reward signal and impact the subsequent learning in a categorization task with mice [START_REF] Lak | Dopaminergic and prefrontal basis of learning from sensory confidence and reward value[END_REF]. This control of the reward signal by confidence could be crucial to implement adjustable learning rates in the brain [Behrens et al., 2007, Meyniel and[START_REF] Meyniel | Brain networks for confidence weighting and hierarchical inference during probabilistic learning[END_REF]. We recall that, within the framework of attractor neural networks, confidence is given by a local signal intrinsic to the nonlinear neural dynamics.

We note that our proposed learning scheme is reminiscent of machine learning heuristics selecting a subset of the available learning examples to efficiently learn a task. For instance, the informative vector machine algorithm [START_REF] Herbrich | Fast sparse gaussian process methods: The informative vector machine[END_REF][START_REF] Lawrence | Learning to learn with the informative vector machine[END_REF] tends to choose points that are maximally informative. As a variant of the Perceptron algorithm, the minimum-overlap learning rule [START_REF] Krauth | Learning algorithms with optimal stability in neural networks[END_REF] leads to a learning with optimal margin by making use of a similar mechanism.

Finally, our confidence-controlled learning scheme is in the spirit of the Three-Threshold Learning Rule (3TLR) [START_REF] Alemi | A three-threshold learning rule approaches the maximal capacity of recurrent neural networks[END_REF] proposed for the encoding and retrieval of memories in recurrent neural networks. There, it is the underlying local field which provides to a given neuron the analogous of a confidence signal: Hebbian potentiation/depression occurs only if the local field is above/below some threshold (smaller than the one for the emission of a spike). The authors show that this 3TLR leads to a storage capacity close to the maximum theoretical capacity. Similarly, we have shown that the confidence-controlled rule achieves near-optimal decision performances.

MATERIAL AND METHODS NEURAL CIRCUIT MODEL AND NUMERICAL PROTOCOL

The neural model is composed of two layers: the coding layer and the decision layer.

Categories and stimuli. We consider a one-dimensional stimulus, x ∈ [0, 1]. Each stimulus belongs to one of two categories, C 1 and C 2 . The categories are characterized by the probabilities P (x|µ), µ = C 1 , C 2 taken as Gaussian in all simulations, with centers at 0 and 1, and width (std) α. The category boundary is thus at x = 0.5. The input x thus also measures the signal ambiguity.

Coding layer. The coding layer consists in N independent Poisson neurons whose firing rates are described by their bell-shaped tuning curves [START_REF] Tolhurst | The statistical reliability of signals in single neurons in cat and monkey visual cortex[END_REF]. For a given input x, the tuning curve of the coding cell i = 1, ..., N , is

f i (x) = exp -(x -x i ) 2 / 2w 2 i (5)
where x i and w i are, respectively, the center (preferred stimulus) and the width of the tuning curve.

Decision layer. The decision network consists in two populations C 1 and C 2 representing the categorical choice [START_REF] Wang | Probabilistic decision making by slow reverberation in cortical circuits[END_REF]. This circuit pool activity from the coding layer and produce a noisy winner-take-all dynamics. This behavior is obtained through global inhibition and recurrent excitation [START_REF] Wong | A recurrent network mechanism of time integration in perceptual decisions[END_REF]]. More detailed information on the dynamical equations can be found in the Supplementary Information.

Sequences of trials. Each simulation consists in a sequence of trials. Each trial consists in the presentation of a randomly chosen stimulus x followed by a 1 s intertrial interval. During the presentation of the stimulus, the neurons in the coding layer fire a Poisson train with rate given by Eq. [5]. The stimulus presentation lasts until one of the population of the decision circuit reaches a threshold θ of 20 Hz. The choice made by the network on this trial corresponds to the category associated to the winning population. Once a decision has been made, there is no more input from the coding neurons to the decision neurons, while the decision neurons receive an inhibitory current (corollary discharge, see Supp. Information) [START_REF] Engel | Choice-correlated activity fluctuations underlie learning of neuronal category representation[END_REF], Berlemont et al., 2019, Berlemont and Nadal, 2019]. This brings the activity back towards the resting state, allowing the network to engage into the next trial.

Plasticity. The synapses connecting the coding layer to the decision layer are plastic [Schultz et al., 1997, Loewenstein and[START_REF] Loewenstein | Operant matching is a generic outcome of synaptic plasticity based on the covariance between reward and neural activity[END_REF]. At the end of each trial, the weights W j i are updated according to the considered learning rule, Eq. [1], [2] or [3]. The reward is equal to R = 1 on correct trials and R = -1 on incorrect trials. The learning rate is δ = 0.0025.

For simplicity, we impose at all times the symmetry W 1 i = -W 2 i for every i, allowing to speed up the simulations. The weight modification is thus only applied to the winning population. Since at the time of decision the firing rate of the latter is equal to the decision threshold θ (with small fluctuations), for each learning rule, we replace r j by θ, here 20Hz.

For each rule, we also added a synaptic normalization mechanism after the update of the weights (Oja's rule),

W j i → W j i / i W j i 2
. This norm-preservation mechanism prevents the divergence of the learning algorithm in the cases PHL and CCHL. It is not necessary for the RMHL but it is made here to make proper comparisons with the two other rules.

Before each numerical simulation, the synapses are initialized from an uniform distribution between [-1, 1]. Unless otherwise specified, the performance of the network for a specific input is computed as the average over 2000 trials.

OPTIMIZATION OF THE CODING LAYER

Most works on optimal perceptual coding consider an encoding layer optimized for the estimation or reconstruction of the stimulus itself. In contrast, here we consider a neural encoding optimized in view of the categorization task. Within a Bayesian/information theoretic approach, the neural code should maximize the mutual information between the category membership and the neural activity in the coding layer [START_REF] Bonnasse-Gahot | Neural coding of categories: information efficiency and optimal population codes[END_REF].

In the limit of a large number of coding cells, this mutual information reads

I(µ, r) = I(µ, x) - 1 2 F cat (x) F code (x) P (x)dx (6)
with µ the category membership, x the stimulus and r the neural response of the coding layer. I(µ, x) is the mutual information between category membership and stimulus, a constant characterizing the signal information content (how much the stimulus carries information about the category). F code is the (standard) Fisher information of the neural code, measuring the sensitivity of the neural activity with respect to small changes in the stimulus value. F cat is the categorical Fisher information quantity, a property of the signal: it measures how a small change in stimulus value affects the category likelihood. More details can be found in [Bonnasse-Gahot andNadal, 2008, 2012].

To derive the optimal distribution of tuning curves for the coding layer, we adapt to our framework a method recently introduced for the optimal coding of a stimulus [START_REF] Ganguli | Efficient sensory encoding and bayesian inference with heterogeneous neural populations[END_REF]. Assuming that the set of tuning curves fully tiles the input space with some local density d(x), one can replace the Fisher information F code (x) by d(x) 2 . We then minimize the integral in the Eq. [6] above under the constraint of a fixed number of neurons. The loss function is then the following:

F cat (x) d 2 (x) P (x)dx + λ d(x)dx -N (7)
with λ a Lagrange parameter. Taking the derivative with respect to d(x), we obtain the optimal local density:

d(x) ∝ (P (x)F cat (x)) 1/3 (8)
The widths of the tuning curves are given by 1/d(x). For the numerical simulations with a given number N of coding cells, we discretize the function 1/d(x) to get the N preferred stimuli.

Since the categorical Fisher information is larger where the stimulus is more ambiguous, one recovers from Eq. [8] the fact that tuning curves are sharper near the category boundaries [START_REF] Bonnasse-Gahot | Neural coding of categories: information efficiency and optimal population codes[END_REF].

FISHER INFORMATION QUANTITIES

We recall here the definitions of the Fisher information quantities mentioned in this paper:

F code (x) = - ∂ 2 ln P (r|x) ∂x 2 P (r|x) d N r (9)
and

F cat (x) = - µ=C1,C2
∂ 2 ln P (µ|x) ∂x 2 P (µ|x).

(10)

For Gaussian categories, F cat (x) is easily computed.

Given that the input to the decoding unit C 1 from the coding cell i is W 1 i r i , one may consider that the effective output of the coding cell i is characterized by the tuning curve W 1 i f i (x) -the weights thus acting as gain modulations. For the analysis whose results are shown in Fig. 8 panels (C) and (D), we compute the Fisher information F code making use of these effective tuning curves.

OPTIMAL PERFORMANCES

To obtain the optimal performances the network model can achieve, we first consider a proxy network obtained by replacing the decision attractor network by a decision function p. This function results from a fit of the relation between the input received by the attractor network and the probability of choosing the most probable category. Given this function p, we then define a cost function as an average of the performances over stimulus space. We find the set of weights maximizing the average performances by solving a non-linear convex optimization problem. See Supp. Information for details.

NUMERICAL SIMULATIONS AND CODES AVAILABILITY

For the simulations of the neural model we made use of the Julia language [START_REF] Bezanson | Julia: A fresh approach to numerical computing[END_REF]. For the integration of the differential equations we made use of the Euler-Maruayama method with a time step of 0.5 ms. For the computation of the optimal network performances, we made use of the Matlab language [Mat, 2018]. Codes will be made available on GitHub (https://github.com/berlemontkevin).

OPTIMAL PERFORMANCES

In the material and methods section we briefly explain how to obtain the optimal performances such neural networks could achieve, irrespective of the learning rule. Here we give more details about this procedure.

To obtain the optimal performances the network model can achieve, we first consider a proxy network obtained by replacing the decision attractor network by a decision function p. This function results from a fit of the relation between the input received by the attractor network and the probability of choosing the most probable category.

This analog system is defined by the probability for the attractor network to respond correctly when a specific current is sent. We numerically obtain this probability by averaging the behavioral results over 1000 trials for a few different currents. Next we fit the resulting non-parametric function by analytic function. Given the observed shape of the empirical function, we fit with a sigmoid curve. This gives a decision function p that directly describes the behavior of the attractor network when it receives a specific current.

Given this function p, we then define the cost function as an average over stimulus space:

L = 1 0 1 -p( i W 1 i f i (x)) dx (17)
with f i (x) the tuning curve of neuron i, x being the stimulus input. We find the set of weights minimizing this cost function by solving a non-linear convex optimization problem, with the same set of constraints as in the neural circuit model: symmetry (W 1 i = -W 2 i ), fixed norm ( i (W 1 i ) 2 = 1), and W 1 i > 0. for 10000 learning trials. Blue (resp. Red) corresponds to a positive (resp. negative) difference, meaning that the performances for the neural network with confidence-controlled learning are less (resp. higher) than the optimum performances for this neural architecture. (A) corresponds to the case where the coding layer is uniform, and (B) when it is optimized. All panels: categories of width α = 0.3.

Figure 1 :

 1 Figure 1: Neural circuit model. (A): Schematic of the circuit model. The network is composed of two layers. Neurons in the coding layer are tuned to specific values of the 1-d stimulus. The decision layer pools activity of the coding neurons through feedforward synapses. The category decision is obtained through competitive attractor dynamics. The synaptic connections between the two layers undergo Hebbian plasticity modulated by a reward signal. (B): Example of the dynamics of the decision layer n presentation of a stimulus. Each color represent the activity of the corresponding population. The decision is reached when the activity of one of the populations reach the decision threshold θ (dotted line). The behavioral confidence is a monotonous function of the difference ∆r in firing rates between the two populations, computed at the time of decision. (C): Synaptic weights from the coding layer to one of the two neural populations, C 1 . Each color represents a different epoch (each of 100 trials) in the learning process, from 100 to 1000 trials. The coding layer is composed of 20 neurons, uniformly distributed. (D): Fisher information of the coding layer. Orange curve: difference in Fisher information between the case of an optimized layer and the one of a uniform distribution of tuning curves (see Materials and Methods). Black curves: categories used for this example. (E): Tuning curves. Red curves: tuning curves for an uniform distribution. Blue curves: tuning curves of the task-optimized coding layer. For this example, coding layer of 20 neurons. For the numerical examples (B, C, D, E), the Gaussian categories are centered at 0 and 1 respectively, and of same width α = 0.25 (black curves, panel D).

Figure 2 :

 2 Figure 2: Performance of the network after learning: Probability of choosing category C 1 with respect to the presented stimulus x after 2000 trials. (A) and (B): Effect of network size. (A): Case of the PHL rule. (B) case of the CCHL rule. For both panels, Green curves: uniform network; Grey curves: task-optimized network. The number of neurons in the coding layer is indicated by going from light to dark colors. (C) and (D): Early stages of learning illustrated on the case of PHL. The gradient of color represents different snapshots of the network state after n = 5, 30, 55, 80 learning trials. (C, blue curves): task-optimized coding layer; (D, red curves): uniform coding layer. The behavioral task is learned after 30 trials. All panels: The width of the Gaussian categories is α = 0.25, the number of coding cells is N = 20; the threshold for confidence control is z = 15 Hz. (E) and (F): Difference in performances after 2000 learning trials between networks with uniform and task-optimized coding layer, for the pure Hebbian learning rule (PHL, panel E) and the Reward-modulated rule (RMHL, panel F). Red (resp. Blue) corresponds to a positive (resp. negative) difference, meaning that the network with an optimized coding layer has better (resp. worse) accuracy that the one with a uniform coding layer. The x-axis gives the stimulus x (category boundary at x = 0.5), and the y-axis to the number of neurons N . The width of the categories (centered at 0 and 1) is α = 0.2.

Figure 3 :

 3 Figure3: Learning with the confidence-controlled Hebbian learning rule (CCHL). (A): Effect of confidence control on the performances for a network with an optimized coding layer after 2000 trials. The y-axis represents the difference in performances, at an ambiguity x = 0.45, for the learning rule controlled by confidence and for the learning without confidence-control. The x-axis represents the percentage of trials for which ∆r was lower than the threshold z. (B): Same as (A) but for a neural circuit model with an uniformly distributed coding neurons. (C) and (D): Fisher information of the coding layer taking into account the gain modulation due to the synaptic weights (see Material and Methods), for N = 15, panel (C), and N = 35, panel (D). The gradient of color represents different snapshots of the network during learning (after an increasing number of trials, from 100 to 900). The network with an uniform coding layer is represented in red and the one with an optimized coding layer in blue. The dashed lines stands for the pure Hebbian learning rule; and the plain lines for the confidence-controlled Hebbian rule. The green color corresponds to the optimized network with the RMHL rule. (E): Performance for the task-optimized network for different numbers of learning trials (logarithmic scale). (F): ∆r averaged over 1000 trials with respect to the stimulus ambiguity. The different curves correspond to different numbers of learning steps. The number of coding cells is N = 20 for panels E and F. All panels: The width of the Gaussian categories is α = 0.25. Panels (C) to (F), the threshold for confidence control is z = 15 Hz.

Figure 4 :

 4 Figure4: Difference in performances after learning (2000 trials, with control by confidence, threshold z = 15 Hz) between networks with task-optimized and uniform coding layer. Red (resp. Blue) corresponds to a positive (resp. negative) difference, meaning that the network with an optimized coding layer has better (resp. worse) accuracy that the one with a uniform coding layer. The x-axis gives the stimulus x (category boundary at x = 0.5), and the y-axis to the number of neurons N . The width of the categories (centered at 0 and 1) is α = 0.2 for (A) and α = 0.3 for (B).

Figure 5 :

 5 Figure5: Difference between confidence-controlled Hebbian learning and optimal performances. (A) and (B) results for 2000 learning trials. Red (resp. Blue) corresponds to a positive (resp. negative) difference, meaning that the performances for the neural network with confidence-controlled learning are less (resp. higher) than the optimum performances for this neural architecture. (A) corresponds to the case where the coding layer is uniform, and (B) when it is optimized. (C) and (D): same as (A) and (B) respectively, but for 10000 learning trials. All panels: categories of width α = 0.2.

Figure 7 :Figure 8 :Figure 9 :Figure 10 :

 78910 Figure7: Late stage of learning. Difference of performances after learning (10000 trials) between networks with an uniform (or optimized) coding layer. Red (resp. Blue) corresponds to a positive (resp. negative) difference meaning that the network with the optimized coding layer has better (resp. worse) accuracy that the one with uniform layer. The x-axis corresponds to a variation of stimulus x, and the y-axis to the number of neurons N . (A), (D) and (G) correspond to the pure Hebbian learning rule described in the main text. The width of the categories is 0.2 for (A), 0.3 for (D) and 0.4 for (G). (B), (E) and (H) correspond to the reward modulated learning rule. The width of the categories is 0.2 for (B), 0.3 for (E) and 0.4 for (H). (C), (F) and (I) correspond to a learning with reward modulation by the confidence, with a threshold z = 15 Hz. The width of the categories is 0.2 for (C), 0.3 for (F) and 0.4 for (I).

Figure 11 :

 11 Figure11: Difference between confidence-controlled Hebbian learning and optimal performances. (A) and (B) results for 10000 learning trials. Blue (resp. Red) corresponds to a positive (resp. negative) difference, meaning that the performances for the neural network with confidence-controlled learning are less (resp. higher) than the optimum performances for this neural architecture. (A) corresponds to the case where the coding layer is uniform, and (B) when it is optimized. All panels: categories of width α = 0.3.

Acknowledgments We are grateful to Laurent Bonnasse-Gahot for useful discussions and suggestions. KB acknowledges a fellowship from the ENS Paris-Saclay.

Competing interests

The authors declare no competing interests.

SUPPLEMENTARY INFORMATION NEURAL CIRCUIT MODEL -DECISION LAYER

Here we give details on the dynamical equations of the decision layer. We model the decision layer by an attractor neural network, considering the mean-field approximation reducing the network to two effective population units [START_REF] Abbott | Drivers and modulators from push-pull and balanced synaptic input[END_REF], [START_REF] Wong | A recurrent network mechanism of time integration in perceptual decisions[END_REF]. Each excitatory neural population is described by a single variable s representing the fraction of activated N-methyl-D-aspartate receptor conductance, governed by:

with γ = 0.641 and τ s = 100 ms. The firing rate r D of a decision unit is given by:

with I the corresponding total synaptic current, a = 270Hz/nA, b = 108Hz and d = 0.154s. The synaptic currents input to populations C 1 and C 2 are, respectively:

with J j,j the synaptic couplings between decision units (J 1,1 = J 2,2 = 0.2609 nA and J 1,2 = J 2,1 = 0.0497 nA). The input received by the decision unit j from neuron i in the coding layer is W j i r i , with W j i the synaptic coupling and r i the spike train of the Poisson neuron i (resulting from the presentation of a stimulus), convoluted with a 100 ms interval. Finally, the noise terms in the synaptic currents correspond to Ornstein-Uhlenbeck processes:

with τ noise = 2 ms a synaptic time constant filtering the white-noise, and I 0 = 0.3255 nA.

When the firing rate of one of the decision units reaches a decision threshold θ, the decision is made, and the stimulus is removed. In the simulations, we take θ = 20 Hz.

During a sequence of trials, after a decision has been made, in the absence of stimulus, we add a relaxation dynamics taking into account a non specific inhibitory input to each decision unit, the corollary discharge [START_REF] Engel | Choice-correlated activity fluctuations underlie learning of neuronal category representation[END_REF], Berlemont et al. [2019], Berlemont and Nadal [2019]. The current of the corollary discharge is of the form : Effect of confidence modulation on the performances for a network with an optimized coding layer after 10000 trials and α = 0.25. The y-axis represents the difference between the performances, at an ambiguity x = 0.45, for a learning with modulation by confidence and for a learning without. The x-axis represents the percentage of trials where ∆r was lower than the threshold z. (B): Same as (A) but for a neural circuit model with an uniformly distributed coding neurons. (C): Effect of confidence modulation on the performances for a network with an optimized coding layer after 1000 trials and α = 0.25. The y-axis represents the difference between the performances, at an ambiguity x = 0.45, for a learning with modulation by confidence and for a learning without. The x-axis represents the percentage of trials where ∆r was lower than the threshold z. (D): Same as (C) but for a neural circuit model with an uniformly distributed coding neurons.


Les réseaux routiers en Bretagne en 1769

Source : Jean Ogée, Atlas Itinéraire de Bretagne contenant les cartes particulières de tous les grands chemins de cette Province avec tous les objets remarquables qui se rencontrent à une demie lieue à droite et à gauche, Nantes, Chez l'auteur, 1769.


Andjelika Bjelajac
email: abjelajac@tmf.bg.ac.rs

Drejc Kopač

Antoine Fecant

Eugénie Tavernier

Rada Petrović

Blaž Likozar

Djordje Janaćković

Eugenie Tavernier

Micro-kinetic modelling of photocatalytic CO2 reduction over undoped and N-doped TiO2

Introduction

The rise of CO 2 emission due to extensive fossil fuel combustion is a major environmental concern. Furthermore, the energy crisis arises since the fossil fuel sources are drying up. A way to deal with these issues is to reduce the concentration of CO 2 by converting it to less harmful, but also valuable products, such as CH 4 that can be directly used as a fuel, or CO that is also an important chemical feedstock and an essential component of syngas, a mixture of H 2 and CO. Syngas can be further converted to liquid hydrocarbon fuels through well-known Fischer-Tropsch synthesis. Much attention is given to studying the artificial photosynthesis phenomenon, which means reduction of CO 2 with H 2 O in the presence of light and for that reaction a catalyst is required 1, 2 . When a photocatalyst absorbs photons whose energy is equal to or greater than the band gap of a catalyst, the charge carriers are generated. Electron from the valence band (VB) of a catalyst goes to the conductive band (CB) leaving behind a hole. The final step of the photocatalysis is the chemical reactions between surface species and charge carriers 3 . However, a high percentage of electron-hole pairs recombine. That depends on the rate of the separation and charge carrier transport. It was found that about 90% or more of the photogenerated electrons recombine within 10 ns 4 . Furthermore, not all the electrons reaching the surface can reduce CO 2 which is an inert and stable compound. The standard reduction potential of CO 2 to generate •CO 2 -is evaluated at E o red = -1.90 V vs the normal hydrogen electrode (NHE) (Table 1) 5, 6 . Even if most of publications on this topic propose

•CO 2 -as an intermediate 7, 8 , suggesting a monoelectronic activation of CO 2 , this activation potential is very high and never reached by any semiconducting material. Other teams suggest that •CO 2 -may not be an intermediate for CO 2 photoreduction pathway thus implying an activation by a bi-electronic mechanism and allowing to reach lower potential to carry the reduction semi-reaction 9, 10 . Nevertheless, in order to reduce CO 2 into carbon monoxide or hydrocarbons, electrons in the CB of a semiconductor photocatalyst are required to have more negative chemical potential than the redox potentials of the corresponding reactions, while for water oxidation, holes in the VB need to lie on more positive potential level than water oxidation level 4 . This means the choice of semiconductor photocatalyst relays primary on its band gap levels positions. From a thermodynamic point of view, formation of methane and methanol are more favorable in CO 2 reduction than carbon monoxide, formaldehyde and formic acid, since these reactions take place at lower potentials (Table 1). However, the kinetic drawback makes methane and methanol formation more difficult because more electrons are required for the former reactions. Notwithstanding, the prospective debate concerning one-or bi-electronic activation pathway for CO 2 , due to the complicated nature of the inorganic photocatalyst surface and no unreachable energetic barriers, the interaction between photocatalyst and absorbed species may undergo a series of one-electron/proton processes as long as the CO 2 activation has taken effect. Thus the actual redox potential required is determined by the reaction pathway. there is an excess of holes, the charge recombination will increase and the lifetime of electrons will be shortened. What is more, the holes can have negative impact on the photocatalytic reaction resulting in photocorrosion of the catalyst 12 . The way to deal this issue is to use an artificial electron donor to scavenge the holes, i.e. water that is considered as an ideal candidate, since it does not cause more CO 2 emission 11 . Furthermore, from both the thermodynamic and kinetic point of view, the reduction of water is more feasible than most of the CO 2 reduction reactions (Table 1). First, the reduction potential of water to hydrogen is more positive than the reduction potential of CO 2 reduction to CO, formic acid and formaldehyde and second, water reduction is just a 2-electrons process, unlike others which require 4-8 electrons or even more when C-C coupling. However, the concentrations of H 2 O need to be optimized since the excessively high H 2 O concentrations would occupy the catalyst active sites and consequently reduce the surface contact of CO 2 molecules with the photocatalyst 13 .

ARTICLE

Please do not adjust margins Please do not adjust margins Since N-doped TiO 2 absorbed more photons, it provided more electrons and holes for degradation of dye. We assumed that doping would have the same effect also on CO 2 reduction 14 . In addition to this, only a few studies explored gas phase CO 2 photoreduction with water involving N-doped TiO 2 15, 16 .

Moreover, N-doping induced formation of oxygen vacancies (V O ) 17 that serve as surface reactive sites (reacting with H 2 O, for example) 18 , and can also donate excess unpaired electrons to Ti atoms that are subsequently reduced into Ti 3+ . These stored electrons may strongly affect the surface chemistry and charge transport properties of TiO 2 3 . V O formed on the surface is considered to be electron-trapping centre that further inhibits the recombination process of the charges 19 . Di

Valentin et al. 20 applied density functional theory calculations and showed that the formation energy of V O in bulk anatase TiO 2 is drastically reduced in the presence of N-impurities, from 4.2 eV in pure TiO 2 to 0.6 eV in N-doped TiO 2 . They explained it as a 'compensation' effect, meaning that the electrons in vacancy related states near the bottom of the conduction band can be transferred to empty nitrogen impurity states near the top of the valence band. This leads to a considerable energy gain.

The aim of this work was to explore the reaction pathway of CO 2 reduction using N-doped TiO 2 and to set up a model that can help us to understand the difference in productivity when compared to the undoped TiO 2 , both synthetized and commercial P25.

Our goal was to follow the thermodynamics and to explain the reaction pathway, leading ultimately to H 2 , CH 4 , and CO products. Micro-kinetic modelling helped us understand the kinetics of the reactions, and enabled us to compare the differences in the rate of reactions and the rate-determining steps among different photocatalysts.

The paper is organized as follows. We first present the experimental details, explaining the synthesis procedure and photocatalysts characterization. In results and discussion section, we present the experimental setup for CO 2 reduction and the experimental results, followed by the micro-kinetic model. The modelling results are discussed and compared with the experimental outcomes, giving clear perspective on reaction steps rates. We conclude in the last section.

Experimental

Samples preparation

The following reactants of p.a. grade were used for the synthesis of TiO 2 : titanium tetrachloride (TiCl 4 ), titanium isopropoxide (Ti(O i Pr) 4 ) and cyclohexane (C 6 H 12 ). The gels were synthetized in a glove box, under a protective atmosphere of nitrogen by the following procedure 21, 22 :

10 cm 3 Ti(OC 3 H 7 ) 4 was mixed with 3.3 cm 3 of a solvent (cyclohexane), followed by adding of 3.7 cm 3 TiCl 4 , drop by drop. The obtained mixture was heated in an autoclave at 160 °C for 3 h in order to form gel. The sosynthetized gel was dried at 100 °C under nitrogen flow. The undoped TiO 2 powder was obtained by gel annealing in air at 500 °C for 3 h. To follow the effect of thermal treatment on N doping and consequently the photocatalytic activity in CO 2 reduction the two conditions were set: the gel that was directly annealed in ammonia (NH 3 ) flow (200 cm 3 /min) in a tube furnace, at 500 °C for 3 h, followed by natural cooling to room temperature under the NH 3 gas, was denoted as N-TiO 2 (G), and the one that was pre-annealed in air at 500 °C for 3 h and then in NH 3 atmosphere at the same conditions was denoted as N-TiO 2 (P). Both of the obtained samples were post-annealed in air at 400 °C for 30 min. For compassion with the synthetized samples, TiO 2 P25 Degussa from Aeroxide ® was used.

Samples characterization

The high resolution transmission electron microscopy (HRTEM) was used for the estimation of the nanoparticles size. The micrographs were obtained using a Jeol JEM-2010 operating at 200 kV. The average size of the nanoparticles was determined by a manual measurement of about thirty nanoparticles per sample.

The XRD analyses of the samples were done using a BRUKER D8 ADVANCE, with a Vario 1 focusing primary monochromator (Cu Kα1 radiation, λ = 1.54059 Å). The average crystallite size was calculated by using the Scherrer's equation like in the previous work 23 . The UV-Visible diffuse reflectance spectra (DRS) were recorded on a Shimadzu 2600 UV-Visible spectrometer with an integrating sphere attached and using BaSO 4 as a reference. The spectra were recorded in the wavelength range 300-600 nm. Using the Tauc plots of the transformed Kubelka-Munk function 23 , the band gap Please do not adjust margins Please do not adjust margins energy of the samples were determined. The Kubelka-Munk functions were calculated as F(R) = (1-R)/(2R), where R is reflectance in %. The absorbance data are also deduced from the DRS measurements.

Photocatalytic experiments

Photocatalytic experiments were carried out continuously in a stainless steel fixed bed reactor gas flow passing through the sample, operating at atmospheric pressure and ambient temperature. Max303 Asahi (equipped with a cut off filter for IR for wavelengths above 600 nm). Moreover, the reactor was fitted with an optical quartz window (S irradiation = 1.08.10 -3 m 2 ) through which the light flux reaches the sample.

Reagents and products were analysed online by a gas chromatograph TCD sensor micro-chromatograph Lan3000 (SRA Instruments, µGC-TCD). Gas phase analyses were performed every 10 minutes. The microchromatograph was calibrated for all gases from different gas cylinders with controlled contents purchased from Air Liquide. CO 2 was purchased from Air Liquide (purity > 99%). Water used for material synthesis and poured in the water tank for photocatalytic tests was demineralized water with a resistivity > 10 MΩ. A typical test duration is about 20 hours on stream.

Two parameters are calculated for each sample: average electron consumption rate -for measuring global activities as several products can be formed -(electron consumed vs time and mass of the photocatalyst), and the apparent effective quantum yield (the number of electrons that has been involved in product formation (i.e. 8 for CH 4 or 2 for H 2 ) divided by the number of absorbable photons regarding band gap values of the materials 24 ). The average electron consumption rate for each catalyst, in µmol.h -g -1 , and apparent effective quantum yield, Φ app.eff.. , in %, were calculated using the following equation ( 1) and ( 2), respectively:

   dt m V Q t i n S O T r S O T i catalyst C m tot e i e            . . . 0 25 60 . . . 1 (1) 3600 10 4 . .         n irradiatio eff catalyst e eff app S PDF m r (2)
Where n i e-is the stoichiometric coefficient of the electrons consumed by the product i (i.e. ), m catalyst is the mass of the catalyst (in g), and PFD eff is the effective Photon Flux Density (in mol.s -1 .m -2 , taken from 315 nm to λ max absorbable by the considered photocatalyst 24 ). The PFD eff was calculated on the basis on total lamp irradiance spectra (Fig. S1 in the supplementary file).

Prior to photocatalytic evaluation of materials, a test has been carried out with empty reactor without detection of any products aforementioned. Also, solids have been tested with argon replacing CO 2 gas, thus only water vapour is fed to the reactor, and no carbon containing products have been detected, only H 2 coming from water splitting. Average electron consumptions are given with a standard deviation of ± 10 % relative.

Experimental results served us to set a micro-kinetic model, applying a "top to bottom" approach. We started by making a wide reaction scheme of CO 2 reduction using TiO 2 catalyst. The reactions that led to formation of the products that were not obtained were not taken into account. Indeed, in our experiments we only observed gaseous products like H 2 , CO, CH 4 and C2/C3 alkanes. Therefore, we assume the reaction pathway follows the scheme given in the supplementary file as Fig. S2. Similar CO 2 reduction mechanisms have been proposed in the literature and studied from the theoretical point of view on different catalysts 25,26 . These studies confirmed that material composition and architecture play a central role in the reaction pathway energetics.

Results and discussion

Photocatalysis

The photocatalytic results are summarized in Table 2, together with the properties of each sample. Concerning the synthetized samples, the information about the phase composition and crystallite size from XRD were taken from our previously published work 23 , whereas the information about P25 were taken from the literature [27][28][29][30] . We also included the TEM measurements results, since we use those data for modelling calculations. The representative TEM micrographs are given in Fig. 1 a), whereas the particle size distribution of the three synthetized samples is provided in the supplementary file as Fig S3.

The E g values were deduced from the Tauc plots (Fig. 1 b)) using the DRS measurements. To demonstrate more clearly the absorbance red shift for the N doped samples, (iii) and (iv), compared to the undoped (ii) and the P25 TiO 2 (i),

ARTICLE

Please do not adjust margins Please do not adjust margins the absorption spectra are also provided in Fig. 1 c). The narrowing of the band gap and the absorbance red shift for the N doped samples is a result of the interstitial incorporation of nitrogen in TiO 2 structure, as it was concluded according to XPS analysis of the samples 23 .

The new inter band level is characteristic for interstitially incorporated N as it is the case here. The nitrogen incorporation was more efficient by the gel annealing in ammonia (N-TiO 2 (G)) than by the annealing of the sample that was pre-annealed in air (N-TiO 2 (P)). Therefore, narrowing of the band gap and absorbance red shift were higher for N-TiO 2 (G) than for N-TiO 2 (P).

Table 2 Properties of the investigated samples and the photocatalytic results.

Please do not adjust margins

Please do not adjust margins The apparent effective quantum yield values go in the following order: N-TiO 2 (P) > undoped TiO 2 > N-TiO 2 (G) > P25. We assume that the smallest apparent effective quantum yield for P25 is partly due to the phase composition. It is likely that the synthetized samples consisting of pure anatase phase is preferable for efficient photocatalytic reactions, compared to the mixed phase P25 powder. However, a mixture of anatase and rutile phases has been reported to be more active than either of the pure phases alone 31 . The high activity of the mixture of phases has been attributed to the separation of photoexcited charge carriers between the two phases. In this mixed-phase of TiO 2 , anatase is considered to be an active component, while rutile is conventionally considered to act as an electron sink because of the lower conduction band energy than that of anatase. However, there is still a controversy over the alignment of the conduction band minima of rutile and anatase phases but the experimental results suggest that the photoexcited electrons of rutile TiO 2 are less active than those of anatase 32 . Furthermore, by comparing the apparent effective quantum yield values for the synthetized samples, we can see that they are in accordance with the corresponding crystallite size, meaning that with smaller crystallites the recombination is higher, therefore smaller number of electrons are used for CO 2 reduction. Koci et al. reported that there is an optimal crystallite size of TiO 2 anatase photocatalyst for photoreduction of CO 2 . They showed that with increasing the crystallites size to 14 nm, the photoreduction of CO 2 is enhanced, but the production yield is smaller for the TiO 2 catalyst of the 29 nm sized crystallites 33 . Also in another work they showed that the photoreactivity of TiO 2 increases as the particle size is decreased from 26 nm to 18 nm 34 . This is in accordance with our results, since the average primary particle size of the P25 sample is 25 nm 27, 28 , whereas the crystallites of the synthetized samples are 15 nm and smaller.

As the average electron consumption rate is the direct measurement of light utilization efficiency, the reported results should be in accordance with the band gap values. However, the highest average electron consumption rate was achieved for N-doped TiO 2 (P) (Table 2), but N-doped TiO 2 (G) has a narrower band gap, meaning more electrons should be produced in the case of N-doped TiO 2 (G). Nevertheless, as already mentioned, the crystallite size affects the recombination, thus in the case of N-doped TiO 2 (P) the crystallite size is the optimal, and the recombination is the smallest leading to the highest average electron consumption rate.

ARTICLE

Please do not adjust margins

Please do not adjust margins Additional comparison of the catalysts performances was done by measuring the average production rate for each product (Fig. 2). The equation ( 1) was used for these calculations. For each product i, the production rate, , was calculated by removing stoichiometric coefficient from the equation and considering only product i concentration and not the sum of all, like for the global electron consumption. A table with the production rates is in the insert of the figure. By analyzing the results one can see that the main difference was that in the case of P25 there was no CH 4 detected, only CO and H 2 . This was also observed by Zhao et al. 35 where the main product of CO 2 reduction with H 2 O using P25 was CO. We assume that donation of electrons coming from water splitting is not efficient enough to produce excess of H for CH 4 formation. From the kinetic point of view, the formation of CO is favored since it requires only 2 electrons compared to the formation of CH 4 which needs 8 electrons (Table 1) 4, 35 .

Given that the undoped TiO 2 has higher average electron consumption rate than P25 (Table 2), more electrons are photogenerated and used for the production of H 2 , CO and CH 4 . For the undoped TiO 2 , the highest productivity rate was for H 2 , then CH 4 and CO.

On the other hand, for the N doped samples, among all detected products (H 2 , CH 4 , CO, C 2 H 6 and C 3 H 8 ), the CH 4 production rate was the largest.

We note that higher number of excited electrons, due to smaller band gaps of N doped samples, provide more H 2 and more CH 4 than in the case of the undoped TiO 2 . As seen from Fig. 2, for both N doped samples, the amount of H 2 produced is not higher than of CH 4 , which means that water splitting reaction is not dominant, like it was reported by Phongamwong et al. where the rate production of H 2 was two order of magnitude higher than of CH 4 18 . Furthermore, when comparing the two N doped samples with each other, the N-TiO 2 (P) provided higher amount of all detected products, even though it has a narrower bang gap than N-TiO 2 (G) and the amount of incorporated N is smaller 23 . The primary reason can be again the bigger crystallites of N-TiO 2 (P) than of N-TiO 2 (G) meaning less recombination of photogenerated electron-hole pairs happens and more efficient the light utilization is, which is in accordance with the results of average electron consumption rate (Table 1). On the other hand, semiconducting photocatalysts have been recently engineered to improved CO 2 conversion by creating intrinsic defects like oxygen vacancies, Vo 36 . It was shown that both the surface oxygen vacancies and bulk single electron trapped oxygen vacancies (SETOVs), formed by i.e. doping by nitrogen, contributed to the enhancement of the light absorption. However, while the surface vacancies facilitated the separation of the photogenerated charge carriers, the bulk SETOVs acted as the recombination center 37 . The photoreduction efficiency of CO 2 was improved with the increase of the ratio of surface oxygen vacancies to bulk SETOVs, but the concentration of both vacancies needed to be optimized: i) the bulk SETOVs, to provide visible-light absorption and minimize recombination, and ii) the surface Vo, to prevent the formation of defect-related recombination centers in the case of excessive Vo. In our case, we assume the N-TiO 2 (P) has lower concentration of both bulk Vo, due to lower N content, and surface Vo, due to bigger crystallites in comparison to N-TiO 2 (G), but the surface/bulk Vo ratio is obviously higher than for N-TiO 2 (G) and the productivity rate for all compounds is higher, especially for CH 4 .

The selectivity of the used catalysts, which is relatively poor, can be increase by optimization of processing parameters or by further modification of the catalysts. For example, the selectivity of CH 4 production can be increased if both H 2 and H 2 O are used as reductants and/or if low initial CO 2 concentrations is applied 38 .

Although CH 4 formation is thermodynamically more feasible than CO (Table 1), it requires more electrons and protons and thus is kinetically limited. The deposition of noble metal nanoparticles, for example Pt, can enrich the electron density that enhances the probability of multi-electron reaction, such is formation of CH 4 . In addition, strong adsorption of CO as an intermediate for CH 4 production on highly electron donating noble metal sites favours total reduction to CH 4 39 . What is more, noble metal nanoparticles are beneficial for the dissociation of H 2 O and production of the activated H species, that are necessary for CH 4 production 36 . However, the importance of choosing an adequate noble metal, tailoring the size of nanoparticles in appropriate range and avoiding the oxidation of noble metal to metal oxide, was indicated 40 .

Micro-kinetic modelling

We followed a "top to bottom" approach for the microkinetic modelling, meaning that the experimental data (Fig. 3) were used to fit the kinetic model of the photocatalytic reduction of CO 2 for four catalysts. Only the production of CH 4 , CO and H 2 were considered in the model, since the amounts of C 2 H 6 and C 3 H 8 when produced were assumed as negligible.

The reaction pathway follows the equations 3-12. The formation of reactive electron and hole at the surface by photon absorption is assumed non-limiting step, so it was not modelled. Also, it is considered that electrons and holes do not occupy a surface site, meaning that an active site carrying an electron or a hole still may adsorb chemical species. Thus, our reaction pathway consists of

    (3) * * 2 H 1 O H + O(g) H 2 k 2 O     (4) * * * * 2 4 3 2 1 2 k O + H + O H   (5) * * 2 * 1 2 2 + H H k   (6) * * * * * 1 2 3 + O H + CO H + CO 2 k 2   (7) * * 2 * * * 6 2 8 4 + O H + CH H + CO 4 k 2   (8) * * 1 CO + CO(g) CO k     (9) * 2 * 2 2 1 O + (g) O O k     (10) * 2 * 2 2 1 H + (g) H H k     (11) * * 4 1 4 k 4 CH + (g) CH CH     (12)
where * indicates the empty active surface site 42 . Firstly, all reactions were assumed reversible, but due to strong parameter correlations and large span of possible parameter values, we later assumed all surface reactions (apart from adsorption/desorption reactions) to be forward direction only, as indicated by arrows.

The micro-kinetic model assumes all reactions are first order 38 . For gaseous reactions (Eqs. 3, 4, 9-12), the adsorption contribution for gaseous species to the rate equation is modelled as (13) where k G fwd is the fitting parameter for the forward rate coefficient of gaseous species (Eqs. 3, 4, 9-12), θ empty is the surface coverage of empty sites, G is the gas fraction, R gas is the gas constant and T is the temperature. The desorption contribution to rate for gaseous species is simply

T GR θ k = r gas empty G fwd G ads
gas G rev G des θ k = r (14)
where k G rev is the fitting parameter for the reverse rate coefficient of gaseous species, and θ gas is the gas surface coverage. Altogether, the 6 differential equations to describe temporal evolution of gaseous species are given in the form of (for each G = CO 2 , H 2 O, O 2 , H 2 , CO, and CH 4 ):

r g in G des G ads V V G G f r r t G         ) ( ) ( (15)
where f represents the factor of available surface sites for gas interaction, G in is the gas inflow (in our case, only CO 2 , H 2 O and small amount of O 2 gases were inflow gases), V g = 0.0003066 l min -1 is the gas volumetric flow throughout the reactor, and V r = 0.01542 l is the reactor volume. The factor f is defined as:

 cat V N f  (16)
where N is the number of active surface sites in mol (different among different catalyst samples), V cat = 0.0005 l is the volume of the catalyst, and ε is the void fraction within the catalytic bed, which we assumed ε = 0.9241 (calculated by the mass and volume of the sample compared to the bulk density of the catalyst). For surface reactions, the differential equations to describe temporal evolution of surface coverage with each intermediate are modelled as:

H CO H CO CO des CO ads CO k k r r t      2 2 2 2 2 4 3       (17) H CO H CO empty O H O H des O H ads O H k k k r r t        2 2 2 2 2 2 4 3 1 2        (18) empty O H O des O ads O k r r t    2 2 2 2 1      (19) H H des H ads H k r r t   2 2 2 2      (20) H CO CO des CO ads CO k r r t    2 3      (21) H CO CH des CH ads CH k r r t    2 4 4 4 4      (22) H CO H CO H empty O H H k k k k t         2 2 2 4 3 2 1 8 2 2 4       (23)           G H CO H CO H empty O H G des G ads empty k k k k r r t         2 2 2 4 3 2 1 6 3 ) ( (24)
Please do not adjust margins Please do not adjust margins where only the forward direction of reactions were considered to simplify the scheme, otherwise the terms for reverse reactions should also be added to the above equations. Also, all reactions were assumed as the first order reactions, so all θ enter the terms only once. Stoichiometry is taken into account.

Solving the model. Overall, the system of 14 ordinary differential equations was solved using Scipy odeint package (Python), which is based on lsoda from the FORTRAN library odepack. The system is fitted with Levenberg-Marquardt algorithm using lmfit package (Python). Initially, 17 free parameters were considered in the system, but this was later reduced to only 4 free parameters when fixing certain parameters, as follows. First, for gaseous species adsorption and desorption, the non-activated adsorption (E A ≈ 0) was assumed, meaning that the rate for the elementary reaction based on the Arrhenius equation is only the pre-exponential factor. Considering this, it was further assumed that the pre-exponential factor for gas adsorption can be approximated with:

T k m A P k B G G fwd  2   ( 25
)
where P is the gas pressure (assuming 1 bar), A is the effective area of the reaction site (calculated to be of the order of A ≈ 0.1 nm 2 for TiO 2 ), and m G is the mass of the gas molecule 43 . It should be noted that the contribution of partition functions (vibrational, rotational and translational) was neglected. Similarly, for the gas desorption, the pre-exponential factor and thus the rate can be approximated with: (26) where h is the Planck's constant. At room temperature (300 K), the above equations give the estimates of k G fwd of the order of 10 8 s -1 to 10 9 s -1 (depending on the gas), while k G rev ≈ 6 •10 12 s -1 . To simplify the fitting procedure, the rates for adsorption and desorption were fixed. In principle they could have been left as free parameters of the fit, accounting also for the contribution of the partition functions, but the testing showed that the overall results do not change significantly if they were left fixed. Thus, the k G fwd and k G rev in Eqs. 13 and 14 were fixed to the estimated values.

h T k = k B G rev
To solve Eqs. 15 and 17-24, the initial values have to be assumed. The system was tested by decoupling the reactions to obtain the initial rate assessment. All G 0 and θ 0 were assumed to be 0 except G 0 CO2 and G 0 H2O for which the initial fractions were known (97% for CO 2 and 3% for H 2 O). For the initial value of empty sites (θ empty,0 ),

Please do not adjust margins

Please do not adjust margins we used the experimentally determined value for each catalyst, calculated using the following expression:

The values for the nanoparticles size, d particle , were taken from Table 2 using TEM measurements. The V TiO2 , which is the volume of a TiO 2 cluster, was calculated to be 0.034205 nm 3 and the Section TiO2 , which is the section of a TiO 2 cluster, was calculated to be 0.127394 nm 2 and they were the same for all the catalysts. These calculations were made by using crystallographic data for anatase TiO 2 given in Material Studio 7.0 software by Accelerys ® .

Thus, the initial values of empty sites (θ empty,0 ) were 4.50 x 10 -4 mol for N-TiO 2 (P), 6.45 x 10 -4 mol for N-TiO 2 (G), 8.86 x 10 -4 mol for undoped TiO 2 , and 9.27 x 10 -4 mol for the reference P25, and they were left fixed during the fitting procedure.

All other values of the fit, which include k 1 -k 4 surface reaction rates, were left as free parameters. We used the initial value of 10 13 s -1 for the surface reaction rates 42 .

Modelling results and discussion. The output of the catalytic CO 2 conversion to CH 4 from 4 different catalysts: N-TiO 2 (G), N-TiO 2 (P), undoped TiO 2 and the reference P25 catalyst was fitted. Table 3 shows the results of the fit, where k 1 is the kinetic constant for water splitting, whereas k 2 , k 3 and k 4 are for H 2 , CO and CH 4 production, respectively (equations (5-8)). Fig. 4 presents the fitted results together with the experimental data points. Note that k 3 and k 4 parameters are many orders of magnitude larger than other parameters, and thus do not vary much with respect to other parameters. Also, when assessing the initial rate, we noticed that if the reaction is fast, i.e., the activation energy is close to 0, then the surface reaction rate can be approximated by k B T/h, which is again of the order of 10 13 s -1 at room temperature. It can be noticed that for the P25 sample, the k 4 was left fixed at 0, since CH 4 production was negligible. The obtained k rates cannot be used directly to extract the activation energy, since pre-exponential factors were approximated. In principle, this could be done via inverse Arrhenius law, but since we do not know precisely the pre-exponential factor (neglecting the contribution of partition functions), and even more importantly, since treating all surface reactions as first order, the rates can be absolutely over-or underdetermined by many orders of magnitude. Nevertheless, the relative comparison is still valid, since all four samples were fitted with the same model.

We note that the trend of measurement does not necessary follow the fitted curve over total time span of the experiment (70,000 seconds). This is particularly visible on the CO yield for N-TiO 2 (P) or H 2 yield for N-TiO 2 (G). This can be due to the fact that on such a long time-scale, the catalyst can have episodes of activation and deactivation, the latter especially if the number of active sites change during the course of reactions due to catalyst surface becoming poisoned with intermediates. This was not taken into account in our model, so all the fitted curves are steady after the initial rise.

is beneficial for this photocatalytic application, unlike the mixed phase of the P25 catalyst.

To our knowledge, this work constitutes the first attempt for micro-kinetic approach of the photocatalytic reduction of CO 2 with H 2 O. In the previous studies about kinetics of CO 2 reduction using TiO 2 nanoparticles the Langmuir-Hinshelwood model was used for rate determination of the adsorption of the reactants and desorption of products over the catalyst surface. Also the difference to our work was that the CO 2 reduction was with CH 4 and N 2 in the feed 44 . Another study, where the Langmuir-Hinshelwood model was used for the adsorption-desorption rates, illustrated the effect of indium loading on TiO 2 photocatalytic activity for transformation of CO 2 with H 2 O to CO and CH 4 45 .

However, there is no study about micro-kinetics providing the insight of rate determination step.

Conclusions

In this study, TiO 2 nanoparticular photocatalysts were synthetized using a non-hydrolytic sol-gel method, where the gel and undoped powder were annealed in NH 3 atmosphere to induce the doping with N. The photocatalytic activity of the samples was tested for CO 2 reduction and they were compared to the results of TiO 2 P25 commercial powder. This study showed that the synthetized samples were more effective compared to the TiO 2 P25, most probably due to smaller band gaps and consequently higher photon absorption of the synthetized samples than of P25. The additional reason may be the optimal crystallite sizes of the synthetized samples and also due to phase composition of pure anatase, unlike the mixed anatase-rutile phase of P25.

The most important difference comparing to P25 was that P25 catalyst did not produce CH 4 like the synthetized samples. The N-doping caused higher productivity rate of H 2 , CH 4 and CO compared to the undoped TiO 2 . Doping of the TiO 2 powder provided better performances of the catalyst compared to the Ndoped TiO 2 obtained by annealing of gel in NH 3 , even though the catalyst obtained by N doping of TiO 2 powder has higher band gap, meaning it absorbed less photons and thus might produce less active electrons and holes. However, the larger crystallite size provided lower recombination of charge carriers and thus higher electron consumption rate.

The experimental data were used to fit the micro-kinetic modelling parameters. The obtained model provided the insight into the reaction pathway and kinetics. We established that the water dissociation to H radical and O 2 is the slowest reaction, meaning it is the ratedetermining step. The kinetic constant for H 2 O dissociation is the highest for N-doped TiO 2 from powder and the lowest for P25. The kinetic constants for the formation of H 2 from H radicals are of high value and similar for all the catalysts meaning that H 2 formation does not depend of the catalyst. The rates that account for the formation of CO and CH 4 from adsorbed CO 2 and H radicals are even higher meaning that those reaction steps are the quickest. To our knowledge, this work constitutes the first attempt for micro-kinetic approach of the photocatalytic reduction of CO 2 with H 2 O in the gas phase. The model developed here could be useful to deduce the rate and product yield at any particular reaction time and could aid as a tool for future optimization study.

Fig. 1 a

 1 Fig. 1 a) TEM micrographs, b) Tauc plots and c) adsorption spectra of the investigated samples: i) P25 TiO 2 , ii) undoped TiO 2 , iii) N-TiO 2 (G) and iv) N-TiO 2 (P).

Fig. 2

 2 Fig. 2 Average production rates of each product for all catalysts.

Fig. 3

 3 Fig. 3 Rate production over 20 hours on stream using: a) P25 TiO 2 , b) undoped TiO 2 , c) N-TiO 2 (G) and d) N-TiO 2 (P) catalysts.

  TiO2 corresponds to TiO 2 dispersion considering the spherical model and expressed by:

11 Table 1 .

 111 Chemical reactions of CO 2 reduction and corresponding thermodynamic potential vs NHE

	Chemical Equation	Thermodynamic
					potential, V
	CO 2 (g) + e	-→ CO 2	-	-1.90
	2H + + 2e	-→ H 2(g)	0.00
	CO 2(g) + 2H + + 2e	-→ HCOOH (g)	-0.114
	CO 2(g) + 4H + + 4e	-→ HCHO (g) +H 2 O (g)	-0.072
	CO 2(g) + 2H + + 2e	-→ CO (g) +H 2 O (g)	-0.104
	CO 2(g) + 6H + + 6e	-→ CH 3 OH (g) + H 2 O (g) +0.031
	CO 2(g) + 8H + + 8e	-→ CH 4(g) + H 2 O (g)	+0.169
	Reduction of CO 2 by photogenerated electrons, should
	produce the equal number of photogenerated holes. If

  Among many proposed semiconductors that are applied for CO 2 reduction, i.e. ZnO, CdS, TaNO, WO 3 , GaP, SiC, ZrO 2 , SnO 2 , Ga 2 O 3 , BiWO 3 , CdS, ZnS, GaN, SiC etc. TiO 2 is widely used due to its non-toxicity, wide abundance, low price, chemical/thermal stability and strong oxidative potential. However, the photocatalytic performances, such as rate production (number of mol of product in time vs mass of used catalyst) of TiO 2 are limited in particular by the fact that TiO 2 absorbs only UV part of the solar spectrum. Therefore, it was necessary to shift the absorption to visible range by i.e. doping with N. Our previous study showed that N-doped TiO 2 nanoparticles are able to degrade organic dye upon visible light illumination more efficiently than the undoped TiO 2 .

  Typical sample masses involved are between 100 and 300 mg. CO 2 :H 2 O molar ratio (obtained by CO 2 bubbling in water tank) was fixed at 30:1 and CO 2 gas flow was fixed at 0.3 cc.min -1 , resulting in the total gas flow rate (CO 2 + H 2 O)

	of 0.31 cc.min	-1 . Irradiance for 315-600 nm wavelength
	range was adjusted to 380 W.m	-2 and controlled with a
	radiometer equipped with CCD captor (Delta Ohm),
	corresponding to an overall Photon Flux Density (PFD) of
	1.52.10	-3 mol. s	-1 .m

-2

, and provided by a Xe lamp

Table 3

 3 Results for reaction rate coefficients for all four catalysts, as obtained from micro-kinetic modelling.

	The rates are

Acknowledgements

The Serbian authors acknowledge with thanks the financial support of the Ministry of Education, Science and Technological Development, Republic of Serbia through the Projects III 45019. A. B. is grateful to EIT Climate KIC to participate in the program "Pioneers into Practice 2018". D. K. and B. L. acknowledge the funding from the Slovenian Research Agency (ARRS) through Core Grant P2-0152 and Project J2-7319.

Notes and references

ARTICLE

Please do not adjust margins Please do not adjust margins Comparing the obtained reaction rates, we notice that for N-TiO 2 (P) catalyst, the rate k 1 for H 2 O dissociation to H* and O 2 is the highest, but it is the lowest for P25. The kinetic constants for the second reaction k 2 are similar and of high value for all the catalysts meaning that formation of H 2 from H* does not depend on the catalyst. The k 3 and k 4 rates that account for the formation of CO and CH 4 , respectively, are the highest for N-TiO 2 (P), which is reflected also in experiments, since this catalyst produces the most CH 4 and also CO. We also note that both k 3 and k 4 rates are much higher than k 1 and k 2 , meaning that those reaction steps are the quickest (fast reactions). The rate-determining step as obtained from the model is thus the water dissociation (k 1 ), which is the slowest reaction, confirming the experimental results.

The k 1 values follow the same order as the apparent effective quantum yield from the experiment (Table 3): N-TiO 2 (P) > undoped TiO 2 > N-TiO 2 (G) > P25. Since the apparent effective quantum yield shows the effectiveness of the absorbable photons to produce electrons for the product formation, it is not surprising the trend is the same. The water dissociation depends greatly of the amount of absorbable photons but also is effected by the materials composition, i.e. the recombination of the electron-hole pairs is higher in the case of smaller crystallites. Since for water dissociation to oxygen and H radicals a hole is needed, higher the recombination is, slower the water dissociation is 33 .

However, in the case of P25, the crystallites size (25 nm) is above the optimal (14 nm) and it leads to smaller photoreactivity. We also commented that pure anatase

Conflicts of interest

There are no conflicts to declare.