Zhaoyang Pang

Callum Biggs O'may

Bhavin Choksi

Rufin Vanrullen
email: rufin.vanrullen@cnrs.fr

Predictive coding feedback results in perceived illusory contours in a recurrent neural network

Keywords: illusory contours, predictive coding, deep learning, Kanizsa squares, feedback, generative models

Modern feedforward convolutional neural networks (CNNs) can now solve some computer vision tasks at super-human levels. However, these networks only roughly mimic human visual perception. One difference from human vision is that they do not appear to perceive illusory contours (e.g. Kanizsa squares) in the same way humans do. Physiological evidence from visual cortex suggests that the perception of illusory contours could involve feedback connections. Would recurrent feedback neural networks perceive illusory contours like humans? In this work we equip a deep feedforward convolutional network with brain-inspired recurrent dynamics. The network was first pretrained with an unsupervised reconstruction objective on a natural image dataset, to expose it to natural object contour statistics. Then, a classification decision layer was added and the model was finetuned on a form discrimination task: squares vs. randomly oriented inducer shapes (no illusory contour). Finally, the model was tested with the unfamiliar "illusory contour" configuration: inducer shapes oriented to form an illusory square. Compared with feedforward baselines, the iterative "predictive coding" feedback resulted in more illusory contours being classified as physical squares. The perception of the illusory contour was measurable in the luminance profile of the image reconstructions produced by the model, demonstrating that the model really "sees" the illusion. Ablation studies revealed that natural image pretraining and feedback error correction are both critical to the perception of the illusion. Finally we validated our conclusions in a deeper network (VGG): adding the same predictive coding feedback dynamics again leads to the perception of illusory contours.

Introduction

The human visual system is remarkably versatile. It is capable of accurately recognizing objects from different angles, over different distances, under distortion, and even under occlusion. Despite this, Manuscript under review it can easily be fooled by simple visual illusions. Figure 1 shows an example of Kanizsa illusory contour (or subjective contour) [START_REF] Kanizsa | Margini quasi-percettivi in campi con stimolazione omogenea[END_REF][START_REF] Kanizsa | Subjective contours[END_REF]. When observing such an image, most humans perceive the presence of a square, despite the only shapes present being the pacman-shaped inducers. In fact, in addition to perceiving edges of the illusory square, humans tend to perceive the interior of the induced shape as being brighter than the exterior, despite their being the same. These two phenomena illustrate two salient features of illusory figures -sharp illusory edges in regions of homogeneous luminance, and a brightness enhancement in the figure [START_REF] Schumann | Beiträge zur analyse der gesichtswahrnehmungen[END_REF][START_REF] Spillmann | Phenomena of illusory form: Can we bridge the gap between levels of explanation?[END_REF][START_REF] Parks | Rock's cognitive theory of illusory figures: a commentary[END_REF]. There are many examples of illusions like this, demonstrating that the human visual system does not always accurately perceive stimuli. Such systematic misperceptions reflect underlying neural constraints and provide insight into the complex structure of visual cortex [START_REF] Changizi | Perceiving the present and a systematization of illusions[END_REF]. Visual illusions have thus been used as a probe for understanding visual processing [START_REF] Eagleman | Visual illusions and neurobiology[END_REF][START_REF] Gori | Visual illusions: An interesting tool to investigate developmental dyslexia and autism spectrum disorder[END_REF]. Alongside traditional neuroscientific approaches to studying visual perception, advances in machine learning have led to new avenues in understanding the mechanisms of visual processing. By developing artificial models of human vision we can investigate neuroscientific principles and reduce the need for experimentation. In particular, computational models allow for rapid, iterative experimentation and development. In fact, the origin of modern artificial neural networks (the backbone of much modern artificial intelligence or AI) is in computational models of neurons [START_REF] Mcculloch | A logical calculus of the ideas immanent in nervous activity[END_REF].

Throughout the history of AI development, researchers have taken inspiration from the brain. A clear example of this is the development of convolutional neural networks (CNNs), which were a critical step in the computer vision revolution [START_REF] Fukushima | Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition[END_REF][START_REF] Lecun | Backpropagation applied to handwritten zip code recognition[END_REF]. Inspired by the hierarchical structure of the brain, CNNs limit the spatial extent of the neuronal receptive fields, resulting in retinotopic feature maps much like the early visual system. In the last decade, the explosion of deep learning research has led to massive improvements in object recognition, even reaching super-human performance [START_REF] He | Delving deep into rectifiers: Surpassing human-level performance on imagenet classification[END_REF]. However, the rapidly changing research landscape has produced many technical developments which are not always directly compatible with neuroscience. As a result, the relationship between human vision and computer vision has become less clear. A striking example of this is learning through error backpropagation, which uses a biologically implausible global error signal [START_REF] Lillicrap | Backpropagation and the brain[END_REF]. Recent attempts have been made to identify more biologically plausible learning rules [START_REF] Whittington | An approximation of the error backpropagation algorithm in a predictive coding network with local hebbian synaptic plasticity[END_REF][START_REF] Millidge | Predictive coding approximates backprop along arbitrary computation graphs[END_REF][START_REF] Ahmad | Gait-prop: A biologically plausible learning rule derived from backpropagation of error[END_REF][START_REF] Lee | Difference target propagation[END_REF]. Many studies have also sought to investigate the similarities and differences between modern computer vision and human vision [START_REF] Khaligh-Razavi | Deep supervised, but not unsupervised, models may explain it cortical representation[END_REF][START_REF] Yamins | Performanceoptimized hierarchical models predict neural responses in higher visual cortex[END_REF][START_REF] Cichy | Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence[END_REF][START_REF] Wen | Neural encoding and decoding with deep learning for dynamic natural vision[END_REF]. Such work highlights the dual benefits of computational neuroscience, which can both provide guidance for designing computer vision algorithms, and contribute to our understanding of brain functioning. Despite all this, a recent report [START_REF] Baker | Deep convolutional networks do not perceive illusory contours[END_REF] shows that convolutional neural networks do not perceive illusory contours in the same way humans do. In their study, a feedforward CNN -AlexNet [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF] -was trained to perform a thin/fat contour discrimination task on real and illusory contours. In both cases, the network could correctly classify the images, but the authors demonstrated that the representations of the illusory contours in the CNN do not resemble those of human observers. Specifically, they employed a so-called classification image technique [START_REF] Gold | Deriving behavioural receptive fields for visually completed contours[END_REF] to give insight into which regions in a given image are important for the illusory formation. For human subjects, the region between inducers is critical to the perceptual decision [START_REF] Gold | Deriving behavioural receptive fields for visually completed contours[END_REF]. The feedforward neural network, on the other hand, failed to interpolate illusory contours between inducers -instead, it relied on the orientation of inducing elements to make its decisions [START_REF] Baker | Deep convolutional networks do not perceive illusory contours[END_REF]. That is, instead of adopting a global processing strategy as humans did, the CNN mainly appeared to rely on local features. In a related study, [START_REF] Kim | Neural networks trained on natural scenes exhibit gestalt closure[END_REF] directly examined the representations in intermediate layers of a feedforward neural network. The authors computed the cosine similarity between the network's representations of physical and illusory contours, and showed that they are more similar to each other than to control non-illusory shapes. However, this representation similarity measure is only an indirect indication of the perception of illusory contours, since it does not explain which features are similar between the illusory contours and the physical contours. It could be, for example, that the similarity is related to the presence of correctly-oriented corners, rather than to the contours between them. Thus, it may still be true that feedforward CNNs do not perceive illusory contours, but instead largely base their decisions on local features. Other evidence shows that feedforward CNNs can perform poorly on tasks which depend explicitly on global processing, like long-range spatial dependencies [START_REF] Linsley | Learning long-range spatial dependencies with horizontal gated recurrent units[END_REF] or object recognition under occlusion [START_REF] Spoerer | Recurrent convolutional neural networks: A better model of biological object recognition[END_REF]. In summary, it would seem that the visual perception of feedforward artificial neural networks is more dependent on local processing than human perception.

Perceptual discrepancies between artificial and biological networks highlight fundamental differences in their underlying structures as well as performed computations. Feedforward CNNs only roughly mimic the visual system, and particularly the feedforward pass through the visual pathway, while feedback connections, although abundant in the brain, are often ignored. Notably, physiological evidence shows that illusory contour perception may rely on feedback connections from V2 to V1 [START_REF] Lee | Dynamics of subjective contour formation in the early visual cortex[END_REF][START_REF] Pak | Top-down feedback controls the cortical representation of illusory contours in mouse primary visual cortex[END_REF] or in higher-level regions [START_REF] Cox | Receptive field focus of visual area v4 neurons determines responses to illusory surfaces[END_REF][START_REF] Pan | Equivalent representation of real and illusory contours in macaque V4[END_REF]. In this respect, the generation of illusory contours could be interpreted within the predictive coding framework [START_REF] Notredame | What visual illusions teach us about schizophrenia[END_REF][START_REF] Raman | Predictive coding: a possible explanation of filling-in at the blind spot[END_REF], a popular computational theory of feedback processing introduced in neuroscience by [START_REF] Rao | Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects[END_REF]. This theory posits that, in a hierarchical system, each layer tries to predict the activity of the layer below, and the prediction errors are used to update the activations. In fact, predictive coding could serve as a unified computational principle for a variety of sensory systems including auditory [START_REF] Kumar | Predictive coding and pitch processing in the auditory cortex[END_REF], olfactory [START_REF] Zelano | Olfactory predictive codes and stimulus templates in piriform cortex[END_REF] and visual sensation [START_REF] Mumford | On the computational architecture of the neocortex[END_REF][START_REF] Nour | Perception, illusions and bayesian inference[END_REF]. In the field of machine learning, a number of recent works have also applied the predictive coding framework to modern deep learning networks. [START_REF] Boutin | Sparse deep predictive coding captures contour integration capabilities of the early visual system[END_REF] demonstrated that a predictive coding network with a sparsity constraint learns similar receptive fields to neurons in V1 and V2, and provided evidence that the feedback helps the network perform contour integration. [START_REF] Lotter | Deep predictive coding networks for video prediction and unsupervised learning[END_REF] showed that the predictive coding framework can be used to learn representations of pose and motion in video streams. Importantly, their model was only trained with unsupervised objectives, arguably closer to human learning than standard supervised learning methods. In addition to the biological plausibility of predictive coding networks, other works have demonstrated that these networks can show improved robustness to random or adversarial noise [START_REF] Choksi | Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics[END_REF][START_REF] Huang | Neural networks with recurrent generative feedback[END_REF][START_REF] Chalasani | Deep predictive coding networks[END_REF]. Wen et al. (2018a) studied whether predictive coding can result in improved classification of clean images, but since their network was not trained to minimize reconstruction error, it did not fulfill the predictive coding objective as described by [START_REF] Rao | Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects[END_REF] (reduction of reconstruction errors over timesteps). A further discussion of the limits of the work of Wen et al. (2018a) can be found in [START_REF] Choksi | Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics[END_REF].

Given both the significant role of feedback connections in biological illusory contour perception, and the potential for predictive coding as neuroscientific inspiration for feedback architectures in deep learning models, we hypothesized that a feedback neural network implementing predictive coding recurrent dynamics may perceive illusory contours in the same way humans do. The recent work of [START_REF] Lotter | A neural network trained to predict future video frames mimics critical properties of biological neuronal responses and perception[END_REF] strengthens this hypothesis. They investigated the responses of PredNet [START_REF] Lotter | Deep predictive coding networks for video prediction and unsupervised learning[END_REF], a predictive coding network, to illusory contours. They compared the response properties of model units to neuronal recordings in the primate visual cortex and showed comparable response dynamics in the presence of illusory contours [START_REF] Lotter | A neural network trained to predict future video frames mimics critical properties of biological neuronal responses and perception[END_REF]. Unlike the current work, which inspects the network's behavioural and perceptual responses, the [START_REF] Lotter | A neural network trained to predict future video frames mimics critical properties of biological neuronal responses and perception[END_REF] work approached the topic entirely at the level of neuronal activations. Although their research supports the idea of a similarity in contour representations between the PredNet artificial layers and the biological cortex, they do not read off network decisions, or inspect reconstructions of the network. Thus, they do not argue that the network is human-like on a behavioural or perceptual level. As with the above-mentioned [START_REF] Kim | Neural networks trained on natural scenes exhibit gestalt closure[END_REF] paper, it could be that the similarities in the representations are due to local low-level features (e.g. the inward-facing corners) rather than the illusory contours. Thus, it remains unclear whether their model truly "perceives" illusory shapes like humans do.

In the current study, we designed a deep predictive coding neural network according to the algorithm previously devised by our group [START_REF] Choksi | Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics[END_REF]. We used a relatively small network, consisting of three stacked autoencoders, intended to roughly mimic the hierarchical structure of the early visual cortex in the primate brain, since neural correlates of illusory contours have been found to involve visual areas V1, V2 and V4 [START_REF] Der Heydt | Illusory contours and cortical neuron responses[END_REF][START_REF] Grosof | Macaque VI neurons can signal 'illusory' contours[END_REF][START_REF] Pak | Top-down feedback controls the cortical representation of illusory contours in mouse primary visual cortex[END_REF][START_REF] Pan | Equivalent representation of real and illusory contours in macaque V4[END_REF][START_REF] Cox | Receptive field focus of visual area v4 neurons determines responses to illusory surfaces[END_REF]. The network had both generative (image reconstruction) and discriminative (image classification) capabilities. Therefore, we could not only directly check whether the network indeed "sees" illusory contours, by examining its reconstruction images from generative feedback connections, but also measure the network's "behavioral" performance, i.e. the discriminative readout values indicating whether or not it detects an illusory shape. In addition, in order to equip our model with sufficient contour knowledge, we propose and verify that it is necessary to pretrain the neural network on natural images; this is in line with evidence that human contour integration and grouping is strongly tied to the statistics of the natural world [START_REF] Geisler | Contour statistics in natural images: Grouping across occlusions[END_REF]. Finally, we extend this approach to a deeper feedforward network (VGG) and demonstrate that this network sees the illusory contours too. For the main body, the predictive coding strategy is implemented in stacked autoencoders, with three feedforward encoding layers (en) and three generative feedback decoding layers (dn). Reconstruction errors ( n) are computed and used for the proposed predictive coding updates which are denoted by "P.C." loops. Dense layers are added on top of the structure to implement a binary classification task.

In summary, we report that our predictive coding feedback networks tend to process illusory contours in a similar way to humans.

Materials and Methods

Architecture

We construct a three-layer hierarchical stacked autoencoder with 3 feedforward encoding layers e n (n ∈ 1, 2, 3) and 3 corresponding feedback decoding layers d n-1 (see Figure 2 and Table 1). This is a conventional network architecture for learning a lower-dimensional latent representation of a high-dimensional input space [START_REF] Kingma | Auto-encoding variational bayes[END_REF]. When considering only the encoding layers, the network can be viewed as a standard feedforward convolutional neural network. To guide the implementation of the feedback connections, we follow the principles of "predictive coding" as introduced by [START_REF] Rao | Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects[END_REF]: in the hierarchical network, the higher layers try to predict the activity of the lower layers and the errors made in this prediction are then used to update their activity. For a given input image, we initiate the activations of all encoding layers with a feedforward pass. Then over successive recurrent iterations (referred to as timesteps t) we update the decoding and encoding layer representations using the following equations:

d n (t) = W b n+1,n e n+1 (t) + e n (t + 1) = β W f n-1,n e n-1 (t + 1) + + λd n (t) + (1 -β -λ)e n (t) -α∇ n-1 (t) (1)
where W f n-1,n denotes the feedforward weights connecting layer n -1 to layer n, and W b n+1,n denotes the feedback weights from layer n + 1 to n. squared error between the representation e n-1 and the corresponding prediction d n-1 :

n-1 (t) = e n-1 (t) -d n-1 (t) 2 2 (2)
Then ∇ n-1 (t) denotes the gradient of the error at layer n -1 with respect to the activations in layer e n . That is, ∇ n-1 (t) is the vector of partial derivatives of the error with respect to each element of e n (and thus has the shape of e n ), so that the ith element of ∇ n-1 (t) will be ∂ n-1 (t) ∂en(t)i . We use the PyTorch automatic differentiation package to calculate this one-step gradient. The parameters β, λ and α act as balancing coefficients for the feedforward drive, feedback error correction, and feedforward error correction terms respectively, and they are treated as hyperparameters of the network (for the present experiments, except where otherwise noted, these hyperparameter values were fixed to β = 0.2, λ = 0.1 and α = 0.1 as this was found to be sufficient for producing the illusion). We can also rewrite Equation 1 by grouping the terms for each hyperparameter, to more clearly illustrate the connection with the Rao and Ballard formulation of predictive coding:

e n (t + 1) = β W f n-1,n e n-1 (t + 1) + + (1 -β)e n (t) + λ(d n (t) -e n (t)) -α∇ n-1 (t) (3)
This clarifies that the λ hyperparameter is the weight for the term d n (t) -e n (t), which is exactly the gradient of n (t) with respect to e n (t). Thus we see that the λ and α terms are exactly the two error terms in Rao and Ballard's formulation. A fuller demonstration of the relationship between the two formulations (under the simplifying assumption of a linear activation function) can be found in the Supplementary Material.

Since the error n-1 is an average over the whole representation in the layer below (see Eq. 4), as the number of units in the representation increases, the error term variance will tend to shrink to 0.

Additionally, the retinotopic nature of the convolution operation (with connectivity restricted to a local neighborhood) means that most connections between the layers are 0, and thus have 0 gradient. These combined effects result in small gradients relative to the layer's activations, so to counteract this we re-scale the gradient terms according to the layer and kernel sizes. Specifically, for each layer, we calculate for the layer below K=channels × width × height and C=channels × kernel size, and then multiplicatively scale the gradient term by a factor of K/ √ C. This directly balances out the two effects discussed. See the Supplementary Material for full details.

To reflect the systematic comparison between decoding and encoding layers, we set e 0 as our input images. Thus, our updating rule in Eq. 1 is only applied to e n (n ∈ 1, 2, 3) and e 0 will remain constant over timesteps. In addition, for the last layer e 3 in our model, there is no feedback, so we ignore the corresponding term in the update equation. All the weights W are fixed during the updates defined by Eq. 1. They are optimized over successive batches of natural images and across all timesteps (see Training procedure) to minimize the total reconstruction error L (Eq. 4)-an unsupervised objective in accordance with the principles of the predictive coding theory. In Equation 4, N is the number of layers (here, N = 3). We note that Equation 1 leads to updates which approximately reduce the loss in Equation 4 over timesteps, as in both Rao and Ballard's formulation and [START_REF] Whittington | An approximation of the error backpropagation algorithm in a predictive coding network with local hebbian synaptic plasticity[END_REF].

L = t N -1 n=0 n (t) = t N -1 n=0 e n (t) -d n (t) 2 2 (4)
Intuitively, each of the four terms in Equation 1 contributes different signals to a layer: (i) the feedforward term (controlled by parameter β) provides information about the (constant) input and changing representations in the lower layers, (ii) the feedback error correction term (parameter λ), hereafter referred to simply as "feedback", guides activations towards their representations from the higher levels, thereby reducing the reconstruction errors over time, (iii) the memory term helps to retain the current representation over successive timesteps, and (iv) the feedforward error correction term (controlled by parameter α) corrects representations in each layer such that their next prediction better matches the preceding layer, also contributing to the reduction in reconstruction errors. Together, the feedback and feedforward error correction terms fulfill the objective of predictive coding as laid out by [START_REF] Rao | Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects[END_REF].

Training procedure

The network's training includes two stages: pretraining and finetuning (both using 10 timesteps for inference). The pretraining (over 150 epochs) was conducted in an unsupervised way with a reconstruction objective (see Eq. 4), wherein both feedforward and feedback convolution weights were optimized over the CIFAR100 natural images dataset [START_REF] Krizhevsky | Learning multiple layers of features from tiny images[END_REF]. This was done to learn a hierarchy of relevant features to describe each natural image, as well as the corresponding generative pathway to reconstruct images from their features.

For the second stage, we added a 3-layer classification head to the network (consisting of three fully-connected layers), and finetuned all parameters of the network on the custom dataset presented below with a supervised binary cross-entropy loss. The weights of the whole network were finetuned for 25 epochs, after which it was tested with a new validation set. We performed three distinct pretrainings (each with different randomly initialized weights), and then each of these pretrainings was used to finetune 3 distinct networks. The reported test results are averaged over the resulting 9 networks.

During both pretraining and finetuning we use the backpropagation-through-time (BPTT) approach, which unrolls the network across all timesteps, and then backpropagates errors. We use the PyTorch automatic differentiation for efficiency and simplicity. We use the ADAM optimizer [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF], a stochastic gradient method with automatically-adapting learning rates. This algorithm automatically uses momentum (and we use the default PyTorch hyperparameters of 0.9, 0.999) to estimate the first and second order moments of gradients. We use an initial learning rate of 5e -5.

When using BPTT, the computational requirements increase with the number of timesteps. Thus there is generally a balance required between the power of the model (where more timesteps allows for more complex computations) and feasibility. We chose 10 timesteps to produce meaningful dynamic trajectories while remaining within our computational constraints. We show in the Supplementary Material that we still see reasonable illusory perception results with other timestep values.

Stimuli

To test the perception of illusory contours we designed a custom dataset based on the Kanizsa illusion. We systematically generated stimuli consisting of an image with either a square, or four pacman-shaped inducers. The network was trained on a simple binary discrimination task: to classify each image as either a square, or pacman inducers. We generated four types of stimuli: (i) the real square; (ii) the illusory condition with all 4 inducers facing inwards (All-in); (iii) a control condition with all inducers facing outwards (All-out); and (iv) a random condition with random inducer orientation, but neither all in nor all out (Random). We finetuned the dataset for classification using only types (i) and (iv), and used (ii) and (iii) only at test time. In this way, the illusory shape is not part of the training set (out-of-sample), so the network cannot explicitly learn to categorise it either as a square or pacmen. The control condition was used to verify how the network reacts to stimuli which are out-of-sample but do not produce an illusion in humans.

Each image had a resolution of 32 × 32, consistent with the size of the CIFAR100 dataset used in the pretraining stage. The stimuli were designed to vary in luminance (for both the background and inducers, varying between 0 and 1), size, and position. This encourages the network to learn general rules and avoids overfitting through simple rote memorisation of the training set. For the same purpose, we also added Gaussian noise to each image (with variance randomly drawn from 7 pre-set levels varying from 0 to 0.3). Figure 3 shows sample stimuli, with the different levels of noise. We generated a training set of 10,000 images, 5,000 each for the Square and Random classes, and another set of 2,500 images for validation. For testing, we generated 1200 images of each class for a total of 4,800 images.

Feedforward baselines

The behaviour of the network at the first timestep (here t = 1, after feedforward initialization of activations throughout the network, but before error correction updates are applied according to Eq. 1) can be understood as a pure feedforward network. However, the network's features have been trained as part of the autoencoder blocks to optimize an unsupervised reconstruction objective -not (or not only) a supervised classification objective as in standard feedforward CNNs. We thus also chose to compare our model to a pure feedforward network (FF) trained only with a supervised classification loss. For a direct comparison, we also trained this network on CIFAR 100, before finetuning it on the shapes. This network has exactly the same architecture and number of parameters as the predictive coding network when considered only at t = 1, with the only difference being the learning objective.

However, this feedforward network has many fewer parameters overall than the full predictive coding network (around half). Thus, to directly compare to a feedforward network with roughly the same number of parameters, we also constructed two other feedforward networks: FF-C with more channels in each layer, and FF-K which uses a larger kernel size [START_REF] Spoerer | Recurrent convolutional neural networks: A better model of biological object recognition[END_REF]. Table 2

Code availability

All the code, along with the pretrained models and stimulus datasets used in this work is available at: https://github.com/rufinv/Illusory-Contour-Predictive-Networks.

Results

Illusory contour perception

Shape classification

After pretraining on natural images and fine-tuning on simple shapes, the network could discriminate between physical squares and randomly oriented inducers in any configuration (except the two critical configurations, All-in and All-out, which had not been seen during training). During testing, all four classes were presented to the network, with All-out (control class) and All-in (illusory contours) as novel stimulus configurations formed by familiar inducers. Our hypothesis was that when presented with the All-in configuration, the network would assign a higher probability to the square class than when presented with the All-out configuration.

For each class, we thus inspected the probability assigned by the network to the square category for each image (Figure 4, results averaged over 9 networks). At timestep 1 (with only feedforward processing having taken place), the network appears to classify images based on low-level, local information, as all the pacmen-made patterns (Random, All-out, All-in) are recognized as non-square shapes. However, over timesteps, the network begins to recognize the All-in condition (the illusory contour) as a square, at a much higher rate than the All-out and Random conditions. After 50 timesteps, the average probability assigned by the network to the square class (an "illusory contour perception") increases by more than 40% for the All-in condition, compared to around 20% (depending on noise level) for the other conditions. Although this measure does not go above 50%, we suggest that even humans would not actually categorize inducers as squares-despite "seeing" the illusion, we easily recognize that there is no actual square in the image. As the noise level increases (Figure 4, right), the likelihood of reporting a square for the All-in condition stays roughly constant, whereas the likelihood for the All-out and Random conditions increases. This can be interpreted as the network becoming more 'confused', as all probabilities are drifting together. The difference between the illusory condition and the control conditions gets smaller as the noise level increases, but remains visible for high noise. This is consistent with previous work which demonstrated that humans perceive illusory contours even under noise [START_REF] Gold | Deriving behavioural receptive fields for visually completed contours[END_REF].

Models

Square The probability of square outcome for each class, for the feedforward baselines and the predictive coding network (including its initial timestep, which can also be viewed as a feedforward network, but with an additional unsupervised pretraining as part of a stacked autoencoder architecture). We see that the feedforward networks assign a square probability of almost 0 to all non-square classes, whereas the predictive coding network, by the last time timestep, has assigned a probability of nearly 0.5.

We also compared our predictive coding network to the various feedforward baselines. Table 3 shows the results. We clearly see that all feedforward networks assign a near-zero average square probability to both the All-in and All-out conditions, showing that they do not perceive the illusion at all. This further confirms the hypothesis that feedback connections are critical for the perception of illusory contours.

Image reconstructions

Although the network assigned higher probability of square to the illusory contours than to the control condition, it remains hard to draw the conclusion that the network could really "see" illusory contours. It could still be the case that the network is basing its classification decision on other features (e.g. the correctly-oriented corners). A major advantage of the predictive coding model is that we can use its generative feedback pathway to inspect the image reconstructions produced by the model. We inspect the reconstruction of the bottom layer of the network (that is,

d 0 (t) = W b 1,0 e 1 (t)). Figure 5A
displays two examples of reconstructed images at timestep 100. Compared to the original images (on the left) whose noise standard deviation is 0.1, the reconstructed images are denoised by the network, and the illusory contour shapes appear clearer. (Note that this is an emergent property of predictive coding, and not a built-in property of our training scheme: the network's body was not trained with a denoising objective, but only to reconstruct clean natural images). To quantify the illusion "perceived" by the network, we examined the luminance profile of the reconstructions: for each image we computed a "Figure-Ground luminance difference" (FG), as illustrated in Figure 5B. This statistic measures whether the network perceives an illusory brightness enhancement like humans. Given the expected position of the illusory contour (the position that the square would have occupied if it had been real instead of illusory), along each of the four cardinal axes we took two pixels inside (red in Figure 5B) and two pixels outside the square (green in Figure 5B). We computed the average difference between the pixel luminance values inside and outside. A polarity factor (-1 or 1) multiplied this measure, to take into account the different configurations: dark inducers (polarity=1) are expected to produce lighter illusory shapes, light inducers (polarity=-1) to produce darker ones. The constructed FG measure is zero in the original All-in images (since it is measured in the background between the inducers), but should be positive in the image reconstructions whenever an illusory contour is perceived. Figure 5C shows how this FG value changes over predictive coding iterations, for illusory contours (All-in class) and the other two non-illusory shapes (All-out and Random classes). For the All-in class, after the initial time step the value is consistently higher than zero, and 5 to 10 times higher than the FG value measured for the control (All-out) or the random inducer classes. The corresponding luminance difference is small but reliable (on the order of 0.02, with 1 denoting the full luminance range). For comparison, the same FG value for the physical square would range from 0.3 to 0.6 (not shown here). Interestingly, as seen above, this perception of illusory contours was still present but somewhat reduced when images were corrupted with extremely high levels of noise (σ = 1.0), a behavior that would be expected in humans. For all other noise-levels that the network was trained to reconstruct, the results are relatively similar; thus, from here onward we only plot results for a single level of noise (0.1), even though all our conclusions equally apply to all levels.

Ablation studies

In order to determine the contribution of the various components of the network to illusory contour perception, we performed systematic ablation studies. For each of the hyperparameters α, β and λ in the update Equation 1, we set the parameter to 0 at test time and observed how this affects the network behaviour. Additionally, we also tested a network with a larger value of α, keeping all other hyperparameters at their default value. Figure 6 shows the results of these ablation experiments.

As expected, removing feedback (λ) leads to the complete disappearance of the illusion. This confirms that the generative feedback plays a critical role in producing the illusion. Interestingly, we see that removing the constant feedforward drive β (after the initial feedforward activation pass) also seems to diminish the illusory effect, although not completely. It thus appears that both feedforward and feedback contributions are important to see the illusion. Finally, when removing the feedforward error correction term α, the square classification probability does not seem to change much; however, the FG values strongly increase, specifically for the All-in condition. This suggests that the network may be seeing the illusion even more strongly. This can be explained by the fact that the role of the feedforward error term α is to update the activations in each layer in order to improve their reconstruction of the layer below. Thus, large FG values (in fact reflecting an imperfect reconstruction of the input image caused by the illusion) are "corrected" slightly by the feedforward error correction. When α is set to 0, the network does not correct these large FG values anymore, and is free to "perceive" a stronger illusion.

To verify that feedforward error correction does indeed suppress the illusory contour perception, we also tested a model with stronger α. In this case, both the square classification probability and FG value are visibly decreased for the All-in condition, confirming our hypothesis.

In Rao and Ballard's original formulation [START_REF] Rao | Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects[END_REF], they treated the two error terms (feedforward and feedback error corrections) together, and did not explore their relative effects. Here we show that only one of these (the feedback error correction) is critical to the perception of illusory contours, but that overall this perception is consistent with the predictive coding framework. However,

Probability (Square)

FG

Full model No α Large α No feedforward

No feedback 4 and5 for σ = 0.1. Probability of square classification is reported in the top row, FG value in the bottom row. Each component of the update equations was set to 0 at test time (with the other parameters at their default value): the feedforward drive term β (2nd column), the feedback term λ (3rd column), the feedforward error correction term α (4th column). In addition, we tested a larger value of α (last column).

α = 0.1 β = 0.2 λ = 0.1 α = 0.1 β = 0.2 λ = 0.1 α = 0 β = 0.2 λ = 0.1 α = 0.3 β = 0.2 λ = 0.1 α = 0.1 β = 0 λ = 0.1 α = 0.1 β = 0.2 λ =
the network with α = 0 is essentially equivalent to a regular feedback network, so this suggests that other feedback architectures may also lead to the perception of illusory contours. 4 and5 for σ = 0.1. Probability of square classification is reported in the top row, FG value in the bottom row. Generally, this produces similar results to just switching off the components at test time. Again, feedback is essential to the perception of the illusion, and feedforward encourages it. On the other hand, α being larger tends to diminish the perception of the illusion.

Training restricted networks

To further explore the effects of the various network components, we also trained networks without each component. The results are shown in Figure 7. The difference from the previous ablation studies (Figure 6) is that here, rather than being trained with all the components and then having them artificially switched off at test time, here the network "knows" during training which components are unavailable, and can potentially learn to compensate for their absence. Thus, where the previous section answered the question "How does our full trained network behave without each component", this approach answers the question "What is the behaviour of the best possible model without each component". That being said, overall this experiment largely confirms the results of the above ablation tests: that the feedback is essential for the perception of the illusion; that feedforward significantly contributes to this perception; and that larger α (feedforward error correction) diminishes this illusion.

One significant difference between Figure 6 and Figure 7 is that training with a large α seems to have a stronger deleterious effect than just increasing it at test time. We speculate that this is due to the fact that when the network is trained with large α, it implicitly learns to "trust" the reconstructions from higher levels more (since they will be corrected faster), and thus learns larger feedback weights. 4,5, 6 and 7, to facilitate comparisons. B. Testing whether networks that have only been pretrained on natural images (no fine-tuning) could perceive illusory contours (this test relies on FG values, as classification probability is not computable for these networks) C. Comparing pretraining on CIFAR100 (the Full model) to training directly on the shapes dataset. D. Comparing finetuning on a noisy shape dataset (the Full model) to a clean alternative, i.e. trained and finetuned without noise. The testing is performed without noise (left) or with σ = 0.1 noise (right).

Pretraining and Finetuning Datasets 350

We also investigated to what extent the specific datasets used for pretraining and/or fine-tuning the network affect its behaviour. Figure 8 shows the results of three comparative tests. In the first test, we took our three pretrained networks (without finetuning on the custom shapes dataset) and tested whether they can "perceive" illusory contours solely based on the learned statistics of natural images (therefore, this test relies only on the FG figure-ground luminance calculation). In the second test, reconstruction pretraining, and supervised classification finetuning). In the third test, we trained without any noise during the finetuning (supervised classification) stage.

Figures 8B andC confirm our hypothesis that pretraining on natural images is necessary and sufficient for illusion perception. For Figure 8B, we can only measure the FG values from image reconstructions, but not the classification probabilities, since the pretrained networks were not equipped with a decision head at this stage. Figure S1-A in the Supplementary Material shows that with the default set of parameters, the pretrained networks can already perceive a certain amount of illusion.

Here, to maximize the networks' ability to perceive illusory contours, we applied a different set of parameters with β = 0.1, λ = 0.2 and α = 0 (since we have observed in our ablation tests above that increasing λ and decreasing α could favour the illusion; default parameters are still used elsewhere unless stated otherwise). We also test these networks on clean shapes, because they were only trained on the clean CIFAR100 natural images. From Figure 8B, it is clear that pretraining on natural images can provide sufficient knowledge about image contour statistics to induce the perceptual illusion. Although the magnitude of the effect is somewhat dependent on the choice of parameters, as detailed above and in our ablation studies (Figures 6 and7), in a separate simulation (Figure S1-B) we show that illusory contours can even arise in a parameter-free alternative training regime (similar to the one described below in 3.5). In this case, feed-forward and feedback connections are trained to optimize reconstruction after a single timestep-so the parameters α, β and λ play no role in the outcome. Yet the networks could still perceive illusory contours (Figure S1-B). Figure 8C shows that when the network convolutions are only trained to reconstruct shapes (squares, pacman inducers) instead of natural images, the model processes the All-in and All-out test images similarly. Both results provide evidence that the perception of illusory contours is an emergent property from the visual system's adaptation to the statistics of the natural world. The third test investigates the effect of noise (and therefore, uncertainty) during shape classification training. Figure 8D demonstrates that training without any noise results in a weaker illusory effect for both the square class probability and the FG value. That is, some form of uncertainty or variety in the learned shapes dataset appears helpful; otherwise the network may find a way to perfectly encode or memorise the stimuli, i.e. "overfit" the training set.

Illusory contour perception in a modern deep network equipped with predictive coding dynamics

Finally, we asked whether our approach -equipping a feedforward network with recurrent predictive coding dynamics -could be applied to other networks to produce illusory contour perception. Here we used VGG, a much deeper 11-layered network [START_REF] Simonyan | Very deep convolutional networks for large-scale image recognition[END_REF], which more closely resembles modern state-of-the-art computer vision networks. Previous work from our group demonstrated that the VGG architecture can be equipped with predictive coding dynamics [START_REF] Choksi | Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics[END_REF], and we follow the same approach here.

The basic VGG architecture is a simple feedforward CNN with 8 convolutional layers with 3x3 kernels, max pooling, and 3 fully-connected layers for the head. Although now a few years out of date, it was state-of-the-art in 2014. Compared with the original architecture that was designed for larger images from the Imagenet dataset [START_REF] Simonyan | Very deep convolutional networks for large-scale image recognition[END_REF], the main difference for our network trained on smaller natural scenes (32x32 pixels) is that the first pooling layer was removed, and the final classification output layer was changed to have 100 instead of 1000 classes. We use a feedforward backbone network where the weights were optimized for natural image classification on the CIFAR100 dataset. The trained model is then augmented with predictive loops consisting of transpose convolution layers, as in the 3-layer model. However for this larger model, the feedback predictions span two feedforward convolutional layers, rather than just one. Thus the inputs to the feedback convolutions are the 2nd, 4th, 6th and 8th convolutional layers (respectively aiming to reconstruct the image input, 2nd, 4th and 6th layers). The weights of these feedback convolutions are optimized for one-step reconstruction over CIFAR100 with a simple mean square error loss (while the feedforward weights remain fixed). Finally the same update equations (Eqs 1 and 4) are used as with the 3-layer model. This recurrent model is hereafter referred to as PVGG (for "predictive-VGG"). Notably, the feedback connections in the PVGG model were trained with a different method (already alluded to above), optimizing image reconstruction on the CIFAR100 dataset after a single timestep. Therefore, the parameters α, β and λ play no role in the training. Then, during testing, we selected a set of parameters according to our observations with the smaller model. As we found that the feedforward error correction term α tends to suppress the illusion and the feedback λ tends to increase it (Figures 6 and7), we here set α = 0, β = 0.1, λ = 0.2.

Since we do not train the model at all on the shapes dataset, we cannot examine the network's classification decisions (since in order to do so, at the very least the classification head would need to be re-trained with the appropriate shape classes). Instead we directly inspected the image reconstructions of the network and calculated the FG values, as illustrated in Figure 5. Figure 9 shows the results: it is clear that over timesteps, once again the average FG value for the illusory inducers rapidly becomes much larger than either the random or control conditions. As in previous experiments, we could verify that feedback error correction is critical for this illusory perception: no illusion occurred when we ablated the feedback at test time, i.e. λ = 0 (Figure 9, right). The average FG value for the inducer condition is significantly larger than for the two control conditions, demonstrating that the network perceives the illusion. Right: Removing feedback (which here leads to constant activations, since there is no feedforward error correction either) clearly destroys the illusion.

Although this network also appears to perceive the illusion, the long-run dynamics of the network are quite different from the smaller model. While in the smaller model, even after 100 timesteps the illusion is still perceived, in the PVGG model it gradually disappears after some timesteps. In a way, presenting a static stimulus for so many timesteps is somewhat unrealistic -for humans, the input stimulus is constantly changing, either because of environmental fluctuations, or because of saccadic eye movements. Indeed, previous work showed that when illusory contours are perceived by humans in the periphery, they disappear after a few seconds [START_REF] Ramachandran | On the perception of illusory contours[END_REF]. However, the difference between PVGG and the smaller model is interesting, and could be an avenue for further research. A number of key differences between the two models could be at play. Most importantly, the weights of PVGG are not learned "over timesteps" -the feedforward weights are trained just for feedforward classification, and the feedback weights are trained just for a single step reconstruction. Thus, the network never updates its weights after predictive coding updates. On the one hand, this makes it even more remarkable that the updates lead to the perception of the illusion, and is strong evidence that the framework is compatible with illusory contour perception. However, it could also be the reason why the long-term dynamics are somewhat unstable (or non-convergent), unlike in the smaller model. The other significant differences are that PVGG is a much deeper network, and was trained only on natural images, never on the shapes dataset. As such it has much more complex, abstract representations, and it is possible that these representations are not well-calibrated to simple shapes or contours which would not appear in CIFAR images. As a result the network might struggle to maintain stable representations of the illusory contours, and might instead "hallucinate" (or "fill in") what it imagines is missing texture or complexity.

The above strategy of avoiding re-training or fine-tuning the network has advantages from both an AI and a neuroscience perspective. First, not only does it save the time and power required for training, but it also allows us to experiment with any pretrained feed-forward model -including models which we would not be able to train ourselves due to computational restrictions. Second, from a neuroscience perspective, the fact that the perception of the illusion can occur in a network which has never seen the shapes dataset before (nor, presumably, any square or pacman shape) demonstrates even more clearly that it results from a combination of feedback connectivity, predictive coding dynamics, and exposure to natural scenes.

Discussion

The purpose of this study was to test whether a feedback neural network with brain-inspired recurrent dynamics would perceive illusory contours (Kanizsa squares) in a similar manner to humans. Augmenting a feedforward CNN with predictive coding recurrent dynamics allowed us to (i) analyse explicit classification decisions (square vs. inducers) and, unlike other related work [START_REF] Lotter | A neural network trained to predict future video frames mimics critical properties of biological neuronal responses and perception[END_REF][START_REF] Baker | Deep convolutional networks do not perceive illusory contours[END_REF][START_REF] Kim | Neural networks trained on natural scenes exhibit gestalt closure[END_REF], (ii) visualize reconstructed inputs from the model's viewpoint. As reported in a preliminary version of this study recently published in a conference workshop [START_REF] Pang | Predictive coding feedback results in perceived illusory contours in a recurrent neural network[END_REF], we found that, compared to a feedforward baseline, the recurrent dynamics led the network to perceive more illusory contours. Notably, by inspecting the network's reconstructions, we were able to directly visualize the network's internal representation of the stimulus, which provides a much clearer measure of "illusory perception" than previous works. We found evidence of modulations of the perceived luminance profiles in the expected direction for illusory shapes, suggesting that the network is truly "perceiving" the contours. We extended this analysis and performed systematic ablation studies, both at test time and at training time, and found that the feedback error correction term is essential to the perception of the illusion, while the feedforward error correction term tends to decrease it. Similarly, exploring the datasets used for pretraining and fine-tuning the network revealed that prior exposure to the statistics of natural scenes is a crucial element of illusory contour perception. Finally, we also implemented the predictive coding dynamics in a standard VGG model, and found that the modified PVGG model also exhibited the perception of illusory contours. This suggests that illusory contour perception arises from predictive coding feedback dynamics, independently of the scale of the model.

In summary, we provide clear evidence that brain-inspired recurrent dynamics can lead networks to perceive illusory contours like humans.

Although there are intrinsic differences between the human visual system and artificial neural networks (e.g., the global error signals required for learning via back-propagation [START_REF] Lillicrap | Backpropagation and the brain[END_REF])), we argue that the current findings highlight three key similarities with biological vision. First, both systems may engage similar global processing of illusory contours. In the visual system, [START_REF] Pan | Equivalent representation of real and illusory contours in macaque V4[END_REF] reported that illusory contours activate equivalent representations in V4 compared to real contours, whereas V1 and V2 differently encode their respective local features. That is, in addition to local processing in early visual regions, there exists a global processing mode whereby illusory inducers form integral contour representations. In a similar way, when presented with illusory contours the current network assigned much higher probability of the "square" class than for either Random or All-out control images, though they shared the same local features as the All-in illusory contour images. This indicates that the network also possesses a capability of global processing. Moreover, this global processing primarily results from the feedback connections, since none of the tested feedforward networks could perceive illusory contours (Table 3). Second, the "behavioural" performance (i.e. decision probability) of the network is also consistent with physiological research on illusory contours. [START_REF] Lee | Dynamics of subjective contour formation in the early visual cortex[END_REF] compared EEG activity for illusory contours and other patterns, and found that the activity for illusory contours is significantly higher than control random stimuli, but still lower than real contours. In the current study, the "Square" class probabilities assigned by the network after the Softmax layer indicate a similar pattern (see Figure 4). Lastly, at the "perceptual" level, we directly checked the internal representation at the first layer of the network (through its generative "image reconstruction" pathway). The FG metric suggested that the network perceives a brighter (or darker) illusory shape, consistently with the "illusory brightness" reported when humans perceive illusory contours [START_REF] Schumann | Beiträge zur analyse der gesichtswahrnehmungen[END_REF][START_REF] Spillmann | Phenomena of illusory form: Can we bridge the gap between levels of explanation?[END_REF][START_REF] Parks | Rock's cognitive theory of illusory figures: a commentary[END_REF].

Having designed a biologically inspired architecture, we performed ablation studies to examine how the behaviour of the network depends on its various components. Most importantly, we found that feedback error correction was critical to the network perceiving the illusory contours, which is in agreement with previous results which demonstrated that feedforward networks do not appear to perceive illusory contours [START_REF] Baker | Deep convolutional networks do not perceive illusory contours[END_REF]. We also found that removing the feedforward error correction term (one half of the predictive coding update proposed by [START_REF] Rao | Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects[END_REF]) seems to enhance the perception of the illusion. This is easily explained as this term serves to correct the representations to minimise reconstruction error of the incoming layer, so the erroneous contour and luminance difference may be "corrected out" when this term is present. These ablation experiments allow us to highlight how the two error correction terms, introduced concurrently in [START_REF] Rao | Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects[END_REF], play two distinct roles: the feedforward error correction term encourages the network to accurately represent the stimuli, anchoring it in the ground truth; whereas the feedback error correction tries to explain the input in terms of the network's implicit priors, and in this case causes the network to "hallucinate" based on its higher-level representations (for example, closed contours or square shapes). Finally, by scaling the predictive coding updating dynamics to VGG11 and finding the same human-like illusory contours (see Figure 9), we provided evidence that these feedback dynamics may induce illusory perception across a large range of deep convolutional network architectures.

In Section 3.4, we investigated what effect pretraining on natural images has on the network. Previous work suggested that CNNs trained on natural scenes are better models of brain activity [START_REF] Maheswaranathan | Deep learning models reveal internal structure and diverse computations in the retina under natural scenes[END_REF]. Indeed, we saw that training directly on the shapes was not sufficient for the network to assign higher probability of the square class to illusory contours, nor to produce the positive FG values which are the hallmarks of illusory contour perception. Training on natural images encourages the network to learn efficient representations of natural stimuli -the constraints that arise from exposure to natural scene statistics in the brain have been proposed as a cause of many visual illusions [START_REF] Eagleman | Visual illusions and neurobiology[END_REF][START_REF] Gori | Visual illusions: An interesting tool to investigate developmental dyslexia and autism spectrum disorder[END_REF]. In other words, pretraining the network on natural images may help the network to learn more human-like representations. We also investigated the effect of training with images corrupted with Gaussian noise versus without. Such data augmentation typically helps in improving generalisation as it forces the network to learn meaningful features from the data instead of rote-memorising individual training examples [START_REF] Bishop | Training with noise is equivalent to tikhonov regularization[END_REF][START_REF] Akbiyik | Data augmentation in training CNNs: Injecting noise to images[END_REF]. We found that networks trained using noisy images perceive stronger and more consistent (as reflected by smaller standard error values) illusory contours.

In summary, by leveraging insights from neuroscience, we designed an original brain-inspired deep learning architecture and thus add to a growing body of literature exploring the cross-pollination of neuroscience and AI. On the one hand, the current study demonstrates that we can effectively use neuroscience principles to design artificial computer vision models, and probe them using classical stimuli and illusions from neuroscience and cognitive science. On the other hand, we also illustrate how modern deep learning techniques can be used as powerful tools in examining our theories of brain function [START_REF] Kriegeskorte | Deep neural networks: a new framework for modeling biological vision and brain information processing[END_REF][START_REF] Cichy | Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence[END_REF][START_REF] Vanrullen | Perception science in the age of deep neural networks[END_REF][START_REF] Marblestone | Toward an integration of deep learning and neuroscience[END_REF]. By building and testing a brain-inspired model, the current study highlights the essential roles of feedback connections [START_REF] Lee | Dynamics of subjective contour formation in the early visual cortex[END_REF][START_REF] Pak | Top-down feedback controls the cortical representation of illusory contours in mouse primary visual cortex[END_REF], predictive coding computation [START_REF] Notredame | What visual illusions teach us about schizophrenia[END_REF][START_REF] Nour | Perception, illusions and bayesian inference[END_REF][START_REF] Raman | Predictive coding: a possible explanation of filling-in at the blind spot[END_REF][START_REF] Shipp | Neural elements for predictive coding[END_REF], and prior experience of natural environments [START_REF] Eagleman | Visual illusions and neurobiology[END_REF] for the perception of visual illusions. Future work could use the same kind of predictive coding model to test other aspects of the predictive coding theory in neuroscience, such as its tendency to produce oscillatory dynamics [START_REF] Alamia | Alpha oscillations and traveling waves: Signatures of predictive coding?[END_REF], or other phenomena observed in human vision, such as ambiguous stimuli and multi-stable perception, or the Gestalt rules of perceptual organization.

Figure 1 :

 1 Figure 1: Kanizsa square illusion.

Figure 2 :

 2 Figure 2: Network architecture. The architecture consists of a main body and a classification head (or dense layers).For the main body, the predictive coding strategy is implemented in stacked autoencoders, with three feedforward encoding layers (en) and three generative feedback decoding layers (dn). Reconstruction errors ( n) are computed and used for the proposed predictive coding updates which are denoted by "P.C." loops. Dense layers are added on top of the structure to implement a binary classification task.

Figure 3 :

 3 Figure 3: Finetuning dataset. Sample training and testing images for 7 different levels of Gaussian noise: Square, Random, All-out, and All-in. All stimuli varied in luminance (both background luminance and inducer luminance), size (square sidelength or distance between inducers), and position. To enhance visualization, we displayed all seven levels of noises with each sample picture; while in practice, only one level of noise was randomly assigned to each stimulus configuration.

Figure 4 :

 4 Figure4: Classification results. The probability of "square" report over recurrent predictive coding iterations (timesteps). Each panel shows a different noise level -for visibility, only three out of six levels of noise are shown here, in addition to the clean images (σ = 0). Feedback iterations increase the likelihood of "square" report, especially for the All-in (illusory contour) condition. Results are averaged over 9 networks and error bars represent standard error of the mean (SEM) across the networks.

Figure 5 :

 5 Figure 5: Quantification of illusory countour perception in the neural network. A. Two examples of illusory contour (All-in) stimuli, and their corresponding reconstructions from the network at timestep 100. B. Computation of the "FG" value, measuring the figure-ground luminance difference. C. FG-luminance difference for classes Random, All-out, and All-in over 100 timesteps (zero is absence of illusion, larger values mean more illusion is perceived). Dashed lines denote the FG values computed from original input pictures of the corresponding classes. With clean images, we expect FG on input pictures to be equal to zero as shown in the first plot, while FG values can be bigger or smaller than zero with noisy input images. Results are averaged over 9 networks (distinct random initializations, pretraining and fine-tuning) and error bars represent SEM across networks.

Figure 6 :

 6 Figure6: Ablation during testing. The full model results (first column) correspond to the same data already reported in Figures4 and 5for σ = 0.1. Probability of square classification is reported in the top row, FG value in the bottom row. Each component of the update equations was set to 0 at test time (with the other parameters at their default value): the feedforward drive term β (2nd column), the feedback term λ (3rd column), the feedforward error correction term α (4th column). In addition, we tested a larger value of α (last column).

Figure 7 :

 7 Figure 7: Training restricted networks. These graphs show the behaviour of networks trained from scratch without the various components of the full model. The full model results (first column) correspond to the same data already reported in Figures4 and 5for σ = 0.1. Probability of square classification is reported in the top row, FG value in the bottom row. Generally, this produces similar results to just switching off the components at test time. Again, feedback is essential to the perception of the illusion, and feedforward encourages it. On the other hand, α being larger tends to diminish the perception of the illusion.

Figure 8 :

 8 Figure 8: Testing the influence of pretraining and finetuning datasets. A. The full model is the same data already reported in Figures 4, 5, 6 and 7, to facilitate comparisons. B. Testing whether networks that have only been pretrained on natural images (no fine-tuning) could perceive illusory contours (this test relies on FG values, as classification probability is not computable for these networks) C. Comparing pretraining on CIFAR100 (the Full model) to training directly on the shapes dataset. D. Comparing finetuning on a noisy shape dataset (the Full model) to a clean alternative, i.e. trained and finetuned without noise. The testing is performed without noise (left) or with σ = 0.1 noise (right).

FG

Figure 9 :

 9 Figure9: FG values in the PVGG network. Left: The average FG value for the inducer condition is significantly larger than for the two control conditions, demonstrating that the network perceives the illusion. Right: Removing feedback (which here leads to constant activations, since there is no feedforward error correction either) clearly destroys the illusion.

Table 1 :

 1 Table of parameters.Each encoding layer is a combination of a convolution layer and a ReLU nonlinearity with parameters Conv(channels, kernel size, stride). All convolutions have padding 4. Decoding layers consist of a deconvolutional layer with ReLU non-linearity, except for layer d 0 , which uses a Sigmoid activation function, in order to compare with the input picture with pixel values ranging from 0 to 1. After flattening the output of the last convolutional layer, and going through a batch normalization function, two dense layers with a structure of Weight(in features, out features) project to the binary decision layer.

	n-1 (t) is the reconstruction error for layer n -1: the mean

  compares the number of parameters for each network.

	Models Kernel Size No. Channels No. Parameters
	FF	5 × 5	(3,128,128)	1,403,620
	FF-C	5 × 5	(3,172,172)	2,248,684
	FF-K	7 × 7	(3,128,128)	2,199,268
	PC	5 × 5	(3,128,128)	2,232,679

Table 2 :

 2 Comparison of parameters for feedforward networks. PC is our predictive coding architecture. FF comprises only the feed-forward pass, but is trained with a supervised classification objective. FF-C is obtained by increasing the number of channels; FF-K by increasing the kernel size.

Table 3 :

 3 Comparison to feedforward networks.

to further confirm the critical role of natural images (instead of other much simpler but task-related images) in illusion perception, we simply removed the pretraining procedure on the CIFAR100 natural image dataset. That is, we trained our network directly on the shape dataset (for both unsupervised

Acknowledgements

The authors would like to thank Milad Mozafari for providing the implementation for PVGG. This work was funded by an ANITI (Artificial and Natural Intelligence Toulouse Institute) Research Chair to RV (ANR grant ANR-19-PI3A-0004), as well as ANR grants AI-REPS (ANR-18-CE37-0007-01) and OSCI-DEEP (ANR-19-NEUC-0004). ZP is supported by China Scholarship Council (201806620059).

545

The authors declare no competing financial interests.