Sumit J Darak

Honggang Zhang
email: honggangzhang@zju.edu.cn

Jacques Palicot
email: jacques.palicot@centralesupelec.fr

Christophe Moy
email: christophe.moy@centralesupelec.fr

Decision Making Policy for RF Energy Harvesting Enabled Cognitive Radios in Decentralized Wireless Networks

Keywords: Decision making policy, multi-armed bandit algorithms, radio

Cognitive radio (CR) paradigm with radio frequency energy harvesting (RFEH) have significant potential to improve the network throughput by utilizing vacant spectrum using battery-operated self-sustainable radio terminals. Research efforts relevant to these paradigms are focused on the mode selection policies which decide when to switch from CR mode (i.e., opportunistic vacant spectrum access mode) to the RFEH mode (i.e., battery charging using ambient RF energy) and vice-versa. So far, very little attention has been paid to the dual but competing task of frequency band selection in CR and RFEH modes under partially observable environment in the decentralized wireless networks. Furthermore, the need of tunable bandwidth frequency band access for CRs and lower subband switching cost (SSC) for energy efficient implementation have made the design of the decision making policy (DMP) more challenging. In this paper, a new CR-RFEH DMP has been proposed for RFEH enabled CR terminals in the decentralized wireless networks. The proposed DMP consists of three sub-units: 1) Bayesian approach based tunable Thompson sampling algorithm for subband statistics estimation, 2) Thompson sampling algorithm based subband access scheme exploiting the past collision events to minimize collisions among CRs, and 3) Mode selection scheme. Simulation results show that the proposed DMP offers 10-35% improvement in the throughput of the decentralized network and 40-90% reduction in the number of subband switchings compared to existing DMPs. The simulation results are then validated using real radio signals on the proposed USRP testbed.

frequency energy harvesting, Thompson sampling.

Introduction

Opportunistic spectrum access (OSA) based cognitive radio (CR) paradigm has received significant attention from the academia as well as industrial partners [START_REF] Asadi | A Survey on Device-to-Device Communication in Cellular Networks[END_REF][START_REF] Phunchongharn | Resource Allocation for Device-to-Device Communications Underlaying LTE-Advanced Networks[END_REF][START_REF] Lu | Dynamic Spectrum Access in Cognitive Radio Networks with RF Energy Harvesting[END_REF][START_REF] Zheng | Information and Energy Cooperation in Cognitive Radio Networks[END_REF][START_REF] Park | Optimal Mode Selection for Cognitive Radio Sensor Networks with RF Energy Harvesting[END_REF][START_REF] Lu | Wireless Networks with RF Energy Harvesting: A Contemporary Survey[END_REF][START_REF] Hoang | Opportunistic Channel Access and RF Energy Harvesting in Cognitive Radio Networks[END_REF][START_REF] Nintanavongsa | Design Optimization and Implementation for RF Energy Harvesting Circuits[END_REF][START_REF] Mohjazi | RF-Powered Cognitive Radio Networks: Technical Challenges and Limitations[END_REF]. In wireless networks, CR paradigm allows unlicensed users to communicate over identified vacant licensed subband(s). In some cases, unlicensed users do not need base stations or access points for establishing the communication link. This is termed as underlay inband communications [START_REF] Asadi | A Survey on Device-to-Device Communication in Cellular Networks[END_REF][START_REF] Phunchongharn | Resource Allocation for Device-to-Device Communications Underlaying LTE-Advanced Networks[END_REF]. OSA based Long Term Evolution-Unlicensed (LTE-U) has been demonstrated as potential and feasible way to improve the throughput of wireless networks. Furthermore, some data intensive and delay tolerant services in the LTE-Advanced and 5G may use OSA to meet their spectrum needs and improve the network throughput [START_REF] Hong | Cognitive Radio in 5G: A Perspective on Energy-Spectral Efficiency Trade-off[END_REF][START_REF] Herranz | Cognitive Radio Enabling Opportunistic Spectrum Access in LTE-Advanced Femtocells[END_REF]. The popular schemes for such OSA communications are: 1) Centralized scheme where base station is responsible for orthogonalization of CR terminals to optimum vacant frequency bands, and 2) Decentralized scheme where CR terminals independently choose the vacant subband(s) without an explicit information exchange or pre-agreement among them. Decentralized approach would be an efficient choice over the centralized approach for larger networks due to advantages such as ease of implementation, robustness to link or node failures, no communication overhead and lower delay [START_REF] Asadi | A Survey on Device-to-Device Communication in Cellular Networks[END_REF][START_REF] Phunchongharn | Resource Allocation for Device-to-Device Communications Underlaying LTE-Advanced Networks[END_REF]. Furthermore, decentralized approach is the preferred choice for public safety networks and proximity-aware social networking services.

Energy harvesting from solar, thermal, radio frequency (RF) sources for green self-sustainable communications has recently generated lots of interest due to rising energy demand and cost, awareness towards clean environment and green taxes etc. Such harvesting approaches improve the battery life of radio terminals making them self-sustainable up to certain extent. The RF energy harvesting (RFEH) is a new trending technique which facilitates the conversion of received RF signals into electricity that can be stored and used later for data transmission [START_REF] Lu | Dynamic Spectrum Access in Cognitive Radio Networks with RF Energy Harvesting[END_REF][START_REF] Zheng | Information and Energy Cooperation in Cognitive Radio Networks[END_REF][START_REF] Park | Optimal Mode Selection for Cognitive Radio Sensor Networks with RF Energy Harvesting[END_REF][START_REF] Lu | Wireless Networks with RF Energy Harvesting: A Contemporary Survey[END_REF][START_REF] Hoang | Opportunistic Channel Access and RF Energy Harvesting in Cognitive Radio Networks[END_REF][START_REF] Nintanavongsa | Design Optimization and Implementation for RF Energy Harvesting Circuits[END_REF][START_REF] Mohjazi | RF-Powered Cognitive Radio Networks: Technical Challenges and Limitations[END_REF][START_REF] Qin | Throughput Analysis for Compressive Spectrum Sensing with Wireless Power Transfer[END_REF]. Today, researchers have claimed that the RF energy of 1μW to as high as 3.5mW can be harvested at 2.4 GHz and 900 MHz frequencies depending on the distance from the source [START_REF] Lu | Dynamic Spectrum Access in Cognitive Radio Networks with RF Energy Harvesting[END_REF][START_REF] Lu | Wireless Networks with RF Energy Harvesting: A Contemporary Survey[END_REF][START_REF] Hoang | Opportunistic Channel Access and RF Energy Harvesting in Cognitive Radio Networks[END_REF][START_REF] Nintanavongsa | Design Optimization and Implementation for RF Energy Harvesting Circuits[END_REF][START_REF] Mohjazi | RF-Powered Cognitive Radio Networks: Technical Challenges and Limitations[END_REF]. In near future, further improvements in the RFEH circuits and an international RFEH standard can be expected.

Consider the decentralized wireless network consisting of multiple RFEH enabled CR terminals as shown in Fig. 1. Primary users (denoted as PUs) are the licensed users which communicate with other PUs over the frequency bands allocated by base station. In CR mode, RFEH enabled CRs communicate directly with other CRs over the identified vacant frequency bands. In RFEH mode, CRs harvest the RF energy from sources such as base stations (denoted as BS), WiFi access points and TV towers. Various decision making policies (DMPs) to identify the optimum vacant bands for CRs have been proposed in the literature [START_REF] Darak | Compute-Efficient Decision Making Policy for D2D Communications and RF Energy Harvesting in Cognitive Radio Networks: Go Bayesian![END_REF][START_REF] Xu | Decision-Theoretic Distributed Channel Selection for Opportunistic Spectrum Access: Strategies, Challenges and Solutions[END_REF][START_REF] Auer | Finite-time Analysis of the Multiarmed Bandit Problem[END_REF][START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Learning-Stage Based Decentralized Adaptive Access Policy For Dynamic Spectrum Access[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF][START_REF] Darak | Efficient Decentralized Dynamic Spectrum Learning and Access Scheme for Multi-standard Multi-user Cognitive Radio Networks[END_REF]. Nonetheless, the subband selection task has become more challenging in the decentralized networks consisting of RFEH enabled CRs. This is because, occupied subbands with higher RF energy and wider contiguous bandwidth are the desired subbands in the RFEH mode while the vacant contiguous subbands of any tunable bandwidth and lower RF energy are the desired subbands in the CR mode (i.e., OSA mode). Furthermore, tunable vacant bandwidth requirements of CRs must be taken into account to support wide variety of services ranging from data to multimedia in 5G and LTE-A. From power consumption perspective, the subband switching cost (SSC), i.e. total penalty incurred in terms of delay, power, hardware reconfiguration and protocol overhead when CR switches from one frequency subband to another, should be as minimum as possible. To the best of our knowledge, existing DMPs [1-9, 12, 14-21] are not mature enough to provide the comprehensive solutions for the subband selection, subband access and mode selection tasks for RFEH enabled CRs in the decentralized network. The design of such DMP is an objective of the work presented in this paper. In this paper, a new CR-RFEH DMP, a significant extension of our work in [START_REF] Darak | Compute-Efficient Decision Making Policy for D2D Communications and RF Energy Harvesting in Cognitive Radio Networks: Go Bayesian![END_REF], has been proposed. The proposed DMP consists of: 1) Tunable Thompson Sampling (TTS) algorithm to estimate the subband statistics for CR as well as RFEH modes, 2) Subband access scheme for orthogonalization of CRs in distinct subbands to minimize the number of collisions, and 3) Mode selection scheme to switch modes i.e., CR to RFEH and vice-versa. To the best of our knowledge, the proposed DMP is the first one designed using Bayesian approach based TS algorithm compared to existing DMPs designed using frequentist approach based learning algorithms for the subband statistic estimation [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF][START_REF] Darak | Efficient Decentralized Dynamic Spectrum Learning and Access Scheme for Multi-standard Multi-user Cognitive Radio Networks[END_REF]. Furthermore, the proposed DMP takes into account the specific subband requirements of the RFEH mode and tunable bandwidth access in the CR mode. Another novelty is that the proposed DMP uses TS algorithm for the subband access scheme compared to the randomization approach based schemes in [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF][START_REF] Darak | Efficient Decentralized Dynamic Spectrum Learning and Access Scheme for Multi-standard Multi-user Cognitive Radio Networks[END_REF]. The detailed simulation as well as experimental results on the USRP testbed validate the superiority of the proposed DMP over existing DMPs.

The paper is organized as follows. The detailed literature review is given in Section 2. The proposed DMP is presented in Section 3 followed by the simulation results in the Section 4. In Section 5, proposed USRP testbed is presented along with the experimental results and performance comparisons of various DMPs. Section 6 concludes the paper.

Literature Review

In this section, a review of existing DMPs for the decentralized wireless networks is presented.

DMP for Decentralized Networks without RFEH

In [START_REF] Xu | Decision-Theoretic Distributed Channel Selection for Opportunistic Spectrum Access: Strategies, Challenges and Solutions[END_REF][START_REF] Auer | Finite-time Analysis of the Multiarmed Bandit Problem[END_REF][START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Learning-Stage Based Decentralized Adaptive Access Policy For Dynamic Spectrum Access[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF][START_REF] Darak | Efficient Decentralized Dynamic Spectrum Learning and Access Scheme for Multi-standard Multi-user Cognitive Radio Networks[END_REF], decentralized wireless networks consisting of CR terminals without RFEH capability are considered. In such CRs, DMP operates only in one mode, i.e., CR mode. For networks where subband statistics are not known, they are estimated using learning algorithms such as frequentist approach based upper confidence bound (UCB) algorithm and its extensions, ε-greedy, Kullback-Leibler UCB (KL-UCB) etc. [START_REF] Xu | Decision-Theoretic Distributed Channel Selection for Opportunistic Spectrum Access: Strategies, Challenges and Solutions[END_REF][START_REF] Auer | Finite-time Analysis of the Multiarmed Bandit Problem[END_REF][START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Learning-Stage Based Decentralized Adaptive Access Policy For Dynamic Spectrum Access[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF][START_REF] Darak | Efficient Decentralized Dynamic Spectrum Learning and Access Scheme for Multi-standard Multi-user Cognitive Radio Networks[END_REF]. Such algorithms, also known as multiarmed bandit (MAB) algorithms, are based on exploration-exploitation dilemma i.e., the trade-off between the exploration of all subbands to accurately estimate their statistics and selecting the subband with the desired statistic, i.e., the subband with the highest probability of being vacant. MAB algorithms are asymptotically optimal with logarithmic regret. Here, the regret means the loss of data transmission opportunities due to the selection of sub-optimal subbands. Recently, Bayesian MAB algorithms such as TS and Bayes-UCB have been proved to have better regret than others and are discussed later in Section 3.2.

Even after the accurate estimation of subband statistics, DMPs in the decentralized network can incur significant regret due to the collision among CRs. To minimize the number of collisions, subband access scheme is desired for orthogonalization of CRs. The design of the subband access scheme for the decentralized networks is non-trivial since CRs do not share information with each other. In ρ rand scheme for the decentralized network with M CRs [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF], each CR randomly and independently chooses the rank, R(k) ∈ {1, 2, ..M } in the beginning. In the subsequent time slots, the underlying MAB algorithm calculates the quality index, based on past sensing events, for each subband.

Then, CR with the rank R(k) selects the subband with the R(k) th best quality index. Another policy in [START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF] follows time division fare share approach where the rank of each CR is rotated in circular fashion between 1 to M to allow an equal access to the optimum subbands to all CRs. In [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF], a new rank is randomly and independently chosen for CR experiencing collision. Though both DMPs are optimal with logarithmic regret, SSC in [START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF] increases linearly with t. The SSC can be minimized using the policy, ρ rand [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF], where the rank changes only when corresponding CR collides with other CRs. The ρ rand is further improved in [START_REF] Zandi | Learning-Stage Based Decentralized Adaptive Access Policy For Dynamic Spectrum Access[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF]. In [START_REF] Zandi | Learning-Stage Based Decentralized Adaptive Access Policy For Dynamic Spectrum Access[END_REF], the number of collisions are reduced by allowing the rank to be greater than M . In [START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF], access strategies in [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF] are combined with MAB algorithms to take into account different subband statistics. However, the SSC of [START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF] is high for the subband statistics where it performs superior than [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF]. In [START_REF] Darak | Efficient Decentralized Dynamic Spectrum Learning and Access Scheme for Multi-standard Multi-user Cognitive Radio Networks[END_REF], variable filtering architecture and its integration with the tunable subband access policy, ρ t rand , is presented that takes into account tunable bandwidth requirements of CRs. Still, the average SSC of [START_REF] Zandi | Learning-Stage Based Decentralized Adaptive Access Policy For Dynamic Spectrum Access[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF][START_REF] Darak | Efficient Decentralized Dynamic Spectrum Learning and Access Scheme for Multi-standard Multi-user Cognitive Radio Networks[END_REF] is high.

DMP for Decentralized Networks with RFEH Capability

Recent research efforts [START_REF] Lu | Dynamic Spectrum Access in Cognitive Radio Networks with RF Energy Harvesting[END_REF][START_REF] Zheng | Information and Energy Cooperation in Cognitive Radio Networks[END_REF][START_REF] Park | Optimal Mode Selection for Cognitive Radio Sensor Networks with RF Energy Harvesting[END_REF][START_REF] Lu | Wireless Networks with RF Energy Harvesting: A Contemporary Survey[END_REF][START_REF] Hoang | Opportunistic Channel Access and RF Energy Harvesting in Cognitive Radio Networks[END_REF][START_REF] Nintanavongsa | Design Optimization and Implementation for RF Energy Harvesting Circuits[END_REF][START_REF] Mohjazi | RF-Powered Cognitive Radio Networks: Technical Challenges and Limitations[END_REF][START_REF] Qin | Throughput Analysis for Compressive Spectrum Sensing with Wireless Power Transfer[END_REF] investigating the different ways to adopt RFEH in the wireless networks can be classified broadly into two categories. The first category is related to the design of RFEH circuits using single band, dual band or broadband antennas and their integration with CR terminals. Detailed discussion on these issues is out of the scope of this paper and reader can refer to [START_REF] Lu | Dynamic Spectrum Access in Cognitive Radio Networks with RF Energy Harvesting[END_REF][START_REF] Lu | Wireless Networks with RF Energy Harvesting: A Contemporary Survey[END_REF][START_REF] Hoang | Opportunistic Channel Access and RF Energy Harvesting in Cognitive Radio Networks[END_REF][START_REF] Nintanavongsa | Design Optimization and Implementation for RF Energy Harvesting Circuits[END_REF][START_REF] Mohjazi | RF-Powered Cognitive Radio Networks: Technical Challenges and Limitations[END_REF] for more information. To summarize, the amount of harvested RF energy increases with the increase in the strength of the RF signal, RFEH duration as well as harvesting bandwidth [START_REF] Lu | Dynamic Spectrum Access in Cognitive Radio Networks with RF Energy Harvesting[END_REF][START_REF] Lu | Wireless Networks with RF Energy Harvesting: A Contemporary Survey[END_REF][START_REF] Hoang | Opportunistic Channel Access and RF Energy Harvesting in Cognitive Radio Networks[END_REF][START_REF] Nintanavongsa | Design Optimization and Implementation for RF Energy Harvesting Circuits[END_REF][START_REF] Mohjazi | RF-Powered Cognitive Radio Networks: Technical Challenges and Limitations[END_REF]. Furthermore, among various harvesting approaches including solar, thermal etc., RFEH circuits have higher number of circuit elements with are present in the AFE of the existing CR terminals making RFEH a preferred choice from architecture perspective.

The second category deals with the design of DMPs for the decentralized network consisting of RFEH enabled CRs. Due to additional RFEH mode, DMPs for RFEH enabled CRs involve more complex decision making compared to DMPs discussed in Section 2.1. In [START_REF] Zheng | Information and Energy Cooperation in Cognitive Radio Networks[END_REF], two schemes, namely time-splitting and power-splitting, have been proposed for information as well as RFEH cooperation between primary users and CRs. In the time-splitting scheme, CR and RFEH modes are separated in time while in the power-splitting approach, CRs can simultaneously extract RF energy and information from the received PU signals. In turn, CRs must help primary users for data relaying or offloading. Though power-splitting scheme is efficient in theory, it has drawbacks such as hardware complexity, incompatibility with existing circuits and unsuitable for out-of-band RFEH. The usefulness of these schemes for large decentralized networks and ambient RFEH is contentious.

The DMP in [START_REF] Park | Optimal Mode Selection for Cognitive Radio Sensor Networks with RF Energy Harvesting[END_REF] deals with the switching actions between CR and RFEH modes and the goal is to maximize overall throughput of the decentralized network. However, subband statistics are known in advance which may not be true for the decentralized networks. In [START_REF] Qin | Throughput Analysis for Compressive Spectrum Sensing with Wireless Power Transfer[END_REF], a new frame structure for RFEH enabled CRs to perform harvesting, spectrum sensing and data transmission in single time slot is proposed. Such approach may not be feasible with existing RFEH circuit due to poor efficiency. The optimization based policy in [START_REF] Lu | Wireless Networks with RF Energy Harvesting: A Contemporary Survey[END_REF] and learning based policy in [START_REF] Hoang | Opportunistic Channel Access and RF Energy Harvesting in Cognitive Radio Networks[END_REF] consider the subband selection for RFEH enabled CRs in the decentralized networks. Though both policies are proved to be optimal, they are designed for networks with single CR which means that the collisions among CRs, a significant factor affecting the subband selection and hence, total regret of the practical decentralized networks, has not been considered. An optimization based DMPs are computationally complex and may not be feasible for battery-operated resource-constrained CR terminals. Hence, a learning algorithm based DMP needs to be investigated. Furthermore, in the existing DMPs, it is assumed that θ ≈ 1 (Here, θ denotes the number of successful RFEH time slots needed for harvesting the RF energy sufficient for single data transmission over the same bandwidth). Considering a feasible range of θ from 4 to 15 for existing RFEH circuits [START_REF] Nintanavongsa | Design Optimization and Implementation for RF Energy Harvesting Circuits[END_REF][START_REF] Mohjazi | RF-Powered Cognitive Radio Networks: Technical Challenges and Limitations[END_REF], with the likelihood of smaller θ in near future, the effect of θ on the total regret of the decentralized networks has not been analyzed yet in the literature.

Proposed CR-RFEH Decision Making Policy

Consider a feasible architecture of RFEH enabled CR as shown in Fig. 2 The total SSC is the product of the number of subband switchings and the penalty due to single subband switching. The number of subband switchings depend on the DMP while single subband switching penalty can be further divided into two parts: 1) Penalty due to the hardware (mainly analog and digital filters) reconfiguration, and 2) Communication cost since the receiver needs to be informed about the change in subband. Taking into account inter-standard as well as intra-standard bandwidth variations and limited flexibility of the analog filters compared to its digital counterpart, the analog front-end performs only coarse filtering task and the digital front-end is responsible for the stringent subband filtering. Furthermore, in multi-standard environment, input signal comprises of multiple communication standards each with distinct bandwidth specifications. In such cases, the digital filtering stage needs variable digital filters which offer complete control over the bandwidth and center frequencies over the entire Nyquist band. Hence, a hardware reconfiguration penalty is mainly contributed by the digital frontend (and hence, variable digital filters) since the analog front-end is fixed. Reader can refer to [START_REF] Ambede | A Low Complexity Uniform and Non-Uniform Digital Filter Bank based on an Improved Coefficient Decimation Method for Multi-Standard Communication Channelizers[END_REF][START_REF] Löwenborg | Minimax Design of Adjustable Bandwidth Linear-Phase FIR Filters[END_REF][START_REF] Darak | Linear Phase VDF Design with Unabridged Bandwidth Control over the Nyquist Band[END_REF][START_REF] Darak | Efficient Implementation of Reconfigurable Warped Digital Filters with Variable Lowpass, Highpass, Bandpass and Bandstop Responses[END_REF][START_REF] Darak | Low Complexity and Efficient Dynamic Spectrum Learning and Tunable Bandwidth Access for Heterogeneous Decentralized Cognitive Radio Networks[END_REF] for more details about variable digital filters. Among them [START_REF] Ambede | A Low Complexity Uniform and Non-Uniform Digital Filter Bank based on an Improved Coefficient Decimation Method for Multi-Standard Communication Channelizers[END_REF][START_REF] Löwenborg | Minimax Design of Adjustable Bandwidth Linear-Phase FIR Filters[END_REF][START_REF] Darak | Linear Phase VDF Design with Unabridged Bandwidth Control over the Nyquist Band[END_REF][START_REF] Darak | Efficient Implementation of Reconfigurable Warped Digital Filters with Variable Lowpass, Highpass, Bandpass and Bandstop Responses[END_REF][START_REF] Darak | Low Complexity and Efficient Dynamic Spectrum Learning and Tunable Bandwidth Access for Heterogeneous Decentralized Cognitive Radio Networks[END_REF], variable digital filters in [START_REF] Löwenborg | Minimax Design of Adjustable Bandwidth Linear-Phase FIR Filters[END_REF][START_REF] Darak | Linear Phase VDF Design with Unabridged Bandwidth Control over the Nyquist Band[END_REF] offer lowest subband switching penalty due to fewer number of variable multipliers. The implementation complexity of the VDF in [START_REF] Darak | Linear Phase VDF Design with Unabridged Bandwidth Control over the Nyquist Band[END_REF] is lower than [START_REF] Löwenborg | Minimax Design of Adjustable Bandwidth Linear-Phase FIR Filters[END_REF].

RFEH circuit:

The RFEH circuit, which receives filtered RF signal from the RF front-end, consists of impedance matching unit, voltage multiplier and capacitor [START_REF] Lu | Dynamic Spectrum Access in Cognitive Radio Networks with RF Energy Harvesting[END_REF][START_REF] Lu | Wireless Networks with RF Energy Harvesting: A Contemporary Survey[END_REF][START_REF] Hoang | Opportunistic Channel Access and RF Energy Harvesting in Cognitive Radio Networks[END_REF][START_REF] Nintanavongsa | Design Optimization and Implementation for RF Energy Harvesting Circuits[END_REF][START_REF] Mohjazi | RF-Powered Cognitive Radio Networks: Technical Challenges and Limitations[END_REF]. To integrate RFEH circuits with CR terminals, analog front-end is divided into the RF front-end and IF front-end.

Decision making policy:

The proposed CR-RFEH DMP comprises of:

(a) Subband detection block: To check whether the filtered subband is vacant or occupied. It is assumed as an ideal i.e., no detection errors, for analysis and simulation purpose. However, experimental results using the proposed USRP testbed in Section 5 take into account detection errors.

(b) Online learning algorithm: To accurately learn subband statistics for CR and RFEH modes.

(c) Subband access scheme: To orthogonalize CRs to optimum vacant subbands in CR mode.

(d) Mode selection scheme: To decide when to switch from the CR to the RFEH mode and vice-versa.

Next, system model for the decentralized network is presented.

System Model

Consider the slotted network consisting of multiple PUs and M RFEH enabled CRs. All the frequency and bandwidth specifications mentioned hereafter are normalized with respect to half of the sampling frequency. Let the desired vacant bandwidth of CRs be B v (k), k ∈ {1, 2, ..., M } and the bandwidth of wideband input signal is divided into N uniform subbands of bandwidth, B cmin (=1/N ). The assumptions made in this paper are as follows:

1. Infrastructure less decentralized network where all CRs employ the same DMP but do not exchange any information with other CRs.

2. Each CR operates either in CR mode or RFEH mode in a given time slot.

The mode selection scheme independently chooses the mode for each CR.

3. CR can sense only one subband in a time slot.

4. Subbands occupied by CRs are not useful for RFEH due to lower levels of RF energy.

5. Subband statistics are fixed, stationary and unknown to CRs.

Consider i th subband switches its state from being vacant to occupied and vice-versa according to a discrete Markov process with the probabilities of p i vo and p i ov , respectively. Then, using Markov chain analysis, steady-state probabilities of subband being vacant, denoted as P vac (i), are given by [START_REF] Park | Optimal Mode Selection for Cognitive Radio Sensor Networks with RF Energy Harvesting[END_REF][START_REF] Hoang | Opportunistic Channel Access and RF Energy Harvesting in Cognitive Radio Networks[END_REF]

P vac (i) = p i ov p i ov + p i vo , ∀i ∈ {1, 2, ...N } (1)
In CR mode, chosen subband(s) are filtered by the digital filtering stage and checked for vacancy in the detection stage. If vacant, it is assumed that the CR transmits over that subband(s). When multiple CRs transmit on the subbands which are non-orthogonal in frequency, collision occurs leading to failed transmission. When no such collision occurs, it is assumed that the CR transmits successfully. Then, the reward of k th CR at time t, ∇(k, t), depends on the amount of the data transmitted by CR and is given by

∇(k, t) = Bv(k) Bcmin No collission 0 Collision/RFEH mode (2)
Let S * (t) and S(t) denote the total reward (i.e., throughput) of the network using the genie-aided policy (i.e. the policy where P vac (i), ∀i, are known a priori and a central unit ensures orthogonal optimum subband allocation to CRs in the CR mode) and the decentralized DMP, respectively. Then, total regret, U (t), up to time t is given by Eq. 3 [START_REF] Xu | Decision-Theoretic Distributed Channel Selection for Opportunistic Spectrum Access: Strategies, Challenges and Solutions[END_REF][START_REF] Auer | Finite-time Analysis of the Multiarmed Bandit Problem[END_REF][START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Learning-Stage Based Decentralized Adaptive Access Policy For Dynamic Spectrum Access[END_REF][START_REF] Darak | Efficient Decentralized Dynamic Spectrum Learning and Access Scheme for Multi-standard Multi-user Cognitive Radio Networks[END_REF] and it should be as minimum as possible.

U (t) = S * (t) -S(t) = M k=1 t-1 v=0 [∇ * (k, v) -∇(k, v)] (3)
Let ξ(k, t) be the available battery charge of k th CR at time t. If ξ d is the battery charge consumed by CR for each data transmission, then

ξ(k, t) = ξ(k, t -1) -ξ d (4)
In RFEH mode, RF energy of the chosen subband(s) is harvested by RFEH circuit. Let ξ e be the battery charge from the energy harvested by RFEH circuit over the chosen subband, then

ξ(k, t) = ξ(k, t -1) + ξ e ( 5
)
ξ d = θ • ξ e (6)
where θ represents the number of time slots over which RFEH needs to be done in order to harvest energy required for single data transmission. Parameter, θ, which depends on the available RF energy and efficiency of RFEH circuit, should be as minimum as possible. From the existing RFEH circuit specifications [START_REF] Lu | Dynamic Spectrum Access in Cognitive Radio Networks with RF Energy Harvesting[END_REF][START_REF] Lu | Wireless Networks with RF Energy Harvesting: A Contemporary Survey[END_REF][START_REF] Hoang | Opportunistic Channel Access and RF Energy Harvesting in Cognitive Radio Networks[END_REF][START_REF] Nintanavongsa | Design Optimization and Implementation for RF Energy Harvesting Circuits[END_REF][START_REF] Mohjazi | RF-Powered Cognitive Radio Networks: Technical Challenges and Limitations[END_REF], it can be safely assumed that θ ≥ 4. Note that harvesting and data transmission bandwidths are assumed to be equal in Eq. 6. In Table 1, all the notations used in this paper are given along with their definitions.

∇ d (k, t) Reward of k th CR in the CR mode ∇ rf (k, t) Reward of k th CR in the RFEH mode ∇ r (k, t) Reward of k th CR using the rank R(k) T d (k, i, t)
No. of times the subband i is chosen by k th CR in the CR mode

T rf (k, i, t)
No. of times the subband i is chosen by k th CR in the RFEH mode T r (k, i, t)

No. of times the rank i is chosen by k th CR in the CR mode r d (k, i, t)

Total reward of k th CR using the subband i in the CR mode r rf (k, i, t)

Total reward of k th CR using the subband i in the RFEH mode r r (k, i, t)

Total reward of k th CR using the rank i in the CR mode c r (k, i, t)

No. of collisions experienced by k th CR using the rank i X d ( Probability of CR to remain in the RFEH mode

Proposed CR-RFEH DMP: Subband Statistic Estimation

Various tasks of the proposed CR-RFEH DMP are shown in Fig. 3. Each time slot is divided into two sub-slots, t s and t d . In slot t s , mode selection, sub-band selection and subband statistics update take place while the data transmission (in CR mode) and energy harvesting (in RFEH mode) over the chosen subband is done in slot, t d .

t d t s t d t s t d t s

Mode Selection Scheme (Section 3. First contribution of this paper is the selection of Bayesian MAB algorithm for subband statistic estimation. To begin with, Bayesian MAB algorithm is discussed followed by its advantages over other algorithms. The basic idea of Bayesian MAB algorithms is to assume some prior distribution on the probability statistics of each subband (e.g. uniform prior) and at any time slot, select the subband according to its posterior probability of being optimum [START_REF] Kaufmann | Thompson Sampling: An Asymptotically Optimal Finite Time Analysis[END_REF][START_REF] Agrawal | Further Optimal Regret Bounds for Thompson Sampling[END_REF][START_REF] Russo | An Information-Theoretic Analysis of Thompson Sampling[END_REF]. Among them, TS algorithm [START_REF] Kaufmann | Thompson Sampling: An Asymptotically Optimal Finite Time Analysis[END_REF][START_REF] Agrawal | Further Optimal Regret Bounds for Thompson Sampling[END_REF][START_REF] Russo | An Information-Theoretic Analysis of Thompson Sampling[END_REF] has been chosen for the proposed CR-RFEH DMP and its usefulness for the subband statistic estimation is given in Algorithm 1.

In the beginning, all subbands are chosen once as shown in steps 1-2. Then, TS algorithm calculates the quality index for each subband based on the past sensing events as shown in steps 5-6. Here, the horizon, H, represents the total number of time slots for which the DMP is executed and Beta represents the complete beta function, i.e., Euler integral of the first kind. Then, CR with the rank R(k) selects the subband having R(k) th highest quality index. In Algorithm 1, parameter T (k, i, t) denotes the number of time slots up to t slots where the subband i is sensed by k th CR and r(k, i, t) denotes the number of times out of T (k, i, t) slots, the chosen subband i is observed as vacant. The advantages of TS algorithm over other MAB algorithms can be summarized as follows:

1. MAB algorithm is said to be optimal in the subband selection task if [START_REF] Kaufmann | Thompson Sampling: An Asymptotically Optimal Finite Time Analysis[END_REF][START_REF] Agrawal | Further Optimal Regret Bounds for Thompson Sampling[END_REF][START_REF] Russo | An Information-Theoretic Analysis of Thompson Sampling[END_REF]

lim inf t→∞ E[T (k, i, t)] ln t ≥ 1 K(P vac (i), P vac (i * ))
, ∀i

where K(p, q) denotes the Kullback-Leibler divergence factor. The algorithm which satisfies Eq. 7 with equality is said to be asymptotically optimal. Frequentist approach based UCB algorithms, optimization based KL-UCB algorithm [START_REF] Russo | An Information-Theoretic Analysis of Thompson Sampling[END_REF], Bayesian approach based TS and Bayes-UCB algorithms [START_REF] Kaufmann | Thompson Sampling: An Asymptotically Optimal Finite Time Analysis[END_REF] have been showed to be asymptotically optimal for Bernoulli rewards. However, it has been recently proved that the scaling constant and hence, total reward of the Bayesian algorithms is better than that of others [START_REF] Kaufmann | Thompson Sampling: An Asymptotically Optimal Finite Time Analysis[END_REF][START_REF] Agrawal | Further Optimal Regret Bounds for Thompson Sampling[END_REF].

2. TS algorithm requires the computation of only one sample from the posterior of subband statistics as shown in Algorithm 1. Its complexity is low compared to optimization problem in KL-UCB algorithm and computation of quantiles in Bayes-UCB algorithm [START_REF] Kaufmann | Thompson Sampling: An Asymptotically Optimal Finite Time Analysis[END_REF][START_REF] Agrawal | Further Optimal Regret Bounds for Thompson Sampling[END_REF][START_REF] Russo | An Information-Theoretic Analysis of Thompson Sampling[END_REF].

3. Bayesian subband selection approach leads to lower SSC than UCB algorithm or its extensions through empirical observations.

To the best of our knowledge, usefulness of TS algorithm, rather any Bayesian MAB algorithm, for the decentralized networks or any other wireless communication applications has not been studied yet. Furthermore, existing TS algorithm needs to be re-designed to take into account tunable bandwidth scenarios, i.e., B v (k) > B cmin . For example, in CR mode, the number of subband choices available to CRs, N d (k), are given by

N d (k) = N + 1 - B v (k) B cmin (9)
When B v (k) = B cmin , ∀k, all the subbands are orthogonal in frequency. Thus, CRs with distinct ranks do not experience any collision [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Learning-Stage Based Decentralized Adaptive Access Policy For Dynamic Spectrum Access[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF]. How-ever, when B v (k) > B cmin , ∀k, each subband is orthogonal to all other subbands except adjacent ( Bv(k) Bcmin -1) number of subbands. Thus, CRs with distinct ranks may also experience collision due to non-orthogonality of subbands. For exam-295 ple, when N =4, M =2 and B v ={2,2}, then N d (k)=3 ∀k. This means that CRs have three subband choices which are contiguous subbands {1,2}, {2,3} and {3,4}. For example, k th CR with rank R(k)=1 will choose contiguous subband {1,2}. Thus, even though two CRs have distinct ranks, say 1 and 2, they will experience collision due to common subband 2. Furthermore, the probability 300 of collision increases with the bandwidth, B v (k) compared to [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Learning-Stage Based Decentralized Adaptive Access Policy For Dynamic Spectrum Access[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF] where B v (k) = B cmin , ∀k and hence, probability of collision is fixed for all CRs. By taking these factors into account, tunable TS (TTS) algorithm is proposed for CRs in the CR mode and is given below in Algorithm 2.

5.

for to do 6.

Compute quality index for all bands: 7. 8. 9. end 10. end In TTS algorithm, the parameter, α k , is introduced to minimize the number of collisions due to non-orthogonality of subbands and the rank, R(k) is obtained using the scheme discussed later in Section 3.3. If the chosen subband,

I d (k, t) is sensed as vacant by k th CR at time t, then ∇ d (k, t) = 1, otherwise ∇ d (k, t) = 0. Based on ∇ d (k, t), r d (k, l k , t) is updated as r d (k, I d (k, t), t) = r d (k, I d (k, t), t -1) + ∇ d (k, t) (10) r d (k, l k , t) = r d (k, l k , t -1) ∀l k = I d (k, t) (11)
On the other hand, if the chosen subband, I d (k, t), is observed as occupied with RF energy higher than the threshold, Γ rf , then ∇ rf (k, t) = 1, otherwise ∇ rf (k, t) = 0. Note that the threshold, Γ rf , depends on the static power consumption of RFEH circuit. Then, r rf (k, l k , t), to be used later in the RFEH mode and analogous to r d (k, l k , t) in the CR mode, is updated as

r rf (k, I d (k, t), t) = r rf (k, I d (k, t), t -1) + ∇ rf (k, t) (12) r rf (k, l k , t) = r rf (k, l k , t -1) ∀l k = I d (k, t) (13)
In this way, the proposed DMP uses single TTS algorithm to learn the subband statistics for the CR as well as RFEH modes. Let X d (k, l k , t) denotes the learned probability of l th k subband being vacant at k th CR. Similarly, X rf (k, l k , t) denotes the learned probability of l th k subband having RF energy higher than Γ rf at k th CR. Then,

X d (k, l k , t) = r d (k, l k , t) T d (k, l k , t) , ∀l k (14) X rf (k, l k , t) = r rf (k, l k , t) T d (k, l k , t) , ∀l k ( 15
)
where

T d (k, l k , t) = t p=1 1 {I d (k,p)=l k } ( 16
)
In the RFEH mode, R(k) = 1, ∀k since there are no collisions among CRs and the task of the TTS algorithm is to identify contiguous set of occupied subbands with higher RF energy. Contiguous bandwidth, B o (k), in the RFEH mode is equivalent to B v (k) in the CR mode but B v (k) is chosen by CR depending on their transmission requirements while B o (k) is fixed and equal to the analog front-end bandwidth, B af e . This is because, the wider the bandwidth, the higher is the harvested RF energy [START_REF] Lu | Dynamic Spectrum Access in Cognitive Radio Networks with RF Energy Harvesting[END_REF][START_REF] Lu | Wireless Networks with RF Energy Harvesting: A Contemporary Survey[END_REF][START_REF] Hoang | Opportunistic Channel Access and RF Energy Harvesting in Cognitive Radio Networks[END_REF][START_REF] Nintanavongsa | Design Optimization and Implementation for RF Energy Harvesting Circuits[END_REF][START_REF] Mohjazi | RF-Powered Cognitive Radio Networks: Technical Challenges and Limitations[END_REF]. Then, the number of choices, N rf , available for the CR in the RFEH mode are given by,

N rf = N + 1 - B af e B cmin ( 17
)
where Then, the task of the TTS Algorithm in the RFEH mode, given in Algorithm 3, is to identify the optimum subband (i.e., the subband with lower probability of being vacant and higher RF energy which will lead to faster RFEH). Similar to the CR mode, r d , r rf , T rf , X d and X rf are updated in each time slot based on the sensing outcome of I rf (k, t). In this way, the TTS algorithm learns from the sensing outcomes in each mode.

N
In this paper, we have assumed that the subband statistics are fixed, stationary and unknown to CRs. This is an underlying assumption to estimate the subband statistics, P vac , based on the past sensing events. Similar assumption has also been made in the existing learning based DMPs [START_REF] Lu | Wireless Networks with RF Energy Harvesting: A Contemporary Survey[END_REF][START_REF] Hoang | Opportunistic Channel Access and RF Energy Harvesting in Cognitive Radio Networks[END_REF][START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Learning-Stage Based Decentralized Adaptive Access Policy For Dynamic Spectrum Access[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF][START_REF] Darak | Efficient Decentralized Dynamic Spectrum Learning and Access Scheme for Multi-standard Multi-user Cognitive Radio Networks[END_REF][START_REF] Kaufmann | Thompson Sampling: An Asymptotically Optimal Finite Time Analysis[END_REF][START_REF] Agrawal | Further Optimal Regret Bounds for Thompson Sampling[END_REF][START_REF] Russo | An Information-Theoretic Analysis of Thompson Sampling[END_REF]. The proposed DMP can be extended to the slowly changing subband statistics, i.e., P vac changes by small value after some finite but variable length time window. In such case, the TS algorithm needs to be modified to track the changes in the subband statistics. One approach is to calculate the quality index using the rewards received during the finite time window instead of infinite time window considered in the Algorithm 1 (and hence Algorithms 2 and 3). The length of the time window can be tuned based on the dynamic behavior of the subband statistics. Next, the subband access scheme of the proposed DMP is presented.

Proposed CR-RFEH DMP: Subband Access Scheme

The task of the subband access scheme, when implemented at all CRs in the decentralized network, is to orthogonalize CRs operating in the CR mode. Since there are no collisions among CRs in the RFEH mode, subband access scheme is not needed when the CR is in the RFEH mode. The proposed subband access scheme is based on the extension of our past works [START_REF] Darak | Compute-Efficient Decision Making Policy for D2D Communications and RF Energy Harvesting in Cognitive Radio Networks: Go Bayesian![END_REF][START_REF] Darak | Efficient Decentralized Dynamic Spectrum Learning and Access Scheme for Multi-standard Multi-user Cognitive Radio Networks[END_REF] and involves two sub-tasks: 1) Calculation of the rank, R(k), for each CR, and 2) Calculation of the upper limit of the rank, R m (k). The proposed solutions are discussed in the following sub-sections.

MAB Algorithm Based Rank Calculation

It is well known that the learning based subband selection schemes are superior to the random subband selection schemes [START_REF] Xu | Decision-Theoretic Distributed Channel Selection for Opportunistic Spectrum Access: Strategies, Challenges and Solutions[END_REF][START_REF] Auer | Finite-time Analysis of the Multiarmed Bandit Problem[END_REF][START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Learning-Stage Based Decentralized Adaptive Access Policy For Dynamic Spectrum Access[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF][START_REF] Darak | Efficient Decentralized Dynamic Spectrum Learning and Access Scheme for Multi-standard Multi-user Cognitive Radio Networks[END_REF]. Albeit, randomization based rank assignment approach has been used in the literature [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Learning-Stage Based Decentralized Adaptive Access Policy For Dynamic Spectrum Access[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF][START_REF] Darak | Efficient Decentralized Dynamic Spectrum Learning and Access Scheme for Multi-standard Multi-user Cognitive Radio Networks[END_REF]. For example, in [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF], when collision occurs, corresponding CR randomly selects a new rank. Due to the random selection approach, there is a high probability of CR colliding with other CRs in the next time slot especially when M is large. Thus, a single collision event leads to multiple collisions even after convergence especially when estimated subband statistics are not accurate. To alleviate this problem, a new rank assignment scheme using TS algorithm, replacing the con- Continue with the same rank 12. end The task of the Algorithm 4 is to choose the rank, R(k), for each CR in the CR mode based on the past collision events. This is analogous to the task of identifying optimum subband based on the past sensing events. In Algorithm 4, T r (k, q k , t) denotes the number of time slots out of t where R(k) = q k , q k ∈ {1, 2, .., R m (k)} and c r (k, q k , t) denotes the number of time slots out of T r (k, q k , t) where k th CR experiences collision on the rank, q k . If the data transmission by k th CR at time t using the rank, R(k) is successful, then ∇ r (k, t) = 1. Otherwise, ∇ r (k, t) = 0. Note that ∇ r (k, t) = 1 also means that the CR does not experience any collision. Then,

r r (k, R(k), t) = r r (k, R(k), t -1) + ∇ r (k, t) (18) r r (k, q k , t) = r r (k, q k , t -1) q k = R(k) (19) c r (k, R(k), t) = c r (k, R(k), t -1) + 1 -∇ r (k, t) (20) c r (k, q k , t) = c r (k, q k , t -1) q k = R(k) (21)
In the proposed approach, the use of TS algorithm guarantees orthogonalization of CR in fewer time slots than [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF]. Furthermore, the proposed approach also leads to fewer number of subband switchings (and hence, lower SSC) than [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF]. The gain is significant for the tunable bandwidth access considered in this paper. This is because, the randomization based approach in [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF] leads to higher SSC when B v (k) > B cmin as such CRs have higher probability of experiencing collisions. In the proposed scheme, first if loop in the Algorithm 4 eliminates this unfairness since the rank is updated only when the number of collisions past the recent rank assignment becomes equal to ( Bv(k) Bcmin ). Another problem in existing learning algorithm based DMPs [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF] is that the number of collisions increases significantly as the number of CRs, M increases. To alleviate this problem, the upper limit on the rank is made tunable as discussed below.

Upper Limit on Rank, R m

In [START_REF] Darak | Efficient Decentralized Dynamic Spectrum Learning and Access Scheme for Multi-standard Multi-user Cognitive Radio Networks[END_REF], we showed that the tunable upper limit of the rank leads to significant improvement in the performance of the randomization based subband access scheme when compared to the fixed subset size based schemes in [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF]. Taking it further, tunable upper limit on the rank, R m (k), ∀k for the proposed TS algorithm based subband access scheme is presented here. The lower and upper bounds on R m (k), denoted by R ml (k) and R mu (k) respectively, for k th CR, are given by,

R ml (k) = M u=1 ( Bv(u) Bcmin ) ( Bv(k) Bcmin ) , ∀k (22) R mu (k) = N d (k) = N + 1 - B v (k) B cmin , ∀k (23)
For example, when N =6, M =2, B v (1) = B cmin and B v (2) = 2 • B cmin , first CR has six subband options, each of bandwidth B cmin while second CR has five options, each of bandwidth 2 • B cmin . Then, R ml ={3,2} and R mu ={6,5}. Note that R ml (k) = M for B v (k) = B cmin ∀k, similar to the uniform bandwidth access considered in [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF]. The optimum value of R m , where R mu ≥ R m ≥ R ml , depends on the lower and upper bounds on the regret, U (t). Such bounds for the tunable bandwidth case, using the bounds for uniform bandwidth case in [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF], are given as

U (t) ≥ M k=1 Rmu(k) v=R ml (k)+1 Δ(k * , v) • E(T (k, v, t)) ( 24
)
U (t) ≤ M k=1 P vac (k * ) Rmu(k) v=R ml (k)+1 E[T (k, v, t)] + R ml (k) u=1 E[c d (k, u, t)] ( 25
)
where c d (k, u, t) denotes the number of time slots up to t slots where k th CR experiences collision while transmitting on the u th subband of bandwidth B v (k),

P vac (k * ) denotes k th highest value of P vac and Δ(k * , v) = P vac (k * ) -P vac (v)
where k * is the subband of bandwidth, B v (k), with k th highest value of P vac .

The upper bound on U (t), given by Eq. 25, is of significant importance in order to reduce the number of collisions among CRs and hence, SSC. Eq. 25 consists of two terms: 1) First term represents the regret incurred due to selection of sub-optimum subbands, and 2) Second term represents the regret incurred due to collisions when optimum subbands are selected. For example, when

N = 4, M = 2, B v = {B cmin , B cmin }
and assuming that P vac (1) > P vac (2) > P vac (3) > P vac (4), the first term in Eq. 25 represents the regret due to selection of third and forth subbands and the second term in Eq. 25 represents the regret due to collisions when CRs select first and second subbands.

If R m (k) ≈ R ml (k) as in [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF], then the second term is dominant leading to the higher number of collisions among CRs and hence, higher regret, U (t), as well as SSC. On the other hand, if R m (k) ≈ R mu (k), the first term is dominant leading to fewer number of collisions as well as lower SSC but it also leads to the higher regret, U (t). Since, the loss in U (t) due to the second term is more significant than the same due to the first term in the CR mode, a new method to make R m (k) ∀k tunable is proposed as follows:

1. For each integer value from R ml (k) to R mu (k), total predicted reward,

r t (k, d k , t) ∀k and d k ∈ {1, 2, .., (R mu (k) -R ml (k) + 1)}, at any time t is given by, r t (k, d k , t) = R ml (k)+d k -1 u=1 Xd (k, u, t) • R ml (k) R ml (k) + d k -1 (26)
where Xd (k, u, t) ∀u is the estimated subband availability statistics, X d (k, u, t) ∀u in Eq. 14, arranged in the descending order, i.e., Xd (k, 1,

t) ≥ Xd (k, 2, t) ≥ • • • ≥ Xd (k, R mu (k), t).
It can be observed from Eq. 26 that r t (k, d k , t) decreases as d k increases due to the selection of non-optimum subbands.

2. For each integer value from R ml (k) to R mu (k), total predicted number of collisions, c t (k, d k , t) ∀k and ∀d k , at any time t is given by

c t (k, d k , t) = 1 - (R ml (k) + d k -1)! (d k -1)! • (R ml (k) + d k -1) R ml (k) (27)
Eq. 27 indicates that the number of collision decreases as d k increases.

3. Finally, R m (k) is selected using the value of d k that leads to lower regret, U (t) and is given by

R m (k) = R ml (k) + s -1 (28)
where s = arg max

d k r t (k, d k , t) • t -c t (k, d k , t) • t ( 29
)
Each CR independently calculates the value of R m (k). To minimize computational complexity, the value of R m (k) is updated only when corresponding CR updates its rank as discussed in Section 3.3.1. Next, the mode selection scheme is presented.

Proposed CR-RFEH DMP: Mode Selection Scheme

In the CR-RFEH DMP, each CR independently takes the mode selection decisions based on the available battery charge, ξ p (k, t) (0 ≤ ξ p (k, t) ≤ 100, ∀k). Consider P d (ξ p (k, t)) (or P rf (ξ p (k, t)) denote the probability of k th CR to remain in the CR mode (or RFEH mode, respectively) given that it is currently in the CR mode (or RFEH mode, respectively) and battery is ξ p (k, t)% charged. Then,

P d (ξ p (k, t)) = 1 -P (ξ p (k, t), 1 , η 1 ) (30)
P rf (ξ p (k, t)) = P (ξ p (k, t), 2 , η 2 ) (31)
P (ξ p (k, t), , η) = S(ξ p (k, t) -) • |ξ p (k, t) -| 1 η + | 1 | 1 η S(1 -) • |1 -| 1 η + | | 1 η ( 32
)
where S denotes the signum function, is the ξ p (k, t) at which there is a step change in P and η decides the slope of this step. Ideally, the total reward (i.e. throughput) of the DMP should be independent of { 1 , 2 } values i.e. the mode switching frequency since many on-field factors such as location of CR, priority of data transmission, available battery charge, type of service, future services and location etc. might influence the mode switching decision. Intuitively, this is true for the proposed CR-RFEH DMP since subbands statistics are assumed as stationary and the subband selection in each mode is based on the past sensing and collision events which means the mode switching events do not affect the performance of the DMP. This is verified using the simulation results presented in the Section 4 where the effects of { 1 , 2 , θ} on the reward and SSC of different DMPs are analyzed.

Simulation Results

In this section, we conduct extensive simulations to evaluate and compare the total reward and SSC of the proposed DMP with the UCB+ρ rand DMP in [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF] and three other DMPs designed by replacing the UCB algorithm in [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF] with TS, KL-UCB and Bayes-UCB algorithms, respectively. The mode selection scheme is same as Section 3.4 5 that the proposed DMP outperforms other DMPs in terms of average reward for all P vac distributions. Average reward is an indication of collision free data transmission opportunities offered by corresponding DMPs and hence, higher the reward, high is the throughput of the decentralized network. Fig. 5 shows that the average reward of the proposed DMP is significantly better than that of TS+ρ rand DMP which validates the superiority of the proposed TTS algorithm. Also, the proposed DMP outperforms other DMPs for each of the P vac distributions which validates the superiority of the proposed TS algorithm based subband access scheme over the randomization based schemes [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF][START_REF] Liu | Distributed Learning in Multi-Armed Bandit with Multiple Players[END_REF][START_REF] Zandi | Learning-Stage Based Decentralized Adaptive Access Policy For Dynamic Spectrum Access[END_REF][START_REF] Zandi | Decentralized spectrum learning and access adaptive to channel availability distribution in primary network[END_REF][START_REF] Darak | Efficient Decentralized Dynamic Spectrum Learning and Access Scheme for Multi-standard Multi-user Cognitive Radio Networks[END_REF].

Next, RFEH enabled CRs in the decentralized networks are considered. RFEH circuit parameters are θ=8 (i.e., CR needs to perform RFEH for eight time slots over the bandwidth B cmin in order to harvest energy required for single data transmission over the same bandwidth B cmin ) and B af e = 3• B cmin . Efficiency of RFEH circuits (i.e. the ratio of output DC power to the input RF power) varies non-linearly with input RF power and are taken from Powercast RFEH circuit specifications [30]. The parameters of the mode selection scheme, discussed in Section 3.4, are { 1 , 2 } = {20, 30} and {η 1 , η 2 } = {10, 80}. It is assumed that the battery of each CR is 50% charged in the beginning. For M =5 and B v ={1/16, 3/16, 2/16, 2/16, 1/16}, the plots of average reward, S(t), in percentage vs. time slots, t, for Case 1, Case 2 and Case 3, are shown in Figures 6a, 6b and 6c, respectively. It is apparent from Fig. 6 that the proposed DMP offers higher reward (i.e., throughput) for all P vac distributions. The decrease in reward in Fig. 6 after first peak is due to CR switching to the RFEH mode after utilizing initial 50% battery. The gain in reward is lower compared to Fig. 5 In Fig. 8, the average SSC in % of different DMPs are compared with respect to that of UCB+ρ rand DMP in [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF]. The SSC of the proposed DMP is very low which makes it energy efficient. Similar to Fig. 7, savings in SSC increases as θ and B af e increases. In summary, the total reward and SSC of the Bayesian algorithms based DMPs are significantly better than frequentist approach algorithm based DMPs. Also, the proposed DMP not only offers superior performance as shown in Figures 5,6, 7 and 8 but also has lower computational complexity as discussed in [START_REF] Kaufmann | Thompson Sampling: An Asymptotically Optimal Finite Time Analysis[END_REF][START_REF] Agrawal | Further Optimal Regret Bounds for Thompson Sampling[END_REF][START_REF] Russo | An Information-Theoretic Analysis of Thompson Sampling[END_REF].

θ =15 θ =8 θ =4 β afe =β cmin { 1 , 2 }= {0.2,0.8} θ =15 θ =8 θ =4 θ =15 θ =8 θ =4 { 1 , 2 }= {0.
θ =4 θ =8 θ =16 θ =4 θ =8 θ =16 { 1 , 2 }={0.

Proposed USRP Testbed

In this section, the functionality of DMPs is validated on the proposed USRP testbed using real radio signals and non-ideal spectrum detector. The proposed testbed, shown in Fig. 9, is an extension of our testbed in [START_REF] Darak | Is Bayesian Multi-armed Bandit Algorithm Superior?: Proof-of-Concept for Opportunistic Spectrum Access in Decentralized Networks[END_REF]. It consists of two units: 1) Left hand side unit is a primary user traffic generator, and 535 2) Right hand side unit acts as the DMP of CRs. The reader may refer to [START_REF] Darak | Is Bayesian Multi-armed Bandit Algorithm Superior?: Proof-of-Concept for Opportunistic Spectrum Access in Decentralized Networks[END_REF] for implementation details.

Primary User Traffic Generator

The main task of this unit is to generate the primary user traffic based on the given subband statistics. The chosen design environment for the primary user traffic generation is the GNU Radio Companion (GRC) and the hardware platform is made of the USRP from Ettus Research. The data modulation used is a differential QPSK modulation with Gray encoding and sub-carrier mapping is done using the OFDM. For the experimental results presented in this paper, the number of sub-carriers, center frequency and transmission bandwidth are 256, 433.5 MHz and 1 MHz, respectively. The center frequency of 433.5 MHz is carefully chosen to avoid the interference to the real licensed users while the transmission bandwidth depends on the analog-front end bandwidth of the USRP. Depending on the number of subbands, N , the number of carriers in each subband are calculated. For example, when N =8, each subband has 32 carriers. For demonstration purpose, the time slot duration is one second so that it can be followed by human eye. However, it can be reduced to the order of milliseconds and will have no direct effect on the performance of DMPs.

Secondary User with Decision Making Policy

The chosen design environment for the CR terminal is Matlab/Simulink and USRP from Ettus Research. The USRP is tuned to receive the signal of bandwidth 1 MHz centered at 433.5 MHz. The received signal is then down-sampled, digitized and passed to the DMP implemented using the Matlab/Simulink. The MAB algorithm of the DMP selects one subband in each time slot. The chosen subband is then sensed using the energy detector. Note that the energy detector is not ideal and sensing errors may occur [START_REF] Bahamou | Noise Uncertainty Analysis of Energy Detector: Bounded and Unbounded Approximation Relationship[END_REF]. In the past works, sensing is assumed as perfect which is not true in real radio conditions. Thus, the proposed testbed with non-ideal spectrum detector enables the study of the performance of DMPs in the presence of sensing errors. In case of multiple CRs, each CR is independently implemented in the Simulink with their respective DMP. For the RFEH mode, the RF energy of the chosen subband is converted to the corresponding battery capacity as discussed before in Section 3.

Experimental Results

In this section, we validate the functionality of DMPs on the proposed USRP testbed. Due to the limited memory and delay constraints of USRP and Matlab, the performance comparisons of only two DMPs, UCB+ρ rand [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF] and the proposed DMP are presented. Consider N =16 and B=1 MHz, then B cmin =62.5 KHz. Each numerical result reported hereafter is the average of values obtained over 15 independent experiments on the USRP testbed and each experiment consider a time horizon of 1000 iterations i.e. 1000 time slots for each CR and one time slot corresponds to one second. It is assumed that all CRs employ the same DMP but do not exchange any information with others.

First, CRs with unlimited battery capacity are considered which means that the RFEH is not needed. Fig. 10a shows the average reward (i.e., throughput), S(t), in percentage for two DMPs w.r.t. that of genie-aided DMP. Note that all parameters are same as those considered in Fig. 5. Similarly, CRs with the RFEH capability are considered in Fig. 10b and all the parameters are same as those considered in Fig. 6. In Fig. 10a and Fig. 10b, solid lines and dotted lines correspond to Case 1 and Case 2, respectively. It can be observed that the proposed DMP offers 10-15% higher reward (i.e., throughput) over UCB+ρ rand DMP in [START_REF] Anandkumar | Distributed Algorithms for Learning and Cognitive Medium Access With Logarithmic Regret[END_REF]. Simulation and experimental results on the USRP testbed validate the superiority of the proposed DMP over existing DMPs.

Conclusions

In this paper, a new decision making policy (DMP) for the opportunistic spectrum access based Cognitive Radios (CRs) with the radio frequency energy harvesting (RFEH) capabilities is proposed. The contributions are the design of Bayesian approach based tunable Thompson sampling algorithm for the CR and RFEH mode, tunable bandwidth subband access scheme using the TS algorithm and the mode selection scheme. Extensive simulations and experimental results using the real radio signal showed the superiority of the proposed DMP over existing DMPs in terms of the throughput of the decentralized network and subband switching cost. Furthermore, lower subband switching cost and lower computational complexity make the proposed DMP suitable for the resourceconstrained battery operated CR terminals.

Figure 1 :

 1 Figure 1: A decentralized wireless network with RFEH enabled CR terminals and RF energy sources.

Figure 2 : 1 .

 21 Figure 2: A feasible architecture of RFEH enabled CR terminal.

Figure 3 :

 3 Figure 3: Different tasks of the proposed CR-RFEH policy

Algorithm 2 :

 2 Subband selection for k th CR in CR mode based on their estimated statistics using

  rf =N when B af e = B cmin and N rf < N when B af e > B cmin . Note that B af e is an integer multiple of B cmin . Let m ∈ {1, 2, .., N rf }. Then, m th choice consists of contiguous subbands from F (m) = m ∈ {1, 2, ..., N } to L(m) = m + B af e -1 ∈ {1, 2, ..., N }. For example, when N =4 and B af e = 2 • B cmin , then N RF =3, F = {1, 2, 3} and L = {2, 3, 4}.

  The illustrative plots of P d and P rf for the different combinations of { 1 , 2 , η 1 , η 2 } are shown in Fig. 4. P d decreases while P rf increases when CR is in the CR mode. A step change in the values of P d and P rf can be seen at 1 and 2 , respectively. Here, η 1 and η 2 control the values of P d and P rf , respectively, at battery charges in the neighborhood of 1 and 2 , respectively.

Figure 4 :

 4 Figure 4: {P d , P rf } vs. battery charge for different combinations of { 1 , 2 , η 1 , η 2 }.

Figure 5 :

 5 Figure 5: Average reward, S(t) in % vs. time slot, t, for different Pvac distributions i.e., (a) Case 1, (b) Case 2, and (c) Case 3 in the conventional CR mode with no need of RFEH (only CR mode).

Figure 6 :

 6 Figure 6: Average reward, S(t) in % vs. time slot, t, for different Pvac distributions i.e., (a) Case 1, (b) Case 2, and (c) Case 3 for the decentralized CRNs with RFEH enabled CRs having limited battery capacity.

Figure 7 :

 7 Figure 7: Comparisons of average S(t) in % of different DMPs with respect to the proposed CR-RFEH DMP.

Figure 9 :

 9 Figure 9: Proposed USRP based testbed for analyzing the performance of DMPs using real radio signals and non-ideal spectrum detectors.

. It mainly consists of RF Front-End ADC Digital Front-End Digital Signal Processing RFEH Unit Anti-aliasing Filter Mixer Sample Rate Convertor Digital Down Convertor Digital Filtering Impedence Matching Voltage Multiplier Capacitor Blocks such as Modulation, Demodulation etc. Detectors Decision Making Policy Sample Rate Convertor Blocks such as Modulation, Control signals Analog filtering Low Noise Amplifier IF Front-End RF Front-End RFEH Unit Mixer Impedence Matching Voltage Multiplier Capacitor Anti-aliasing Filter Low Noise Amplifier Analog filtering IF Front-End Spectrum Bands Analog Front-End

Table 1 :

 1 Notations and definitions

	Notations Definitions
	M	No. of CRs in the decentralized network
	N	No. of subbands of bandwidth B cmin
	N d (k)	No. of subbands of bandwidth B v (k)
	N rf	No. of subbands of bandwidth B af e
	B cmin	Smallest subband bandwidth
	B af e	Analog front-end bandwidth of CR terminal
	B v (k)	Desired bandwidth of k th CR in CR mode
	B o (k)	Desired bandwidth of k th CR in RFEH mode
	P vac (i)	Probability of vacancy of subband i
	R(k)	Rank of k th CR
	I

d (k, t) Subband chosen by k th CR in the CR mode I rf (k, t) Subband chosen by k th CR in the RFEH mode

Algorithm 1: Original TS algorithm replacing UCB algorithm in DMP in [14] for k th CR

	Parameters:
	Output:
	1. if any band not chosen once
	2.		s.t.
	3. else
	4.	for	to do
	5.	Compute quality index for all bands:
	6.
	7.
	8.	end
	9. end
		where
			i

* = arg max i P vac (i)

Algorithm 3: Subband selection for k th CR in RFEH mode based on their estimated statistics using TTS algorithm

	ventional random rank assignment scheme [16-20], is proposed and given in
	Algorithm 4.
	Algorithm 4:
	Parameters:
	Output:
	1. if any band not chosen once
	2.		s.t.
	3. else
	4.	for	to do
	5.	Compute quality index for all bands:
	6.
	7.
	8.	end
	9. end

Rank selection for k th CR in CR mode based on past collisions statistics and TS algorithm

	Parameters:
	Output:
	1. if
	2.	Choose the rank, R(k) randomly in the beginning
	3. else
	4. if
	5.	for	to do
	6.	Compute quality index for all ranks:
	7.
	8.
	9.	end
	10. else
	11.

  for all DMPs. Consider N =16 (i.e., B cmin =1/16), M ∈ {3, 4, ..12} and three different distributions of P vac which are Case 1: P vac :-[.05 .10 .15 .20 .25 .30 .35 .40 .45 .50 .55 .60 .65 .70 .75 .80] Case 2: P vac :-[.95 .90 .85 .80 .75 .70 .65 .60 .55 .50 .45 .40 .30 .20 .10 .05] Case 3: P vac :-[.45 .40 .15 .01 .20 .20 .05 .50 .60 .10 .20 .40 .30 .80 .25 .15]It is assumed that all CRs employ the same DMP but do not exchange any information with others. Each numerical result reported hereafter is the average of the values obtained over 50 independent experiments and simulations consider a time horizon of 50000 iterations. (t) is the average reward of the genie-aided policy. It is apparent from Fig.

	To begin with, conventional CR based OSA in the decentralized network
	is considered. In such scenario, CR has unlimited battery capacity which
	means that the RFEH is not needed and CR has only one mode of opera-
	tion i.e., CR mode. For M =5 and B v ={1/16, 3/16, 2/16, 2/16, 1/16}, the
	plots of average reward, S(t), in percentage vs. time slots, t, for Case 1,
	Case 2 and Case 3, are shown in Figures 5a, 5b and 5c, respectively. Here,
	S(t) % = [100 • (S

* (t) -S(t))/S * (t)] where S *

  since CRs spent substantial amount of time in the RFEH mode due to the higher value of θ considered in the simulations.Next, 50 different combinations of M ∈ {3, 4, .., 12}, B v (1/2) and P vac ∈ {Case 1, Case 2, Case 3} for each of the thirteen distinct parameter settings are considered. The first setting corresponds to the conventional CR mode with no need of RFEH. For the remaining settings, RFEH enabled CRs with limited battery capacity are considered. These twelve settings are divided into two groups of six settings each with B af e = 3 • B cmin for group 1 and B af e = B cmin for group 2. Group 1 takes into account wider contiguous bandwidth for RFEH while group 2 limits the bandwidth to the smallest possible bandwidth i.e., B cmin . Parameters { 1 , 2 , θ} for these two groups are {0.B cmin compared to when B af e = B cmin . This is due to the proposed Algorithm 4 which identifies the optimum subband of bandwidth, B af e , for the RFEH mode. Lower the value of θ, fewer are the number of time slots for which CRs are in the RFEH mode. As a result, the gain in S(t) increases as θ decreases as shown in Fig.7. It is expected that more efficient RFEH circuits with the lower value of θ (< 4) and multi-band harvesting capability (B af e > B cmin ) will be available in near future for which the proposed DMP offers more than 15% gain in S(t) over existing DMPs.

	2,0.3,4}, {0.2,0.3,8},
	{0.2,0.3,15}, {0.2,0.8,4}, {0.2,0.8,8}, {0.2,0.8,15}, respectively. Other parame-
	ters are same as that of Fig. 6.
	In Fig. 7, the comparison is made between different DMPs based on the av-

erage reward, S(t) in % with respect to S(t) of the proposed DMP. For example, if the height of bar is 80 %, it means that the reward, S(t), of the corresponding DMP is 80 % of that of the proposed DMP. It can be observed that the proposed DMP outperforms other DMPs in all scenarios. The gain in reward, S(t), is higher when B af e = 3 •

2,0.8} { 1 , 2 }= {0.2,0.3} β afe =3*β cmin

	Only CR	θ =4	θ =8	θ =15
	mode, no RFEH	{ 1 , 2 }= {0.2,0.3}

2,0.3} { 1 , 2 }={0.2,0.3} { 1 , 2 }={0.2,0.8} { 1 , 2 }={0.2,0.8} θ =4 θ =8 θ =16 θ =4 θ =8 θ =16 β afe =3*β cmin β afe =β cmin

	Only
	CR
	mode,
	no RFEH
	Figure 8: Comparisons of average SSC in % of different DMPs with respect to the DMP in
	[16].

Acknowledgment

This work has received the funding support from the department of science and technology (DST), Government of India under the INSPIRE Faculty Fellowship. The authors would like to thank Dr. Emilie Kaufmann, Junior Researcher, CRIStAL, Lille France and Dr. Aditya Gopalan, Assistant Professor,

IISc India, for their help on the analysis and implementation of the