Senior Member, IEEE Marios Kountouris

The Role of Fidelity in Goal-Oriented Semantic Communication: A Rate Distortion Approach

We study a variant of a robust description source coding framework via its corresponding characterization, which is a relevant model for goal-oriented semantic information transmission. Considering two individual single-letter separable distortion constraints and input and output data acting as the intrinsic and extrinsic message, respectively, we first derive bounds on the optimal rates of the problem, as well as necessary and sufficient conditions for these bounds to be tight. Subsequently, we prove a general result that provides in parametric form the optimal solution of the characterization of this problem. Capitalizing on these results, we examine the structure of the solution for one case study of general binary alphabets under Hamming distortions and solve in closed form a special case. We also solve another general binary alphabet case where a Hamming and an erasure distortion coexist, as a means to highlight the importance of selecting the type of the distortion constraint in goal-oriented semantic communication. We also develop a semantic-aware Blahut-Arimoto (BA) algorithm, which can be used for the computation of any finite alphabet intrinsic or extrinsic message under individual distortion criteria. Finally, we revisit the problem for multidimensional independent and identically distributed (IID) jointly Gaussian processes with individual mean-square error (MSE) distortion constraints, providing new insights that have previously been overlooked. This work reveals the cardinal role of context-dependent fidelity criteria in goal-oriented semantic communication.

I. INTRODUCTION

Shannon, in his seminal work [START_REF] Shannon | A mathematical theory of communication[END_REF], has deliberately considered the semantic aspects and the effectiveness of transmitted messages as irrelevant to the communication problem. Setting aside Part of the material of this paper was accepted for publication in IEEE International Symposium on Information Theory (ISIT), Espoo, Finland, 2022 [START_REF] Stavrou | A rate distortion approach to goal-oriented communication[END_REF].

The authors are with the Communication Systems Department at EURECOM, Sophia-Antipolis, France, email: {fotios.stavrou, marios.kountouris}@eurecom.fr an issue which is otherwise confusing, this dichotomy between information content and its significance has been instrumental in achieving reliability and efficiency in information transmission over noisy channels. Nevertheless, in [START_REF] Shannon | Coding theorems for a discrete source with a fidelity criterion[END_REF], Shannon has indirectly provided a means to study semantic information sources because the coding aspect determined by the probabilistic model of the source is dictated by a distortion constraint imposed in the system. Various endeavors have been made to incorporate semantics into Shannon's communication theory. Letting aside epistemic and doxastic logic theories, the most important efforts include probabilistic logic approaches [START_REF] Bar-Hillel | Semantic information[END_REF]- [START_REF] Bao | Towards a theory of semantic communication[END_REF], complexity theory approaches [START_REF] Goldreich | A theory of goal-oriented communication[END_REF], [START_REF] Okamoto | A unified paradigm of organized complexity and semantic information theory[END_REF], and semantic coding and communication games [START_REF] Willems | Semantic coding: Partial transmission[END_REF], [START_REF] Güler | The semantic communication game[END_REF]. The effectiveness problem has been considered using the concepts of pragmatic information [START_REF] Sharma | On measures of useful information[END_REF] and value of information [START_REF] Stratonovich | On the value of information[END_REF], [START_REF] Howard | Information value theory[END_REF]. Nevertheless, Shannon's communication model has remained virtually unchallenged. None of the proposed extensions has ever been recognized as a general theory of semantic or pragmatic information. The aforementioned theories have remained at a conceptual level, failing to have any tangible practical applications to or impact on communication networks. The quest for a goal-oriented semantic communication theory has recently gained new impetus [START_REF] Popovski | Semantic-effectiveness filtering and control for post-5G wireless connectivity[END_REF]- [START_REF] Uysal | Semantic communications in networked systems[END_REF], fueled by the emergence of networks of autonomous agents with advanced sensing, learning, and decision-making capabilities.

In this work, we consider the problem of communicating a memoryless source, whose semantic, remote or intrinsic information is not directly observable, based on noisy observations. Our objective is to investigate the impact of context-dependent fidelity criteria and distortion measures on goal-oriented information transmission and semantic source reconstruction. For that, we revisit a lossy compression framework, recently introduced in [START_REF] Liu | A rate-distortion framework for characterizing semantic information[END_REF], [START_REF] Liu | An indirect rate-distortion characterization for semantic sources: General model and the case of Gaussian observation[END_REF], considering both finite alphabets and Gaussian i.i.d random variables (RVs), and we study the effect of multiple individual distortion criteria in goal-oriented semantic information transmission. The objective of this work is twofold. First, we aim at complementing and extending the work in [START_REF] Liu | An indirect rate-distortion characterization for semantic sources: General model and the case of Gaussian observation[END_REF], which only considers continuous alphabet sources (i.e., i.i.d Gaussian sources) and mean-square error (MSE) distortion criteria, providing results and new insights that have been overlooked in prior work.

Second, we aim at further emphasizing on the role of the context-dependent fidelity criteria in goal-oriented semantic communication by showing cases with new outcomes that do not appear through the analysis of [START_REF] Liu | A rate-distortion framework for characterizing semantic information[END_REF], [START_REF] Liu | An indirect rate-distortion characterization for semantic sources: General model and the case of Gaussian observation[END_REF].

A. Related work

The problem considered here falls within the realm of remote source coding problem [START_REF] Dobrushin | Information transmission with additional noise[END_REF]- [START_REF] Witsenhausen | Indirect rate distortion problems[END_REF] and rate distortion with multiple distortion constraints [START_REF] Gamal | Achievable rates for multiple descriptions[END_REF], [START_REF] Berger | Rate Distortion Theory: A Mathematical Basis for Data Compression[END_REF], [START_REF] Cover | Elements of Information Theory[END_REF]Problem 10.19]. The most closely related setup to our work is the rate distortion framework introduced in [START_REF] Liu | A rate-distortion framework for characterizing semantic information[END_REF], [START_REF] Liu | An indirect rate-distortion characterization for semantic sources: General model and the case of Gaussian observation[END_REF], which provides characterizations and optimal closed form expressions for i.i.d scalar-valued Gaussian processes and sub-optimal characterizations assuming linear based state-observation models with Gaussian observations with ways to numerically compute the solution beyond scalar RVs. Some additional results in [START_REF] Liu | A rate-distortion framework for characterizing semantic information[END_REF], include a case study for the classification of a binary uniform semantic source with an extrinsic observable scalar-valued Gaussian mixture model. It should be noted that the rate distortion framework considered here and in [START_REF] Liu | A rate-distortion framework for characterizing semantic information[END_REF], [START_REF] Liu | An indirect rate-distortion characterization for semantic sources: General model and the case of Gaussian observation[END_REF] can be seen as a generalization of the robust description problem for two individual distortion criteria, which in turn is a special case of the two description coding problem [START_REF] Gamal | Achievable rates for multiple descriptions[END_REF]. Rate distortion with two individual distortion criteria has been studied in many papers under various contexts, see, e.g., [START_REF] Gray | A new class of lower bounds to information rates of stationary sources via conditional rate-distortion functions[END_REF]- [START_REF] Stylianou | Joint rate distortion function of a tuple of correlated multivariate Gaussian sources with individual fidelity criteria[END_REF]. Another relevant yet different setup is the recently introduced rate-distortion-perception representations, see, e.g., [START_REF] Blau | Rethinking lossy compression: The rate-distortion-perception tradeoff[END_REF], [START_REF] Zhang | Universal rate-distortion-perception representations for lossy compression[END_REF] (and the references therein), in which perception quality, measured by some divergence between distributions, is included in addition to the classical distortion criterion.

One major difference between rate-distortion-perception problems and the setup here is that in the former the characterizations are solved for various examples using separately each distortion constraint, whereas in the latter, one can study from an optimization standpoint the joint behavior of the two distortion penalties.

B. Contributions

In this paper, we consider a variation of the robust lossy source coding model, similarly to [START_REF] Liu | A rate-distortion framework for characterizing semantic information[END_REF], [START_REF] Liu | An indirect rate-distortion characterization for semantic sources: General model and the case of Gaussian observation[END_REF], which captures goal-oriented semantic attributes and intrinsic representation of information (e.g., features, structural/qualitative properties, embedding). For this setup and its corresponding characterization (see Lemma 1, eq. ( 6)), we derive the following new results.

• We obtain bounds on the semantic rate distortion function (cf. ( 6)) and identify necessary and sufficient conditions based on which these bounds can be tight (see Lemma 2).

• We prove a general theorem, which gives parametrically the implicit solution of (6) for arbitrary finite alphabet sets with individual semantic and observable distortion criteria (see Theorem 1).

• We develop a semantic-aware Blahut-Arimoto algorithm (see Algorithm 1) that allows the computation of (6) for any finite alphabet set of intrinsic or extrinsic messages with arbitrary individual single-letter distortion criteria.

• We revisit the problem for multidimensional jointly Gaussian RVs with individual MSE distortion constraints, initially studied in [20, Section IV], providing a different angle that is also aligned to the proposed semantic information transmission setup and our findings.

For this class of input data we also derive an example that demonstrates the non-triviality of our lower bound in Lemma 2 (see Example 2).

The aforementioned results are not the sole contributions of this paper. Lemma 2 and Theorem 1 are applied into two examples (see Problems 1, 2) using specific setups with general binary alphabets and two types of distortion measures, namely Hamming and erasure distortions. For Problem 1, we derive structural properties on the optimal minimizer (test channel) consistent with Lemma 2 and characterize its solution (see Theorem 2). We enhance this result by solving in closed form a special case to illustrate the rate distortion surface of the problem (see Example 1). For Problem 2, we characterize and solve in closed form the solution (see Theorem 3). An interesting observation that stems from Theorem 3 is that depending on the distortion constraint, we can make the system choose which source (i.e, semantic or observation) to transmit. Simply put, in goal-oriented semantic communication, selecting the type of individual distortion measures or context-dependent fidelity criteria according to the application/task requirements can significantly affect the remote reconstruction of the semantic source.

II. PROBLEM STATEMENT AND NEW BOUNDS

We consider a memoryless source described by the tuple (x, z) with probability distribution p(x, z) in the product alphabet space X × Z. The semantic or intrinsic information of the source is in x, which is not directly observable, whereas z is the noisy observation of the source at the encoder side. The goal is to study how the distortion penalties can affect goal-oriented information transmission and source reconstruction using lossy compression. Formally, the system model (without the distortion penalties) is illustrated in Fig. 1 and can be interpreted as follows. An information source is a sequence of n-length i.i.d RVs (x n , z n ).

In this setup, we assume we know p(x) and the transition probability distribution p(z|x). The encoder (E) and the decoder (D), are modeled by the mappings

f E : Z n → W g D o : W → Z n , g D s : W → X n , (1)
where the index set W ∈ {1, 2, . . . , M } and (g D o , g D s ) denote the observations and semantic information decoder, respectively.

Encoder Decoder

p z x n n ( | ) n z n x ( ) E f n z W ) n n (x ,z
Fig. 1: System model.

We consider two per-letter distortion measures responsible to penalize the semantic and observations information source in Fig. 1, given by d s : X × X → [0, ∞) and

d o : Z × Z → [0, ∞),
respectively, and their corresponding average per-symbol distortions by

d n s (x n , x n ) = 1 n n t=1 d s (x i , x i ) (2)
d n o (z n , z n ) = 1 n n t=1 d o (z i , z i ). (3)
The encoding and decoding is done in blocks of length n and the fidelity criterion for the semantic and observable information is the pair of average distortions defined as

∆ s ≜ E {d n s (x n , x n )} , ∆ o ≜ E {d n o (z n , z n )} . (4)
Next, we give the definitions of the achievable rates and the infimum of all achievable rates.

Definition 1. For two distortion levels

D o ≥ 0, D s ≥ 0, a number R is said to be (D o , D s )-achievable
if for an arbitrary ϵ > 0, there exists, for n large enough, a semantic-aware lossy source code

(n, M, ∆ o , ∆ s ) with M ≤ 2 n(R+ϵ) such that ∆ o ≤ D o + ϵ and ∆ s ≤ D s + ϵ. Moreover, suppose that sequences of distortion functions {(d n o , d n s ) : n = 1, 2, . . .} are given. Then, R(D o , D s ) = inf {R : (R, D o , D s ) is achievable} . ( 5
)
The goal of the setup in Fig. 1 and of our results in the following sections is to further demonstrate the impact of the fidelity criterion in a remote source coding problem with individual distortion measures.

A. Characterization of the operational rates

The information theoretic characterization of ( 5) is given by the following lemma.

Lemma 1. For a given p(x) and p(z|x), the semantic rate distortion function (SRDF) of the setup in Fig. 1 is characterized as follows

R(D s , D o ) = inf q( z, x|z) E[ ds(z, x)]≤Ds E[do(z, z)]≤Do I(z; z, x), (6)
where

d s (z, x) = x∈X p(x|z)d s (x, x), D s ∈ [0, ∞], D o ∈ [0, ∞], I(z; z, x) ≜ E log q( z, x|z) ν( z, x) ≡ I(p, q), (7)
with I(p, q) demonstrating the functional dependence of the mutual information on {p(z), q( z, x|z)}.

A detailed proof of Lemma 1 is omitted because the achievability part follows from a special case of the achievability proof of the multiple description source coding problem called robust description [START_REF] Gamal | Achievable rates for multiple descriptions[END_REF]Theorem 2] in view of the fact that one can modify the indirect to a direct rate distortion function (RDF) formulation using an amended version for the semantic distortion constraint, i.e., E [d s (x n , x n )] = E d s (z n , x n ) (see e.g., [19, Theorem 1] and references therein).

Although the converse part is not provided due to space limitations, it can be easily obtained following standard arguments, see, e.g., [26, p. 316].

We conclude this subsection with certain functional and topological properties of (6).

Remark 1. The following functional properties of SRDF can be obtained using standard arguments that stem from classical rate distortion theory, see, e.g., [START_REF] Berger | Rate Distortion Theory: A Mathematical Basis for Data Compression[END_REF].

(i) R(D o , D s ) is a non-increasing function of D s ∈ [0, ∞) and D o ∈ [0, ∞) and (jointly)
convex with respect to (D s , D o ).

(ii) In (6), I(p, q) is a convex functional of q( z, x|z) for a fixed p(z).

(iii) If R(D o , D s ) < ∞, then R(•) is continuous for D o ∈ [0, ∞) and D s ∈ [0, ∞).
We conclude this remark by pointing out that the constrained set in (6) is compact (for both finite or abstract alphabets) and the objective function in (6) is lower semi-continuous with respect to q( z, x|z). As a result, from Weierstrass extreme value theorem, we know that the infimum is attained by a q * ( z, x|z) and we can formally replace it with minimum in the sequel.

B. Bounds and conditions for the tightness of these bounds

In what follows, we derive new bounds on (6), as well as information structures (i.e., conditional independence constraints) that allow for these bounds to be tight.

Lemma 2. (1)

The optimization problem in (6) admits the following bounds:

max {R(D s ), R(D o )} = R L (D o , D s ) ≤ (6) ≤ R U (D o , D s ) = R(D o ) + R(D s ), (8)
where

I(z; x). ( 9
) (2) R L (D o , D s ) is tight if and only if z -z -x and z -x -z, (10)
are concurrently satisfied. On the other hand,

R U (D o , D s ) is tight if and only if z -z -x, (11) is satisfied.
Proof: (1) Clearly, R L (D o , D s ) in (8) corresponds to the best possibly achievable rates because (6) cannot be lower than the best rate achieved in either less constrained problem (individual rate distortion problems in [START_REF] Okamoto | A unified paradigm of organized complexity and semantic information theory[END_REF]). On the other hand, R U (D o , D s ) in ( 8) is always allowed because we can always minimize assuming z independent of x (denoted herein after by z ⊥ x), i.e., ν( z, x) = ν( z)ν( x).

(2) Next, we derive conditional independent constraints (i.e., information structures) which allow the bounds on (6) to be tight. Recall that by the chain rule of mutual information [START_REF] Bao | Towards a theory of semantic communication[END_REF] (see, e.g., [START_REF] Cover | Elements of Information Theory[END_REF]), we have

I(z; z, x) = I(z; x) + I(z; z| x) = I(z; z) + I(z; x| z). (12)
From [START_REF] Sharma | On measures of useful information[END_REF], we obtain

I(z; z, x) (a)
≥ I(z; x), I(z; z, x)

(b) ≥ I(z; z), (13)
where (a) follows from the fact that I(z; z| x) ≥ 0 and (b) follows from the fact that I(z; x| z) ≥ 0.

Clearly the bounds in [START_REF] Stratonovich | On the value of information[END_REF] are tight if and only if for inequality (a) I * (z; z| x) = 0, i.e., the Markov chain zxz holds and for inequality (b) I * (z; x| z) = 0, i.e., the Markov chain zzx holds. In view of the previous simple observation we arrive to the following bounds on [START_REF] Floridi | Semantic Conceptions of Information[END_REF]. Now for R U (D s , D o ) to be tight, we need to make sure that provided that if x ⊥ z, the set of minimizers in ( 6) coincides with the smaller set of the individual minimizers in both problems in [START_REF] Okamoto | A unified paradigm of organized complexity and semantic information theory[END_REF], i.e., q( x, z|z) = q( x|z)q( z|z), which is guaranteed if and only if [START_REF] Güler | The semantic communication game[END_REF] is satisfied. In other words, from [START_REF] Bao | Towards a theory of semantic communication[END_REF] we obtain ( 7)

(e) = E log q( z|z)q( x|z) ν( z)ν( x) = I(z; z) + I(z; x) (14)
where (e) follows if and only if [START_REF] Güler | The semantic communication game[END_REF] is true. The equality in [START_REF] Howard | Information value theory[END_REF] implies that the upper bound in [START_REF] Goldreich | A theory of goal-oriented communication[END_REF] is tight. This completes the proof.

III. MAIN RESULTS

In this section, we provide the majority of our new results. Before giving our first result, we note that the constrained problem in Lemma 1 can be written as an unconstrained problem via the Lagrange duality theorem [START_REF] Luenberger | Optimization by Vector Space Methods[END_REF] as follows

R(D o , D s ) = max s 1 ≤0 s 2 ≤0 min q( z, x|z)≥0 z, z q( z, x|z)=1 I(z; z, x) -s 1 E d s (z, x) -D s -s 2 (E [d o (z, z)] -D o ) (15)
where s 1 ≤ 0 and s 2 ≤ 0 are the Lagrange multipliers.

In view of [START_REF] Popovski | Semantic-effectiveness filtering and control for post-5G wireless connectivity[END_REF] we can prove the following general result.

Theorem 1. Suppose that p(x) and p(z|x) are given. Then, the following parametric solutions for (6) may appear.

(i) If s 1 < 0 and s 2 < 0, the implicit optimal form of the minimizer that achieves the minimum in (6) is

q * ( z, x|z) = e s 1 ds(z, x)+s 2 do(z, z) ν * ( z, x) z, x e s 1 ds(z, x)+s 2 do(z, z) ν * ( z, x) (16)
where (s 1 , s 2 ) are the Lagrange multipliers associated with the individual distortion penalties and ν * ( z, x) = z q * ( z, x|z)p(z) is the Z × X -marginal of the output process ( z n , x n ).

Moreover, the optimal parametric solution of (6

) when R(D * s , D * 0 ) > 0 is given by R(D * o , D * s ) = s 1 D * s + s 2 D * o - z p(z) log   z, x e s 1 ds(z, x)+s 2 do(z, z) ν * ( z, x)   , (17)
where

D * s = z, x d s (z, x)q * ( x|z)p(z), (18)
D * o = z, z d o (z, z)q * ( z|z)p(z), (19)
and q * ( x|z) = z q * ( z, x|z), q * ( z|z) = x q * ( z, x|z). (ii) If s 1 < 0, s 2 = 0, and R(D * s , D * 0 ) > 0, we obtain R(D * o , D * s ) ≡ R(D * s ) = s 1 D * s - z p(z) log x e s 1 ds(z, x) ν * ( x) , (20)
where D * s is given by (18) and q * ( x|z) = e s 1 ds(z, x) ν * ( x)

x e s 1 ds(z, x) ν * ( x)

.

(iii) If s 1 = 0, s 2 < 0, and R(D * s , D * 0 ) > 0, we obtain R(D * o , D * s ) ≡ R(D * o ) = s 2 D * o - z p(z) log z e s 2 do(z, z) ν * ( z) , (21)
where D * s is given by ( 19) and q * ( z|z)

= e s 1 do(z, z) ν * ( z) z e s 1 do(z, z) ν * ( z) . (iv) If s 1 = 0 and s 2 = 0, then, R(D * o , D * s ) = 0. Proof: See Appendix A.
Theorem 1 is pivotal as it can be used in various ways including the derivation of analytical expressions of (6) or for the construction of generalizations of the BA algorithm [START_REF] Blahut | Computation of channel capacity and rate-distortion functions[END_REF], which can optimally find parametrically the solution of (6) for arbitrary finite alphabet sets and general bounded distortion functions. In the sequel we also study these directions.

A. Binary alphabets with individual Hamming distortions

In what follows, we utilize both Theorem 1 and Lemma 2 to study the case of binary alphabets, i.e., X = Z = X = Z = {0, 1} with individual probability of error distortion penalties.

Problem 1. Suppose in the setup of Fig. 1, the remote source x and the noisy channel of z given

x are modeled as follows

p(x) =   p(x = 0) p(x = 1)   =   α 1 -α   , p(z|x) =   p(z = 0|x = 0) p(z = 0|x = 1) p(z = 1|x = 0) p(z = 1|x = 1)   =   β γ 1 -β 1 -γ   (22)
where We provide now a major result of this paper.

(α, β, γ) ∈ [0, 1] × [0, 1] × [0, 1], β ̸ = γ 1 and d s (x, x) =      0 if x = x 1 if x ̸ = x , d o (z, z) =      0 if z = z 1 if z ̸ = z .
Theorem 2. Consider the setup in Fig. 1 restricted to the given data of Problem 1. Then, the following hold:

(i) the necessary and sufficient conditions in (10) hold;

(ii) R(D * o , D * s ) = R L (D * o , D * s ). Proof: See Appendix B.
The general result of Theorem 2 shows that the rate-splitting bound in ( 8) is achievable for the specific class of input data under the probability of error distortions assumed in Problem 1.

In what follows, we study the structural solution derived in Theorem 2. Recall that since

R(D * o , D * s ) = R L (D * o , D * s )
holds, then, we need to compute the rate distortion functions in [START_REF] Okamoto | A unified paradigm of organized complexity and semantic information theory[END_REF].

For the direct rate distortion problem with binary source, it is relatively easy to see that the closed form solution will be a straightforward generalization of the analytical solution derived for instance in [

R(D * o ) =      H b (p) -H b (D o ), if 0 ≤ D o ≤ min{p, 1 -p} 0, if D o > min{p, 1 -p} (23)
where p = p(z = 0) is computed in (52) and H b (•) denotes the binary entropy function. We stress that an optimal closed form solution of the binary indirect RDF ( 9) is not known, in general, and only bounds exist in the literature, see, e.g., [START_REF] Kipnis | The indirect rate-distortion function of a binary i.i.d source[END_REF]. Nevertheless, one can always use straightforward generalizations of the classical BA iterative schemes to numerically compute the optimal solution. The semantic-aware BA algorithm that we develop in the sequel, includes as a special case this particular case.

Example 1. In the particular case where the semantic remote source is i.i.d Bernoulli( 12 ), i.e., p(x = 0) = 1 2 , and the binary channel in

(22) is symmetric with p(z = 1|x = 1) = β = 1 -c and crossover probability p(z = 0|x = 1) = γ = c, c ∈ [0, 1 2 ) 2 , one can easily infer via (23) that H b (p) = 1 bit source/sample and R(D * o ) =      1 -H b (D o ), if 0 ≤ D o ≤ 1 2 0, if D o > 1 2 . ( 24
)
Moreover, for the same input data, it can be shown, see e.g., [START_REF] Berger | Rate Distortion Theory: A Mathematical Basis for Data Compression[END_REF]Exercise 3.8], that

R(D * s ) =      1 -H b Ds-c 1-2c , if β < D s ≤ 1 2 0, if D s > 1 2 . ( 25
)
Substituting ( 24), ( 25) in Theorem 2 we obtain

R(D * o , D * s ) = max [1 -H b (D o )] + , 1 -H b D s -c 1 -2c + ( 26
)
where

[•] + = max{0, •}.
The rate distortion surface for c = 0.15 is displayed in Fig. 2.

Based on (26), we observe an interesting interplay between (c, D s , D o ) regarding the choice of the maximum achievable rates. In particular, it appears that if D o > Ds-c 1-2c , then the system benefits more by encoding subject to a Hamming distortion only the semantic information,

d s (x, x) =            0 if x = x 1 if x = e ∞, x ̸ = x , ( 27
)
where X = {0, e, 1}.

Based on the given data of Problem 2, we derive the following solution for the SRDF characterization. First we re-state an equivalent way to arrive to the parametric solution of Theorem 1 using instead the alternating minimization approach [START_REF] Blahut | Computation of channel capacity and rate-distortion functions[END_REF] (see also [START_REF] Yeung | Information theory and network coding[END_REF]). This result and the subsequent corollaries form the basis of the semantic-aware BA algorithm that we aim to develop.

Lemma 3. Let s 1 ≤ 0, s 2 ≤ 0 be given. Then the following statements hold.

(1) The optimal parametric solution of (6) can be expressed as follows:

R(D * o , D * s ) = s 1 D * s + s 2 D * o + min ν( z, x) min q( z, x|z) z z, x log q( z, x|z) ν( z, x) q( z, x|z)p(z) -s 1 z z, x d s (z, x)q( z, x|z)p(z) -s 2 z z, x d o (z, z)q( z, x|z)p(z) (28)
where D * s and D * o are given by ( 18) and (19), respectively. (2) For fixed q( z, x|z), the right hand side (RHS) in (28) is minimized by

ν * ( z, x) = z q( z, x|z)p(z).
(3) For fixed ν( z, x) the RHS in (28) is minimized by q * ( z, x|z) given by (16), i.e., q * ( z, x|z) = e s 1 ds(z, x)+s 2 do(z, z) ν( z, x)

z, x e s 1 ds(z, x)+s 2 do(z, z) ν( z, x)

.

Proof: (1) Due to the convexity and monotonicity of the optimization problem in (6) (see Remark 1), we can reformulate it as an unconstrained problem as follows

R(D s , D o ) = min q( z, x|z) I(z; z, x) -s 1 E d s (z, x) -D s -s 2 (E [d o (z, z)] -D o ) (29)
where the expectation operator E[•] is taken with respect to the joint distribution p(z, z, x). Using [START_REF] Xiao | Compression of correlated Gaussian sources under individual distortion criteria[END_REF], the double minimization in [START_REF] Stjernvall | Dominance-a relation between distortion measures[END_REF] follows by applying [37, Theorem 5.2.6], namely, we can reformulate mutual information I(z; z, x) as follows

I(z; z, x) = min ν( z, x) z z, x log q( z, x|z) p( z, x) , ( 30
)
where the minimization is over an arbitrary chosen output marginal distribution ν( z, x) defined on Z × X and the minimization over ν( z, x) follows from the condition for equality in that theorem.

(2), (3) To optimize, we use Karush-Kuhn-Tucker (KKT) conditions similar to the ones utilized for the derivation of Theorem 1 hence we omit it.

In Lemma 3 we consider the Lagrange multipliers (s 1 , s 2 ) to be non-positive. One can easily obtain all the special cases discussed in Theorem 1 by choosing to have only one or none of the distortion constraints active. Clearly, if we consider optimizing jointly with respect to {q( z, x|z), ν( z, x)} in [START_REF] Stjernvall | Dominance-a relation between distortion measures[END_REF], then, the result of Lemma 3, will coincide to the general result of Theorem 1.

Next, we give two corollaries that are instrumental in the development of our algorithm because they give the implicit solution of {q( z, x|z), ν( z, x)} parametrized by (s 1 , s 2 ).

Corollary 1. If q( z, x|z) achieves a point on R(D o , D s ) parametrized by (s 1 , s 2 ), then q( z, x|z) is given by [START_REF] Kountouris | Semantics-empowered communication for networked intelligent systems[END_REF] and

ν( z, x) = ν( z, x) z p(z) e s 1 ds(z, x)+s 2 do(z, z) z, x e s 1 ds(z, x)+s 2 do(z, z) ν( z, x) . ( 31
)
Proof: The proof follows using the simultaneous satisfaction of Lemma 3, (2), (3).

Corollary 2. In terms of (s 1 , s 2 ), R(D o , D s ) can be written as follows:

R(D s 1 o , D s 2 s ) = s 1 D s 1 s + s 2 D s 2 o + min ν( z, x)   - z p(z) log   z, x
e s 1 ds(z, x)+s 2 do(z, z) ν( z, x)

    , (32)
with

D s 1 s = z, z, x d s (z, x)
e s 1 ds(z, x)+s 2 do(z, z) ν * ( z, x)

z, x e s 1 ds(z, x)+s 2 do(z, z) ν * ( z, x) p(z),

D s 2 o = z, z, x d o (z, z)
e s 1 ds(z, x)+s 2 do(z, z) ν * ( z, x)

z, x e s 1 ds(z, x)+s 2 do(z, z) ν * ( z, x)

p(z)
where ν * ( z, x) achieves R(D s 1 o , D s 2 s ).

Proof: The proof follows by substituting q * ( z, x|z) of Lemma 3, (3) in Lemma 3, (1).

We continue our analysis with a general theorem reminiscent of the one derived for the classical BA algorithm. This theorem demonstrated the convergence of the algorithm to the optimal limit point. In the theorem we denote p = p(z), ν ≜ ν( z, x), and q = q( z, x|z) the probability vector of those distributions, e.g., for Z = {0, 1, . . . , M 1 }, M 1 ∈ Z + , p = (p(z = 0), . . . , p(z = M 1 )).

Theorem 4. Let the parameters s 1 ≤ 0 and s 2 ≤ 0 be given and denote A(z, z, x) = e s 1 ds(z, x)+s 2 do(z, z)

and k ≥ 1 the number of iterations. Let any ν (0) be any initial marginal probability distribution on X × Z (given as a probability vector) with all the components to be nonzero. Let ν (k+1) be given in terms of ν (k) as follows

ν (k+1) ( z, x) = ν (k) ( z, x) z p(z)A(z, z, x) z, x A(z, z, x)ν (k) ( z, x) (33)
Then,

D s (q(ν (k) )) -→ D s 1 s , D o (q(ν (k) )) -→ D s 2 o , I(p; q(ν (k) )) -→ R(D s 1 s , D s 2 o ) as k → ∞, (34)
where

q(ν (k) ) = A(z, z, x)ν (k) ( z, x) z, x A(z, z, x)ν (k) ( z, x) and (D s 1 s , D s 2 o , R(D s 1 s , D s 2 o )) is a point on R(D o , D s ) parametrized by (s 1 , s 2 ).
Proof: See Appendix D.

In order to develop an algorithm to compute (6) for arbitrary finite alphabets sets, we also need a termination criterion, for which we need a generalization of [37, Theorem 6.3.9] to our setup. This is stated next as a lemma.

Lemma 4. An alternative definition of the RDF in (6) is the following

R(D o , D s ) = max s 1 ≤0,s 2 ≤0,λ∈Λs 1 ,s 2   s 1 D s + s 2 D o + z∈Z{0,1,...,M 1 } p(z) log λ(z)   , (35)
where Λ s 1 ,s 2 is the set of all vectors λ = (λ(z = 0), . . . , λ(z = M 1 )) with non-negative elements that satisfy inequality constraints z∈Z p(z)λ(z)e s 1 ds(z, x)+s 2 do(z, z) ≤ 1.

Proof: We sketch the proof because it can be easily understood. Let

s 1 ≤ 0, s 2 ≤ 0, λ ∈ Λ s 1 ,s 2 and q ∈ {q( z, x|z) : E[ d s (z, x)] ≤ D s , E[d o (z, z)] ≤ D s }.
Then, from these conditions, it can be proved that

I(p, q) -s 1 D s -s 2 D o -z∈Z p(z) log λ(z) ≥ 0, which implies that R(D o , D s ) ≥ s 1 D s + s 2 D o + z∈Z p(z) log λ(z)
. However, we know that for some (s 1 , s 2 , λ)

this inequality holds with equality, i.e., λ(z) = z, x e s 1 ds(z, x)+s 2 do(z, z) ν * ( z, x)

-1
with (s 1 , s 2 )

and ν * (•) achieving R(D o , D s ).
The next theorem gives bounds on the RDF, which allow to estimate in the algorithm the residual error at each iteration. This theorem serves as a stopping criterion for our algorithm.

Theorem 5. Let the parameters s 1 ≤ 0, s 2 ≤ 0 be given and let A(z, z, x) = e s 1 ds(z, x)+s 2 do(z, z) .

Suppose that ν is any output probability vector and let

c( z, x) = z p(z) A(z, z, x) z, x A(z, z, x)ν( z, x) . ( 36
)
Then at the points

D s = z, z, x d s (z, x) A(z, z, x)ν( z, x) z, x A(z, z, x)ν( z, x) p(z), D o = z, z, x d o (z, z) A(z, z, x)ν( z, x) z, x A(z, z, x), ν( z, x) p(z), (37)
and by setting Γ ≜ s 1 D s + s 2 D oz p(z) log z, x A(z, z, x)ν( z, x) , we have the following bounds

R(D o , D s ) ≤ Γ - z, x ν( z, x)c( z, x) log c( z, x), (38)
R(D o , D s ) ≥ Γ -max z, x log c( z, x). (39)
Proof: See Appendix E.

We are now ready to give the semantic-aware BA algorithm for the setup in Fig. 1. This is illustrated in Algorithm 1.

Algorithm 1 Semantic-aware Blahut-Arimoto algorithm Initialize: Choose p = (p(z = j), j = 0, 1, . . . , N 1 ), and p ′ ≜ (p(z = j|x = i), i = 0, 1, . . . , M 1 , j = 0, 1, . . . , N 1 ), M 1 ∈ Z + , N 1 ∈ Z + ; s 1 ≤ 0; s 2 ≤ 0; error tolerance ϵ;

choose the alphabet of Z, X ; choose the distortion functions d s (x, x) and d o (z, z); initial output probability vector ν (0) (all the elements to be non-zero).

Compute

d s (z, x) = x∈X p(x|z)d s (x, x), X = {0, 1, . . . , R 1 }, R 1 ∈ Z + .
Compute A(z, z, x) = e s 1 ds(z, x)+s 2 do(z, z) , for all z ∈ Z, x ∈ X .

while flag = 0 do

Step 1: Set k = 1.

Step 2: For all z ∈ Z, x ∈ X , Compute c (k) ( z, x) = z∈Z p(z)

A(z, z, x) z, x A(z, z, x)ν (k-1) ( z, x) . Compute ν (k) ( z, x) = ν (k-1) ( z, x)c (k) ( z, x). Compute diff=max z, x log c (k) ( z, x) -z, x ν (k) ( z, x) log c (k) ( z, x). if diff> ϵ then Set k = k + 1 and return to Step 2. else flag ← 1 end if end while Output: q( z, x|z) = A(z, z, x)ν (k) ( z, x) z, x A(z, z, x)ν (k) ( z, x) ; D s = z, z, x d s (z, x)q( z, x|z)p(z); D o = z, z, x d o (z, z)q( z, x|z)p(z); R(D s , D o ) = s 1 D s + s 2 D o - z p(z) log z, x A(z, z, x)ν (k) ( z, x) -z, x ν (k) ( z, x) log c (k) ( z, x).
Clearly Algorithm 1 is a generalization of the well-known BA algorithm [START_REF] Blahut | Computation of channel capacity and rate-distortion functions[END_REF]. By choosing appropriately the value of the Lagrangians (s 1 , s 2 ) one can recover the classical BA algorithm or its extension to the indirect rate distortion problem.

IV. REVISITING THE PROBLEM FOR JOINTLY GAUSSIAN i.i.d PROCESSES

In this section, we revisit the problem for jointly Gaussian i.i.d processes, which have recently been studied in [START_REF] Liu | An indirect rate-distortion characterization for semantic sources: General model and the case of Gaussian observation[END_REF]. Therein, the authors consider (x, z) to be zero mean jointly Gaussian random vectors such that x ∈ R l , z ∈ R m , m ̸ = l, with covariance matrix

Σ (x,z) ≜   Σ x Σ xz Σ T xz Σ z   , (40)
with given p(z) ∼ N (0; Σ z ), Σ z ⪰ 0 and given p(x|z) represented by a linear state-observations model, i.e., x = Hz + n, such that p(x|z) ∼ N (Hz; Σ n ), where

H = Σ xz Σ -1 z and Σ n = Σ x -Σ xz Σ -1 z Σ T xz ⪰ 0. Then, assuming in (2), (3), that d o (z i , z i ) = ||z i -z i || 2 2 and d s (x i , x i ) = ||x i -x i || 2
2 (or the quadratic Euclidean norm), respectively, SRDF ( 6) is as follows

R(D o , D s ) = min Σ z| x, z ⪰0 trace(HΣ z| x, z H T )≤Ds-trace(Σn) trace(Σ z| x, z )≤Do 1 2 log det(Σ z ) det(Σ z| x, z ) + , (41)
where det(•) is the determinant of a matrix, the conditional covariance defined by

Σ z| x, z ≜ E [(z -E [z| x, z])(z -E [z| x, z]) T
] is a design variable in the optimization in (41) and [•] + = max{0, •}. Log-determinant convex problems of the form (41), can be solved optimally using standard convex programming solvers that exist in the CVX platform [START_REF] Grant | CVX: Matlab software for disciplined convex programming[END_REF].

Next,we state some important technical remarks about the result of [START_REF] Liu | An indirect rate-distortion characterization for semantic sources: General model and the case of Gaussian observation[END_REF]. (3) The problem setup described in [20, Section IV] is not the same with the problem statement of Section II because according to the latter, we are given p(x) and p(z|x) and we want to design p(x|z) and p(z), respectively. In other words, following Section II, the linear state-observations model described in [20, eq. ( 19)] should be designed, i.e., matrix H ∈ R l×m and the covariance matrix Σ n need to be designed instead of been freely chosen.

Under the technical observation of Remark 3,[START_REF] Shannon | Coding theorems for a discrete source with a fidelity criterion[END_REF], in what follows, we re-formulate the problem to be consistent with Section II. Suppose that (x, z) are zero mean jointly Gaussian random vectors such that x ∈ R l , z ∈ R m , m ≤ l, with covariance matrix given by [START_REF] Blahut | Information bounds of the Fano-Kullback type[END_REF]. Moreover, assume that x ∼ N (0, Σ x ), Σ ≻ 0 and p(z|x) is modeled by the linear realization of the form

z = Ax + s, (42)
where A ∈ R m×l assumed to be full row rank matrix, and s ∼ N (0; Σ s ), Σ s ⪰ 0. Then, the conditional Gaussian distribution p(x|z) ∼ N (E[x|z]; Σ x|z ), where the conditional mean

E[x|z] = Σ xz Σ -1 z z and the conditional covariance Σ x|z = Σ x -Σ xz Σ -1 z Σ T xz ⪰ 0. However, since Σ xz = E[x(Ax + s) T ] = Σ x A T and Σ z = AΣ x A T + Σ s , we can further infer that E[x|z] = Σ x A T (AΣ x A T + Σ s ) -1
z and similarly, one can apply the same values in Σ x|z .

Remark 4. If one wants to draw connections with the problem studied in [20, Section IV], it is clear that in our approach we obtain, by design, the matrices (H, Σ n ), i.e.,

H = Σ x A T (AΣ x A T + Σ s ) -1 and Σ n = Σ x|z = Σ x A T (AΣ x A T + Σ s ) -1 AΣ x , (43)
instead of freely choosing them. Hence we are consistent with the problem setup introduced in Section II.

The rest of the analysis follows using similar arguments to [START_REF] Liu | An indirect rate-distortion characterization for semantic sources: General model and the case of Gaussian observation[END_REF] resulting into the optimization problem of ( 41) with (H, Σ n ) designed as in (43). We provide next an example where we first compute and illustrate the solution of ( 41) and then we compare with the solution of the bounds in ( 8). On the other hand, using the bounds of 8, we should first solve independently the direct and the indirect rate distortion problem as these are defined in (9) and, then, for the lower bound on SRDF to choose the maximum between the two and for the upper bound to simply add the two rates. Since these two problems independently are already well studied in the literature, see, e.g., [START_REF] Dobrushin | Information transmission with additional noise[END_REF], [START_REF] Wolf | Transmission of noisy information to a noisy receiver with minimum distortion[END_REF], [START_REF] Berger | Rate Distortion Theory: A Mathematical Basis for Data Compression[END_REF], [START_REF] Cover | Elements of Information Theory[END_REF], [START_REF] Ihara | Information theory -for Continuous Systems[END_REF], we do not get into the details of their derivations but instead we only state the corresponding optimization problems for jointly Gaussian RVs and individual MSE distortion constraints. In particular,

R L (D o , D s ) = max{R(D s ), R(D o )}, and R U (D o , D s ) = R(D s ) + R(D o ), (45)
with

R(D s ) = min Σ z| x ≻0 trace(HΣ z| x H T )≤Ds-trace(Σn) 1 2 log det(Σ z ) det(Σ z| x ) , (46)
R(D o ) = min Σ z| z ≻0 trace(Σ z| z )≤Do 1 2 log det(Σ z ) det(Σ z| z ) , (47)
whereas the design variables for (46) and (47) correspond to the conditional covariance matrices

Σ z| x ≜ E [(z -E [z| x])(z -E [z| x]) T ] and Σ z| z ≜ E [(z -E [z| z])(z -E [z| z]) T ]
, respectively. In Table I, we compare for random values of the pair (D o , D s ), the solution of (41) with the values of the bounds obtained in (45). From our numerical results we observe that for certain distortion

pairs (D o , D s ), sometimes R L (R * o , R * s ) < R(D * o , D s ) and sometimes R L (R * o , R * s ) = R(D * o , D * s ) whereas for this example R(D * o , D s ) < R U (D * o , D * s )
. These results verify numerically the bounds in Lemma 2.

Type of Solution Distortion Values

(Do, Ds) = (0.6, 0.9) (Do, Ds) = (1.1, 0.85) (Do, Ds) = (0.2, 1.9) (Do, Ds) = (1.5, 1.9)

R

V. CONCLUSIONS AND ONGOING RESEARCH

A variant of a robust description source coding problem with two individual criteria, which is a relevant model for goal-oriented semantic communication, was studied here. We derived bounds on the semantic rate distortion function, as well as necessary and sufficient conditions for their tightness. We then proved a general theorem that provides the optimal solution of the characterization for this problem. Capitalizing on these results, we examined the structure of the solution for the case of general binary alphabets under Hamming and erasure distortion criteria.

We also constructed a semantic-aware BA algorithm for the computation of any finite alphabet sets under individual distortion criteria. Finally, we provided new insights overlooked so far for multidimensional jointly Gaussian RVs with individual MSE distortion constraints. A key takeaway from our results is that the class of the fidelity criteria may significantly affect the system behavior irrespective of its task, hence it should be chosen appropriately.

Our ongoing research activity builds on two directions. First, constructing analytical examples where the presence of the additional distortion fidelity affects semantic communication. Second, we seek to generalize our goal-oriented communication setup beyond separable distortion criteria, since in real world applications one is more interested in relations between the input and output symbols that are highly non-linear.

APPENDIX A PROOF OF THEOREM 1

The fully unconstrained problem of ( 6) using ( 15) is as follows

L({s i } 2 i=1 , λ(z), µ(z, z, x)) = x, z, z log q( z, x|z) ν( z, x) q( z, x|z)p(z) -s 1 E d s (z, x) -D s -s 2 (E [d o (z, z)] -D o ) + z λ(z)   z, x q( z, x|z) -1   - z, z, x µ(z, z, x)q( z, x|z), (48)
where s 1 ≤ 0, s 2 ≤ 0 are the Lagrangian multipliers associated with the individual distortion

constraints E d s (z, x) ≤ D s and E [d o (z, z)] ≤ D o
, respectively, whereas λ(z) ≥ 0 is associated with the equality constraint z, x q( z, x|z) = 1, and µ(z, z, x) ≥ 0 is responsible for the inequality constraint q( z, x|z) ≥ 0.

Due to the convexity of L(•) with respect to q(•, •|x), a necessary and sufficient condition for q * (•, •|z) to be the optimal minimizer is when

∂L({s i } 2 i=1 ,λ(x),µ(x, z, x)) ∂q( z, x|z)) = 0 when q * (•, •|z) > 0 and ∂L({s i } 2 i=1 ,λ(x),µ(x, z, x)) ∂q( z, x|z)) ≤ 0 when q * (•, •|z) = 0, ∀( z, x) ∈ Z × X .
Since there is nothing to prove for the latter case, we focus on the former case, in which the derivative of the fully unconstrained problem (48) is

z p(z) log q * ( z, x|z) ν * ( z, x) -s 1 d s (z, x) -s 2 d o (z, z) + λ * (z) = 0, (49)
where we took µ(z, z, x) = µ * (z, z, x) = 0 ∀(z, z, x) ∈ Z × Z × X . Moreover, in (49) we have that λ(z) = λ * (z) > 0, ∀z ∈ Z because we require z, x q * ( z, x|z) = 1. Applying this result in (49) and solving with respect to q * (•, •|z) we obtain q * ( z, x|z) = e s 1 ds(z, x)+s 2 do(z, z)-λ(z) ν * ( z, x).

Since z, x q * ( z, x|z) = 1, we average both sides with respect to ( z, x) ∈ Z × X and solve to obtain λ * (z) > 0, which is given by

λ * (z) = log   z, x
e s 1 ds(z, x)+s 2 do(z, z) ν * ( z, x)

  . (51)
Substituting (51) in (50), we obtain the implicit expression of ( 16) for s 1 ≤ 0, s 2 ≤ 0. Moreover, substituting [START_REF] Kountouris | Semantics-empowered communication for networked intelligent systems[END_REF] in (48) we obtain [START_REF] Qin | Semantic communications: Principles and challenges[END_REF], provided that R(D * o , D * s ) > 0, hence we obtain (i). Clearly, the cases in (ii)-(iv) follow as special cases of case (i). This completes the proof.

APPENDIX B PROOF OF THEOREM 2

Recall that the input data and the distortion functions are introduced in Problem 1. We first start with some preliminary calculations. In particular, using [START_REF] Wolf | Transmission of noisy information to a noisy receiver with minimum distortion[END_REF], we can obtain p(z) as follows:

p(z) = x∈{0,1} p(z|x)p(x), which gives p(z) =   p(z = 0) p(z = 1)   =   αβ + (1 -α)γ α(1 -β) + (1 -α)(1 -γ)   . ( 52
)
Using the fact that p(z, x) = p(z|x)p(x) we obtain

p(z, x) =        p(z = 0, x = 0) p(z = 0, x = 1) p(z = 1, x = 0) p(z = 1, x = 1)        =        αβ γ(1 -α) α(1 -β) (1 -α)(1 -γ)        . ( 53
We can now proceed to prove (i).

(i) We will prove this by finding the analytical solutions of {ν * ( z, x), ( z, x) ∈ {0, 1} × {0, 1}}

and {q * ( z, x|z), (z, z, x) ∈ {0, 1} × {0, 1} × {0, 1}}. We first recall, from Theorem 1, that

ν * ( z, x) = z∈{0,1} q * ( z, x|z)p(z), (55)
where {q * ( z, x|z), (z, z, x) ∈ {0, 1} × {0, 1} × {0, 1}} can be computed via [START_REF] Kountouris | Semantics-empowered communication for networked intelligent systems[END_REF].

Writing down (55) results into four third degree polynomial equations with four unknowns and one extra equation that ensures {ν * ( z, x), ( z, x) ∈ {0, 1} × {0, 1}} is a column vector that sums up to one. It can be shown that the system of non-linear equations has the following trivial solutions

ν * ( z, x) =        p * ( z = 0, x = 0) p * ( z = 0, x = 1) p * ( z = 1, x = 0) p * ( z = 1, x = 1)        =                     1 0 0 0        ,        0 1 0 0        ,        0 0 1 0        ,        0 0 0 1                     , (56)
which can be excluded from the general solution because they lead to zero rates, whereas the non-trivial solutions are as follows

ν * ( z, x) =                     d 1 (h 1 -e 1 )-p(z=0)(a 1 h 1 -d 1 e 1 ) (a 1 -d 1 )(e 1 -h 1 ) 0 0 p(z=0)(a 1 h 1 -d 1 e 1 )-a 1 (h 1 -e 1 ) (a 1 -d 1 )(e 1 -h 1 )        ,        b 1 (f 1 -e 1 )-p(z=0)(a 1 f 1 -b 1 e 1 ) (a 1 -b 1 )(e 1 -f 1 ) p(z=0)(a 1 f 1 -b 1 e 1 )-a 1 (f 1 -e 1 ) (a 1 -b 1 )(e 1 -f 1 ) 0 0        , (57)
       c 1 (g 1 -e 1 )-p(z=0)(a 1 g 1 -c 1 e 1 ) (a 1 -c 1 )(e 1 -g 1 ) 0 p(z=0)(a 1 g 1 -c 1 e 1 )-a 1 (g 1 -e 1 ) (a 1 -b 1 )(e 1 -f 1 ) 0        ,        0 c 1 (g 1 -f 1 )-p(z=0)(b 1 g 1 -c 1 f 1 ) (b 1 -c 1 )(f 1 -g 1 ) p(z=0)(b 1 g 1 -c 1 f 1 )-b 1 (g 1 -f 1 ) (b 1 -c 1 )(f 1 -g 1 ) 0        , ( 58
)        0 0 d 1 (h 1 -g 1 )-p(z=0)(c 1 h 1 -d 1 g 1 ) (c 1 -d 1 )(g 1 -h 1 ) p(z=0)(c 1 h 1 -d 1 g 1 )-c 1 (h 1 -g 1 ) (c 1 -d 1 )(g 1 -h 1 )        ,        0 d 1 (h 1 -f 1 )-p(z=0)(b 1 h 1 -d 1 f 1 ) (b 1 -d 1 )(f 1 -h 1 ) 0 p(z=0)(b 1 h 1 -d 1 f 1 )-b 1 (h 1 -g 1 ) (b 1 -d 1 )(f 1 -h 1 )                     , (59)
where a 1 = e s 1 ds(0,0) , b 1 = e s 1 ds(0,1) , c 1 = e s 1 d S (0,0)+s 2 , d 1 = e s 1 ds(0,1)+s 2 , e 1 = e s 1 ds(1,0)+s 2 , f 1 = e s 1 ds(1,1)+s 2 , g 1 = e s 1 ds(1,0) , h 1 = e s 1 ds (1,1) . Since we found analytically the marginal on the i.i.d output process in (57)-(59), we subsequently proceed to find the corresponding explicit expressions of the optimal minimizer for each of the explicit expressions in (57)-(59). Herein, we only give the explicit solution of the optimal minimizer q * ( z, x|z) that corresponds to the left hand side (LHS) solution of (57); the other cases, e.g., RHS matrix of (57), (58), (59), follow by simply substituting in [START_REF] Kountouris | Semantics-empowered communication for networked intelligent systems[END_REF]. Substituting the LHS matrix of (57) in ( 16) we obtain

q * ( z, x|z) =        q( z = 0, x = 0|z = 0) q( z = 0, x = 0|z = 1) q( z = 0, x = 1|z = 0) q( z = 0, x = 1|z = 1) q( z = 1, x = 0|z = 0) q( z = 1, x = 0|z = 1) q( z = 1, x = 1|z = 0) q( z = 1, x = 1|z = 1)        , (60)
where

q * ( z = 0, x = 0|z = 0) = a 1 d 1 (h 1 -e 1 ) + a 1 p(z = 0)(a 1 h 1 -d 1 e 1 ) p(z = 0)(a 1 h 1 -d 1 e 1 )(d 1 -a 1 ) q * ( z = 0, x = 1|z = 0) = p( z = 1, x = 0|x = 0) = 0 q * ( z = 1, x = 1|z = 0) = a 1 p(z = 0)(a 1 h 1 -d 1 e 1 ) -a 1 d 1 (h 1 -e 1 ) p(z = 0)(a 1 h 1 -d 1 e 1 )(d 1 -a 1 ) q * ( z = 1, x = 0|z = 0) = p( z = 1, x = 0|x = 1) = 0 q * ( z = 0, x = 0|z = 1) = d 1 e 1 (h 1 -e 1 ) + e 1 p(z = 0)(a 1 h 1 -d 1 e 1 ) p(z = 1)(a 1 h 1 -d 1 e 1 )(e 1 -h 1 ) q * ( z = 0, x = 1|z = 1) = p( z = 1, x = 0|x = 1) = 0 q * ( z = 1, x = 1|z = 1) = h 1 p(z = 0)(a 1 h 1 -d 1 e 1 ) -a 1 h 1 (h 1 -e 1 ) p(z = 1)(a 1 h 1 -d 1 e 1 )(e 1 -h 1 ) q * ( z = 1, x = 0|z = 1) = p( z = 1, x = 1|x = 1) = 0.
The general explicit solution of the optimal minimizer allows to find the information structure of {q * ( z| x, z), (z, z, x) ∈ {0, 1} × {0, 1} × {0, 1}} and {q * ( x| z, z), (z, z, x) ∈ {0, 1} × {0, 1} × {0, 1}}, respectively, via the following expressions

q * ( z| x, z) = q * ( z, x|z)p(z) z∈{0,1} q * ( z, x|z)p(z) , q * ( x| z, z) = q * ( z, x|z)p(z) x∈{0,1} q * ( z, x|z)p(z) . (61)
Surprisingly, the closed form expressions in (61), admit the following simple structure

q * ( z| x, z) =   1 1 0 0 0 0 1 1   , q * ( x| z, z) =   1 1 0 0 0 0 1 1   , (62)
where for the left hand side (LHS) expression of (62) we have

q * ( z = 0| x = 0, z = 0) = q * ( z = 0| x = 0, z = 1) = 1 q * ( z = 0| x = 1, z = 0) = q * ( z = 0| x = 1, z = 1) = 0 q * ( z = 1| x = 0, z = 0) = q * ( z = 1| x = 0, z = 1) = 0 q * ( z = 1| x = 1, z = 0) = q * ( z = 1| x = 1, z = 1) = 1, (63)
and similarly, for the RHS expression in (62) each component represents the corresponding optimal minimizer given in (63) with z and x, interchanged. The identical information structure in both expressions of (62), reveals the conditional independence q * ( z| x, z) = q * ( z| x) and q * ( x| z, z) = q * ( x| z) or, equivalently, the fact that zxz and xzz. We note that the previous information structure of {q * ( z| x, z), (z, z, x) ∈ {0, 1} × {0, 1} × {0, 1}} and {q * ( x| z, z), (z, z, x) ∈ {0, 1}×{0, 1}×{0, 1}} can be observed if we pick any {ν * ( z, x), ( z, x) ∈ {0, 1} × {0, 1}} from (57)-(59) hence we omit the re-derivation. This proves (i).

(ii) Since we proved (i), then, from Lemma 2, (2), the use of the Lagrange duality theorem [START_REF] Luenberger | Optimization by Vector Space Methods[END_REF] guarantees that the solution is the one for which the Lagrangian, i.e., s 1 or s 2 in our case, yields the greater rates hence R(D * o , D * s ) = max{R(D * o ), R(D * s ))}. This completes the proof.

APPENDIX C PROOF OF THEOREM 3

We start the proof by noting that {p(z), p(z|x) : (x, z) ∈ {0, 1} × {0, 1}} are given by ( 52) and (53), respectively. Moreover, from [START_REF] Gray | A new class of lower bounds to information rates of stationary sources via conditional rate-distortion functions[END_REF], we obtain

d s (z, x) =   d s (z = 0, x = 0) d s (z = 0, x = e) d s (z = 0, x = 1) d s (z = 1, x = 0) d s (z = 1, x = e) d s (z = 1, x = 1)   =   ∞ 1 ∞ ∞ 1 ∞   . (64)
Using (64) and the input data, we obtain that the optimal minimizer has the following structure

q * ( z, x|z) =             
q * ( z = 0, x = 0|z = 0) q * ( z = 0, x = 0|z = 1)

q * ( z = 0, x = e|z = 0) q * ( z = 0, x = e|z = 1)

q * ( z = 0, x = 1|z = 0) q * ( z = 0, x = 1|z = 1) q * ( z = 1, x = 0|z = 0) q * ( z = 1, x = 0|z = 1)

q * ( z = 1, x = e|z = 0) q * ( z = 1, x = e|z = 1)

q * ( z = 1, x = 1|z = 0) q * ( z = 1, x = 1|z = 1) (2), (3). Minimizing over q gives a value F (ν (k) ) between V (ν (k+1) ) and V (ν (k) ), namely,

             =              0
F (ν (k) ) = - z p(z)   log z, x ν (k-1) ( z, x)A(z, z, x)   , (67)
and due to the successive minimizations, we have the nonincreasing sequence . . . ≥ V (ν (k) ) ≥ F (ν (k) ) ≥ V (ν (k+1) ) ≥ F (ν (k+1) ) ≥ . . .. Combining the above two operations, we obtain a recursive definition of ν (k) of the theorem which by construction has positive elements and

V (ν k ) is nonincreasing. Since V (ν (k)
) is bounded, it must converge to some constant number Let q (k+1) = q(ν (k) ). Let ν * be such that V (ν * ) = R(D o , D s ) -s 1 D s -s 2 D o , and let q * = q(ν * ). Now let z, z, x p(z)q * ( z, x|z) log q (k) ( z, x|z) q (k+1) ( z, x|z)

where (a) follows by the fact that we let q (k+1) = q(ν (k) ). Next, we show that the first RHS in (68) is increased if we replace q (k) and ν (k) are replaced by q * and ν * , respectively.

z, z, x p(z)q * ( z, x|z) log q (k) ( z, x|z) ν (k) ( z, x) z, z, x p(z)q * ( z, x|z) log q * ( z, x|z) ν * ( z, x) = z, z, x p(z)q * ( z, x|z) log q (k) ( z, x|z)q * ( z, x|z) ν (k) ( z, x)ν * ( z, x) .

(75)

Applying the quantities of (75) in z p(z)λ(z)A(z, z, x) we can observe that it is always less than one, therefore the lower bound obtain from Lemma 4 holds. Substituting the values of (75)

in that bound we obtain the RHS of [START_REF] Ihara | Information theory -for Continuous Systems[END_REF]. This completes the proof.

  (R(D s ), R(D o )) represent the standard direct and indirect RDFs obtained via their individual distortion criteria, i.e,, R(D o ) = min q( z|z) E{do(z, z)}≤Do I(z; z), and R(D s ) = min q( x|z) E{ ds(z, x)}≤Ds

Case 1 :

 1 R(D o , D s ) (c) ≥ R(D s ) where (c) holds with equality if and only if the condition of inequality (13), (a) holds.Case 2: R(D o , D s ) (d) ≥ R(D o ) where (d) holds with equality if and only if the condition of inequality (13), (b) holds. If both Case 1 and Case 2 are concurrently true, i.e., (10) holds, then, from Lagrange duality theorem [33], we can write the individual unconstrained dual problems for R(D s ) and R(D o ) associated with their corresponding Lagrangian multipliers, say (s 1 , s 2 ), and choose the Lagrangian multiplier that corresponds to the maximal rates between R(D s ) and R(D o ) which means precisely R L (D s , D o ) in (8).

Remark 2 .

 2 (Special case) If we assume that the model of the noisy channel in (22) becomes "deterministic", i.e., β = 1 and γ = 0, then, it can be easily shown that z = x and the problem recovers the well-known robust description setup for binary alphabets studied in [24, Section VII].

Fig. 2 :

 2 Fig. 2: R(D * o , D * s ) for binary alphabets with an equiprobable semantic source and binary symmetric channel with c = 0.15.

Theorem 3 .

 3 Consider the setup in Fig. 1 restricted to Problem 2. Then, for the choice of the semantic distortion penalty in (27), the characterization in (6) satisfies the Markov chain z-z-x and R(D * o , D * s ) = R(D * o ) which can be explicitly computed via (23). Proof: See Appendix C. Interestingly, the choice of the erasure distortion measure in Theorem 3 demonstrates that the amended distortion of the semantic (remote) source allows only the erasures to be sent, which in turn results into the zero rate of the indirect rate distortion problem. This result comes as a rather extreme case of the general result of Theorem 2 and demonstrates the cardinal role of the distortion penalties into the solution. B. Semantic-aware Blahut-Arimoto Algorithm Next, we propose a generalization of the celebrated BA algorithm to treat the case of arbitrary finite alphabet sets with individual distortions.

Remark 3 . ( 1 )

 31 For scalar jointly Gaussian RVs with individual MSE distortion constraints, [19, Corollary 1] "indirectly" showed that R L (D * o , D * s ) in Lemma 2 is tight and also the information structures that are necessary and sufficient for this bound are satisfied. (2) If in (41) we assume that H is an orthogonal matrix, i.e., H T H = I m×m (m-dimensional identity matrix), then, as [20] recognized, both distortion constraints simplify to trace(Σ z| x, z ) ≤ min{D s -trace(Σ n ), D o } (consequence of Lagrange duality theorem) which is nothing more than R L (D o , D s ) of Lemma 2.

Example 2 .

 2 Suppose that p(x) ∼ N (0; Σ x ) and p(z|x) ∼ N (Ax; Σ s ) are randomly chosen such that , using (42), we can compute Σ z = (43), we can compute the design variables (H, Σ n ) as follows Solving (41) for Σ z| z, x ≻ 0 using the designed values obtained in (44), results into Fig.3. For this simulation, we note that the optimization problem (41) is well-defined if Σ z| z, x ≻ 0 and D s > D min s = trace(Σ n ) = 0.8315 (for this example).

Fig. 3 :

 3 Fig. 3: R(D * o , D * s ) for jointly Gaussian RVs with individual MSE distortion constraints.

  ) Moreover, from (52), (53) and the fact that d s (z, x) = x∈X p(x|z)d s (x, x) (from the characterization in Lemma 1), we can obtain d s (z, x) as follows d s (z, x)

e s 2 ν

 2 0 ν * ( z=0, x=e) ν * ( z=0, x=e)+e s 2 ν * ( z=1, x=e) e s 2 ν * ( z=0, x=e) e s 2 ν * ( z=0, x=e)+ν * ( z=1, x=e) * ( z=1, x=e) ν * ( z=0, x=e)+e s 2 ν * ( z=1, x=e) ν * ( z=1, x=e) e s 2 ν * ( z=0, x=e)+ν * ( z=1, x=e) ν * ( z = 0, x = e) and ν * ( z = 1, x = e) can take the trivial solutions 0 or 1 and a non-trivial solution of the formν * ( z = 0, x = e) = p(z = 0)(1 + e s 2 ) -e s 2 1 -e s 2 , ν * ( z = 1, x = e) = 1 -p(z = 0)(1 + e s 2 ) 1 -e s 2 .(66)The explicit structure of (65) reveals that the optimal solution to this problem is parametrized only by the Lagrangian multiplier s 2 < 0, which further means that this solution should beR(D * o , D * s ) = R(D * o ) (fromTheorem 1). The latter implies that the Markov chain zzx holds. A way to compute analytically R(D * o , D * s ) is the following. Find s * 2 < 0 from (18) using the explicit expressions (65) and (66). This will give s * 2 = log D * o 1-D * o . Then, by substituting all the pieces together in R(D * o , D * s ) = E q * ( z, x|z) p * ( z, x)we obtain[START_REF] Witsenhausen | Indirect rate distortion problems[END_REF]. This completes the proof.APPENDIX D PROOF OF THEOREM 4 Let V (ν) ≜ I(p; q(ν (k) )) -s 1 D s (ν) -s 2 D o (ν) with D s (ν) = z, z, x d s (z,x)q(ν)p(z) and D o (ν) = z, z, x d o (z, z)q(ν)p(z). To show that V (ν) is nonincreasing, we can use Lemma 3,

V

  ∞ . It remains to show that V ∞ = R(D o , D s ) -s 1 D s -s 2 D o .

  )q * ( z, x|z) log q (k) ( z, x|z) ν (k) ( z, x) z, z, x p(z)q * ( z, x|z) log A(z, z, x) + z, z, x p(z)q * ( z, x|z) log   z, x ν (k) ( z, x)A(z, z, x)   ,

  , x p(z)q (k) ( z, x|z) ν * ( z, x) ν (k) ( z, x) z, z, x p(z)q * ( z, x|z) matrix that gives D s and D o . Then, the following inequality holdsR(D o , D s ) , z, x)ν( z, x) z, x A(z, z, x)ν( z, x) ( z q( z, x|z)p(z))   q( z, x|z)p(z)follows by definition of (6); (b) follows from (73); (c) follows after some algebra and substituting (36), (37) and (73).Next, we prove[START_REF] Ihara | Information theory -for Continuous Systems[END_REF]. Using Lemma 4, we have the boundR(D o , D s ) ≥ s 1 D s + s 2 D o + z∈Z p(z) log λ(z)where λ is any vector such that z p(z)λ(z)A(z, z, x) ≤ 1. To prove the lower bound, we letc max ( z, x) = maxz, x c( z, x), and λ(z) =   c max ( z, x) z, x A(z, z, x)ν( z, x)

  26, Theorem 10.3.1], [25, Example 2.7.1], yielding

TABLE I :

 I Comparison of the optimal values obtained via (41), (45). The simulation studies are performed on CVX platform.

If β = γ then we end up having p(x, z) = p(x)p(z), which is not consistent with the setup we assume in Section II hence this scenario is not allowed.

The result for c ∈ [ 1 2 , 1] can be treated similarly.

where (b) follows from the inequality log(x) ≤ x -1 for x > 0; (c) because both terms after the inequality average to 1. As a result, we can bound from above the LHS of (68) as follows

where (c) follows from (69) and the identify q * ( z, x) = z q * ( z, x|z)p(z); (d) follows by definition of V (ν * ) and (67). Now by summing on k until some finite K, we obtain

where (e) follows from (70); (f ) follows from the telescopic sum of the logarithmic ratio; (g) follows from Kullback's discrimination inequality [40, Theorem 1], i.e., the information is a measurement that cannot be increased by subsequent processing.

Finally, with a change of sign in the LHS of (71) we obtain the inequality

).

The RHS in (72) is a constant independent of K and the LHS is smaller than this constant for any K. Since F (ν (k) ) is a larger value than V (ν * ) and is nonincreasing, then, the term in the brackets is positive and must vanish as K → ∞. This further means that

which completes the proof.

APPENDIX E PROOF OF THEOREM 5

We first prove [START_REF] Grant | CVX: Matlab software for disciplined convex programming[END_REF]. Recall that q( z, x|z) = A(z, z, x)ν( z, x) z, x A(z, z, x)ν( z, x) ,