On the entropy of a noisy function
August 06, 2015 Β· Declared Dead Β· π IEEE Transactions on Information Theory
"No code URL or promise found in abstract"
Evidence collected by the PWNC Scanner
Authors
Alex Samorodnitsky
arXiv ID
1508.01464
Category
cs.IT: Information Theory
Cross-listed
math.CO
Citations
39
Venue
IEEE Transactions on Information Theory
Last Checked
6 months ago
Abstract
Let $0 < Ξ΅< 1/2$ be a noise parameter, and let $T_Ξ΅$ be the noise operator acting on functions on the boolean cube $\{0,1\}^n$. Let $f$ be a nonnegative function on $\{0,1\}^n$. We upper bound the entropy of $T_Ξ΅ f$ by the average entropy of conditional expectations of $f$, given sets of roughly $(1-2Ξ΅)^2 \cdot n$ variables. In information-theoretic terms, we prove the following strengthening of "Mrs. Gerber's lemma": Let $X$ be a random binary vector of length $n$, and let $Z$ be a noise vector, corresponding to a binary symmetric channel with crossover probability $Ξ΅$. Then, setting $v = (1-2Ξ΅)^2 \cdot n$, we have (up to lower-order terms): $$ H\Big(X \oplus Z\Big) \ge n \cdot H\left(Ξ΅~+~ (1-2Ξ΅) \cdot H^{-1}\left(\frac{{\mathbb E}_{|B| = v} H\Big(\{X_i\}_{i\in B}\Big)}{v}\right)\right) $$ As an application, we show that for a boolean function $f$, which is close to a characteristic function $g$ of a subcube of dimension $n-1$, the entropy of $T_Ξ΅ f$ is at most that of $T_Ξ΅ g$. This, combined with a recent result of Ordentlich, Shayevitz, and Weinstein shows that the "Most informative boolean function" conjecture of Courtade and Kumar holds for high noise $Ξ΅\ge 1/2 - Ξ΄$, for some absolute constant $Ξ΄> 0$. Namely, if $X$ is uniformly distributed in $\{0,1\}^n$ and $Y$ is obtained by flipping each coordinate of $X$ independently with probability $Ξ΅$, then, provided $Ξ΅\ge 1/2 - Ξ΄$, for any boolean function $f$ holds $I\Big(f(X);Y\Big) \le 1 - H(Ξ΅)$.
Community Contributions
Found the code? Know the venue? Think something is wrong? Let us know!
π Similar Papers
In the same crypt β Information Theory
R.I.P.
π»
Ghosted
R.I.P.
π»
Ghosted
A Vision of 6G Wireless Systems: Applications, Trends, Technologies, and Open Research Problems
R.I.P.
π»
Ghosted
Towards Smart and Reconfigurable Environment: Intelligent Reflecting Surface Aided Wireless Network
π
π
The Cartographer
Wireless Communications with Unmanned Aerial Vehicles: Opportunities and Challenges
R.I.P.
π»
Ghosted
Reconfigurable Intelligent Surfaces for Energy Efficiency in Wireless Communication
π
π
The Cartographer
An Overview of Signal Processing Techniques for Millimeter Wave MIMO Systems
Died the same way β π» Ghosted
R.I.P.
π»
Ghosted
Federated Learning: Strategies for Improving Communication Efficiency
R.I.P.
π»
Ghosted
In-Datacenter Performance Analysis of a Tensor Processing Unit
R.I.P.
π»
Ghosted
Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning
R.I.P.
π»
Ghosted