On generalized Csiszár-Kullback inequalities

Abstract

The classical Csiszar-Kullback inequality bounds the L^{1}-distance of two probability densities in term of their relative (convex) entropies. Here we generalize such inequalities to not necessarily normalized and possibly non-positive L^{1} functions. Also, our generalized Csiszar-Kullback inequalities are in many important cases sharper than the classical ones (in terms of the functional dependence of the L^{1} bound on the relative entropy). Moreover our construction of these bounds is rather elementary.