BLASSO Extends Mixture Recovery to Unknown Diagonal Covariances

Gaussian mixture estimation is hard in the regime this paper targets: the number of components is unknown, each component has its own diagonal covariance, and the estimator must work over a continuous parameter space rather than a fixed grid. Classical likelihood methods can fit such models, but they do not by themselves give an order-free sparse recovery theory. Earlier BLASSO analyses handled mixture-like inverse problems with stronger covariance assumptions, often known and shared covariances. This paper moves that theory to multivariate Gaussian mixtures with component-specific diagonal covariance parameters.

The authors model the mixture as a nonnegative discrete measure over component parameters. A point $x=(t,u)$ contains a mean vector $t\in\mathbb{R}^d$ and diagonal scale parameters $u$, and the target density is obtained by applying a Gaussian feature map $\Phi$ to the measure. The estimator is a Beurling-LASSO over measures: it minimizes a data-fitting term plus a total-variation penalty, so sparsity is imposed directly in the continuous space of Gaussian components.

Core Contribution

The main contribution is a recovery theory for a more flexible GMM than prior BLASSO settings. The paper proves that, under a non-degenerate dual-certificate condition, the estimator controls mass away from the true components, controls mass inside local neighborhoods of each component, and yields parameter recovery once those local mass controls are combined with the paper's reweighting map.

The technical point is that unknown diagonal covariances break translation invariance. The kernel induced by the normalized Gaussian feature map depends on both location and scale, so earlier certificate constructions for translation-invariant kernels do not transfer directly. The paper builds a new geometric analysis around a Fisher-Rao metric and a semi-distance adapted to the $(t,u)$ parametrization. That semi-distance is the object used in the separation condition and in the local recovery bounds.

Technical Approach

The estimator is analyzed through the normalized kernel $K_{\mathrm{norm}}$ and dual certificates of the form

These certificates interpolate the signs and first-order conditions at the true atoms. The authors then prove that, if the true atoms are separated in the proposed semi-distance, the certificates are non-degenerate: they stay strictly below one away from true components and have controlled local curvature near each component. The explicit separation theorem assumes $s\geq 2$, $0<\tau\leq u_{\min}$, compact parameter domain $\mathcal{X}\subset\mathbb{R}^d\times[u_{\min},u_{\max}]^d$, and uses the Fisher-Rao distance $\vartheta_g$. The stated sufficient separation has constants including $r=0.3025/\sqrt{d}$, $c_p=2$, and a term $\Delta=2\sqrt{11.9+3\ln(d+6.62)+\ln(s-1)}$.

Figure 2 illustrates why the theorem is conservative. In the displayed two-component synthetic mixture with $n=400$, weights $(12,12)$, means $(-1,0)$ and $(1,0)$, and scales $(1,2)$ and $(3,1)$, the empirical separation for $\tau=0.8$ is about 1.1, while the theorem's sufficient threshold is about 8.6.

Results and Analysis

The headline results are formal rates, not a broad empirical benchmark. For estimation, Theorem 3.1 and Proposition 3.1 give expected bounds for mass in far and near regions when the regularization is chosen as $\kappa=\rho_n/\sqrt{c_p}$. The paper states local-region trade-offs: choosing $r_e=(\ln n)^{-\alpha}$ gives a bound of order $s(\ln n)^{2\alpha}/\sqrt{n\tau^{d/2}}$, while $r_e=n^{-\alpha}$ with $\alpha<1/4$ gives order $s/n^{1/2-2\alpha}/\tau^{d/2}$. Corollary 3.1 translates local mass control into weight recovery using $r_e=n^{-1/6}$, yielding an $n^{-1/6}$ dependence up to constants involving sparsity, $\tau$, dimension, and the reweighting factor $W$.

For density prediction, the paper separates two regimes. With small regularization, Proposition 4.1 obtains

without assuming dual certificates. With larger regularization and Assumption 1, Theorem 4.1 gives a sparsity-dependent prediction bound of order $s(\ln n)^{d/2}/n$. The authors explicitly compare this with direct kernel density estimation, whose rate degrades as $(\ln n)^{d/2}/(n^{4/(4+d)}u_{\min}^{4d/(4+d)})$ in their discussion. The editorial reading is that the theory justifies continuous sparse estimation when component recovery matters; if only density prediction matters, the small-regularization regime is cleaner because it avoids certificate assumptions.

Figure 3 shows a limited computational check: CPGD initialized with three particles on the same two-component synthetic target reduces the training loss and moves particle weights, means, and scales toward the two target atoms, with one extra particle effectively suppressed. This supports the geometry qualitatively, but it is not an empirical study of scaling, noise sensitivity, or high-dimensional behavior.

Limitations

The paper's evidence is mainly mathematical. The separation condition is explicit but conservative, as the authors' own example shows with a threshold around 8.6 for a case whose observed separation is about 1.1. Several rates hide constants depending on dimension, compactness parameters, $\tau$, and the target mixture. The experiments are illustrative rather than comparative, and the algorithmic side is not evaluated against EM, likelihood penalization, NPMLE solvers, or other practical GMM estimators.