Resolution-enhanced radar/SAR imaging: an experiment design framework combined with neural network-adapted variational analysis regularization

Abstract

The convex optimization-based descriptive experiment design regularization (DEDR) method is aggregated with the neural network (NN)-adapted variational analysis (VA) approach for adaptive high-resolution sensing into a unified DEDR -VA-NN framework that puts in a single optimization frame high-resolution radar/SAR image formation in uncertain operational scenarios, adaptive despeckling and dynamic scene image enhancement for a variety of sensing modes. The DEDR -VA-NN method outperforms the existing adaptive radar imaging techniques both in resolution and convergence rate. The simulation examples are incorporated to illustrate the efficiency of the proposed DEDR-VA-related imaging techniques.

1. Introduction

In this article, we consider the problem of enhanced remote sensing (RS) imaging stated and treated as an ill-posed nonlinear inverse problem with model uncertainties. The problem at hand is to perform high-resolution reconstruction of the power spatial spectrum pattern (SSP) of the wavefield scattered from the extended remotely sensed scene via space-time adaptive processing of finite recordings of the imaging radar/SAR data distorted in a stochastic uncertain measurement channel. The SSP is defined as a spatial distribution of the power (i.e., the second-order statistics) of the random wavefield backscattered from the remotely sensed scene observed through the integral transform operator [1, 2]. Such an operator is explicitly specified by the employed radar/SAR signal modulation and is traditionally referred to as the signal formation operator (SFO) [2, 3]. The operational uncertainties are attributed to inevitable random signal perturbations in inhomogeneous propagation medium with unknown statistics, possible imperfect radar calibration, and uncontrolled sensor displacements or carrier trajectory deviations in the SAR case. The classical imaging with an array radar or SAR implies application of the method called "matched spatial filtering (MSF)" to process the recorded data signals [2, 3]. A number of approaches had been proposed to design the constrained regularization techniques for improving the resolution in the SSP obtained by ways different from the MSF, e.g., [1–9] but without aggregating the minimum risk (MR) descriptive estimation strategies with convex projection regularization. In [7], an approach was proposed to treat the uncertain RS imaging problems that unifies the MR spectral estimation strategy with the worst case statistical performance (WCSP) optimization-based convex regularization resulting in the descriptive experiment design regularization (DEDR) method. Next, the variational analysis (VA) framework has been combined with the DEDR in [2, 9] to satisfy the desirable descriptive properties of the reconstructed RS images, namely: (i) convex optimization-based maximization of spatial resolution balanced with noise suppression, (ii) consistency, (iii) positivity, (iv) continuity and agreement with the data. In this study, we extend the developments of the DEDR and VA techniques originated in [2, 7, 9] by performing the aggregation of the DEDR and VA paradigms and next putting the RS image enhancement/reconstruction tasks into the unified neural network (NN)-adapted computational frame addressed as a unified DEDR-VA-NN method. We have designed a family of such significantly speeded-up DEDR-VA-related algorithms, and performed the simulations to illustrate the effectiveness of the proposed high-resolution DEDR-VA-NN-based image enhancement/fusion approach.

The rest of the article is organized as follows. In Section 2, we provide the formalism of the radar/SAR inverse imaging problem at hand with necessary experiment design considerations. In Section 3, we adapt the celebrated maximum likelihood (ML) inspired amplitude phase estimation (APES) technique for array sensor/SAR imaging. The unified DEDR-VA framework for high-resolution radar/SAR imaging in uncertain scenarios is conceptualized in Section 4, adapted to the NN-oriented sensor systems/methods fusion mode in Section 5, next, is followed by illustrative simulations in Sections 6 and the conclusion in Section 7.

2. Problem formalism

The general mathematical formalism of the problem at hand is similar in notation and structural framework to that described in [2, 7, 9] and some crucial elements are repeated for convenience to the reader. Following [1, 2, 9], we define the model of the observation RS wavefield u by specifying the stochastic equation of observation (EO) of an operator form u = $\mathcal{S}$e + n, where e = e(r), represents the complex scattering function over the probing surface R ∋ r, n is the additive noise, u = u(p), is the observation field, p = (t, ρ) defines the time (t)-space(ρ) points in the temporal-spatial observation domain p ∈ P = T × P (t ∈ T, ρ ∈ P) (in the SAR case, ρ = ρ(t) specifies the carrier trajectory [7]), and the kernel-type integral SFO $\mathcal{S}:E\left(R\right)\to U\left(P\right)$ defines a mapping of the source signal space $E\left(R\right)$ onto the observation signal space $U\left(P\right)$. The metrics structures in the corresponding Hilbert signal spaces $U\left(P\right)$$E\left(R\right)$ are imposed by scalar products, ${\left[u,u^{\prime }\right]}_U=\underset{P}{\int }u\left(\mathbf{p}\right)u^{\prime }*\left(\mathbf{p}\right)d\mathbf{p},$, ${\left[e,e^{\prime }\right]}_E=\underset{R}{\int }e\left(\mathbf{r}\right)e^{\prime }*\left(\mathbf{r}\right)d\mathbf{r},$ respectively [1]. The functional kernel S(p, r) of the SFO $\mathcal{S}$ is referred to as the unit signal[2] determined by the time-space modulation employed in a particular RS system. In the case of uncertain operational scenarios, the SFO is randomly perturbed [7], i.e. $\tilde{\mathcal{S}}=\mathcal{S}+$$\Delta \mathcal{S}$ where $\Delta \mathcal{S}$ pertains to the random uncontrolled perturbations, usually with unknown statistics. The fields e, n, u. are assumed to be zero-mean complex valued Gaussian random fields [1, 7]. Next, since in all RS applications the regions of high correlation of e(r) are always small in comparison with the resolution element on the probing scene [1–3], the signals e(r) scattered from different directions r, r ' ∈ R of the remotely sensed scene R are assumed to be uncorrelated with the correlation function R _e (r, r') = 〈e(r)e*(r') 〉 = b(r) δ(r-r');r,r'∈R where b(r) = 〈e(r)e*(r) 〉 = 〈|e(r)|²〉; r∈R represents the power SSP of the scattered field [1]. The problem of high-resolution RS imaging is to develop a framework and related method(s) that perform optimal estimation of the SSP (referred to as a scene image) from the available radar/SAR data measurements. It is noted that in this study we are going to develop and follow the unified DEDR-VA-NN framework.

The RS radar/SAR system-oriented finite-dimensional (i.e., discrete-form) approximation of the EO is given by [7]

$$\mathbf{u}=\tilde{\mathbf{S}}\mathbf{e}+\mathbf{n}=\mathbf{Se}+\mathbf{\Delta }\mathbf{e}+\mathbf{n},$$

(1)

in which the disturbed M×K SFO matrix $\tilde{\mathbf{S}}$ = S + Δ is the discrete-form approximation of the integral SFO for the uncertain operational scenario, and e, n, u represent zero-mean vectors composed of the sample (decomposition) coefficients {e _k , n _m , u _m ; k = 1,...,K; m = 1,...,M}, respectively [1–3]. These vectors are characterized by the correlation matrices: R _e = D = D(b) = diag(b) (a diagonal matrix with vector b at its principal diagonal), R _n , and R _u = < $\tilde{\mathbf{S}}{\mathbf{R}}_{\mathbf{e}}{\tilde{\mathbf{S}}}^{+}$ > _p(Δ₎ + R _n , respectively, where <·> _p(Δ)defines the averaging performed over the randomness of Δ characterized by the usually unknown probability density function p(Δ), and superscript "+" stands for Hermitian conjugate. Vector b composed of the elements, {b _k =$\mathcal{B}\left\{e_k\right\}$ = <e _k e _k *> = <|e _k |²>; k = 1,...,K} is referred to as a K-D vector-form approximation of the SSP, where $\mathcal{B}$ represents the second-order statistical ensemble averaging operator [1, 2]. The SSP vector b is associated with the lexicographically ordered pixel-framed image [1, 7]. The corresponding conventional K _y ×K _x rectangular frame-ordered scene image B = {b(k _x , k _y ); k _x , = 1,...,K _x ; k _v , = 1,...,K _y } relates to its lexicographically ordered vector-form representation b = $\mathcal{L}\left\{\mathbf{B}\right\}$ = {b(k); k = 1,...,K = K _y ×K _x } via the standard row-by-row concatenation (i.e., lexicographical reordering) procedure, B = ${\mathcal{L}}^{-1}${b} [1]. It is noted that in the simple case of certain operational scenario [2, 3], the discrete-form (i.e., matrix-form) SFO S is assumed to be deterministic, i.e., the random perturbation term in (3) is irrelevant, Δ = 0.

The enhanced RS imaging problem is stated generally as follows: to map the scene pixel-framed image $\widehat{\mathbf{B}}$ via lexicographical reordering $\widehat{\mathbf{B}}={\mathcal{L}}^{-1}\left\{\widehat{\mathbf{b}}\right\}$ of the SSP vector estimate $\widehat{\mathbf{b}}$ reconstructed from whatever available measurements of independent realizations of the recorded data (1). The reconstructed SSP vector $\widehat{\mathbf{b}}$ is an estimate of the second-order statistics of the scattering vector e observed through the perturbed SFO and contaminated with noise; hence, the imaging problem at hand must be qualified and treated as a statistical nonlinear uncertain inverse problem [1, 7, 9]. The enhanced high-resolution imaging implies solution of such inverse problem in some optimal way. We know that in this article we intend to develop and follow the unified DEDR-VA framework, next adapted to NN-based computational implementation.

3. Adaptation of APES technique for array sensor/SAR imaging

In this section, we perform an extension of the recently proposed high-resolution ML inspired APES, i.e., the ML-APES method [6], for solving the SSP reconstruction inverse problem via its modification adapted to radar imaging of distributed RS scenes. In the considered low snapshot sample case (e.g., one recorded SAR trajectory data signal in a single look SAR sensing mode [7]), the sample data covariance matrix Y = $\left(1∕J\right){\sum }_{j=1}^J{\mathbf{u}}_{\left(j\right)}{{\mathbf{u}}^{+}}_{\left(j\right)}$ is rank deficient (rank-1 in the single radar snapshot and single look SAR sensing modes, J = 1). The convex optimization problem of minimization of the negative likelihood function $\mathsf{\text{lndet{}}{}_{\mathbf{R}}^{\mathbf{u}}\mathsf{\text{} + tr}}\left\{{\mathbf{R}}_{\mathbf{u}}^{-1}\mathbf{Y}\right\}$ with respect to the SSP vector b subject to the convexity guaranteed non-negativity constraint results in the celebrated APES estimator [6]

$${\widehat{b}}_k=\frac{{\mathbf{s}}_k^{+}{\mathbf{R}}_{\mathbf{u}}^{-1}{\mathbf{YR}}_{\mathbf{u}}^{-1}{}_{\mathbf{s}}^k}{{\left({\mathbf{s}}_k^{+}{\mathbf{R}}_{\mathbf{u}}^{-1}{}_{\mathbf{s}}^k\right)}^2};k=1,...,K.$$

(2)

In the APES terminology (as well as in the minimum variance distortionless response (MVDR) and other ML-related approaches [1, 4, 6] etc.), s_k represents the so-called steering vector in the k th look direction, which in our notational conventions is essentially the k th column vector of the regular SFO matrix S. The numerical implementation of the APES algorithm (2) assumes application of an iterative fixed point technique by building the model-based estimate ${\widehat{\mathbf{R}}}_{\mathbf{u}}={\mathbf{R}}_{\mathbf{u}}\left({\widehat{\mathbf{b}}}_{\left[i\right]}\right)$ of the unknown covariance R _u from the latest (i th) iterative SSP estimate ${\widehat{\mathbf{b}}}_{\left[i\right]}$ with the zero step initialization ${\widehat{\mathbf{b}}}_{\left[0\right]}={\widehat{\mathbf{b}}}_{MSF}$ computed applying the conventional MSF estimator [2].

In the vector form, the algorithm (2) can be expressed as

$${\widehat{\mathbf{b}}}_{\mathsf{\text{APES}}}={\widehat{\mathbf{b}}}^{\left(1\right)}={\left\{{\mathbf{F}}^{\left(1\right)}\mathbf{u}{\mathbf{u}}^{+}{{\mathbf{F}}^{\left(1\right)}}^{+}\right\}}_{\mathsf{\text{diag}}}=\left({\mathbf{F}}^{\left(1\right)}\mathbf{u}\right)\bullet {\left({\mathbf{F}}^{\left(1\right)}\mathbf{u}\right)}^{*},$$

(3)

Where · defines the Schur-Hadamar [1] (element wise) vector/matrix product, F_APES = F⁽¹⁾$={}_{\widehat{\mathbf{DS}}}^{+}{\mathbf{R}}_{\mathbf{u}}^{-1}\left(\widehat{\mathbf{b}}\right)$ represents the APES matrix-form solution operator (SO), in which

$$\widehat{\mathbf{D}}=\mathbf{D}\left(\widehat{\mathbf{b}}\right)=\mathsf{\text{diag}}\left(\widehat{\mathbf{b}}\right)\mathsf{\text{and}}{\mathbf{R}}_{\mathbf{u}}^{-1}\left(\widehat{\mathbf{b}}\right)={\left(\mathbf{S}\widehat{\mathbf{D}}{\mathbf{S}}^{+}+{\mathbf{R}}_{\mathbf{n}}\right)}^{-1}$$

(4)

where operator {·}_diag returns the vector of a principal diagonal of the embraced matrix. The algorithmic structure of the vector-form nonlinear (i.e., solution-dependent) APES estimator (3) guarantees positivity but does not guarantee the consistency. In the real-world uncertain (rank deficient) RS operational scenarios, the inconsistency inevitably results in speckle corrupted images unacceptable for further processing and interpretation. To overcome these limitations, in the next section we extend the unified DEDR-VA framework of [2, 9] for the considered here uncertain operational scenarios to guarantee consistency and significantly speed-up convergence.

4. Unified DEDR-VA framework for high-resolution radar/SAR imaging in uncertain scenarios

4.1. DEDR-VA approach

The DEDR-VA-optimal SSP estimate $\widehat{\mathbf{b}}$ is to be found as the regularized solution to the nonlinear equation [7]

$${\widehat{\mathbf{b}}}_{\mathsf{\text{DEDR}}}=\mathcal{P}{\left\{{\mathbf{F}}_{\mathsf{\text{DEDR}}}{\mathbf{YF}}_{\mathsf{\text{DEDR}}}^{+}\right\}}_{\mathsf{\text{diag}}}=\mathcal{P}{\left\{\mathbf{D}\left({\widehat{\mathbf{b}}}_{\mathsf{\text{DEDR}}}\right)\right\}}_{\mathsf{\text{diag}}},$$

(5)

where F_DEDR represents the adaptive (i.e., dependent on the SSP estimate $\widehat{\mathbf{b}}$) matrix-form DEDR SO and $\mathcal{P}$ is the VA inspired regularizing projector onto convex solution sets (POCS). Two fundamental issues constitute the benchmarks of the modified DEDR-VA estimator (5) that distinguish it from the previously developed kernel SSP reconstruction algorithm [2], the DEDR method [7, 9] and the detailed above APES estimator (3). First, we reformulate the strategy for determining the DEDR SO F_DEDR in (5) in the MR-inspired WCSP convex optimization setting [1, 7], i.e., as the MR-WCSP constrained DEDR convex optimization problem (specified by [7, Equations 8 and 11]) to provide robustness of the SSP vector estimates against possible model uncertainties. The second issue relates to the VA inspired problem-oriented codesign of the POCS regularization operator $\mathcal{P}$ in (5) aimed at satisfying intrinsic and desirable properties of the solution such as positivity, consistency, model agreement (e.g., adaptive despeckling with edge preservation), and rapid convergence [1, 8]. The solution to the MR-WCSP conditioned optimization problem [7, Equation 43] yields the DEDR-optimal SO

$${\mathbf{F}}_{\mathsf{\text{DEDR}}}={\mathbf{F}}^{\mathsf{\text{(2)}}}=\mathbf{K}{\mathbf{S}}^{+}{\mathbf{R}}_{\Sigma }^{-1}$$

(6)

where $\mathbf{K}={\left({\mathbf{S}}^{+}{\mathbf{R}}_{\Sigma }^{-1}\mathbf{S}+\alpha {\mathbf{A}}^{-1}\right)}^{-1}$ defines the so-called reconstruction operator (with the regularization parameter α and stabilizer A^-1), and ${\mathbf{R}}_{\Sigma }^{-1}$ is the inverse of the diagonal loaded noise correlation matrix [7]R _Σ = N_ΣI with the composite noise power N_Σ = N₀+β, the additive observation noise power N₀ augmented by the loading factor β = γη/α ≥ 0 adjusted to the regularization parameter α, the Loewner ordering factor γ > 0 of the SFO S[1] and the uncertainty bound η imposed by the MR-WCSP conditional maximization (see [7, 8] for details).

It is noted that other feasible adjustments of the processing-level degrees of freedom {α, N_Σ, A} summarized in [7, 8] specify the family of relevant POCS-regularized DEDR-related (DEDR-POCS) techniques that we unify here in the following general form

$${\widehat{\mathbf{b}}}^{\left(p\right)}=\mathcal{P}{\left\{{\mathbf{F}}^{\left(p\right)}\mathbf{Y}{\mathbf{F}}^{\left(p\right)+}\right\}}_{\mathsf{\text{diag}}}=\mathcal{P}{\left\{{\mathbf{K}}^{\left(p\right)}\mathbf{Q}{\mathbf{K}}^{\left(p\right)+}\right\}}_{\mathsf{\text{diag}}};p=\mathsf{\text{1}},\mathsf{\text{ 2}},\mathsf{\text{ 3}},\dots ,P$$

(7)

where Q = S⁺YS defines the MSF measurement statistics matrix independent on the solution $\widehat{\mathbf{b}}$, and different (say P) reconstruction operators {K^(p); p = 1,...,P} specified for P different feasible assignments to the processing degrees of freedom {α, N_Σ, A} define the corresponding DEDR-POCS estimators (7) with the relevant SO's {F^(p)= K^(p)S⁺; p = 1,...,P}.

4.2. Convergence guarantees

Following the VA regularization formalism [1, 7, 9], the POCS regularization operator $\mathcal{P}$ in (7) could be constructed as a composition of projectors ${\mathcal{P}}_n$ onto convex sets ${ℂ}_n$; n = 1,...,N with non-empty intersection, in which case the (7) is guaranteed to converge to a point in the intersection of the sets {${ℂ}_n$} regardless of the initialization ${\widehat{\mathbf{b}}}_{\left[0\right]}$ that is a direct sequence of the fundamental theorem of POCS (see [7, Part I, Appendix B]). Also, any operator that acts in the same convex set, e.g., kernel-type windowing operator (WO) can be incorporated into such composite regularization operator $\mathcal{P}$ to guarantee the consistency [1]. The RS system-oriented experiment design task is to make the use of the POCS regularization paradigm (5) employing the practical imaging radar/SAR-motivated considerations that we perform in the next section.

4.3. VA-motivated POCS regularization

To approach the superresolution performances in the resulting SSP estimates (5), (7), we propose to follow the VA inspired approach [2, 7, 9] to specify the composite POCS regularizing operator

$$\mathcal{P}={\mathcal{P}}_2{\mathcal{P}}_1.$$

(8)

The ${\mathcal{P}}_2$ in (8) represents the convergence-guaranteed projector onto the nonnegative convex solution set (the POCS operator) specified as the positivity operator, ${\mathcal{P}}_2={\mathcal{P}}_{+}$, that has an effect of clipping off all the negative values [1], and ${\mathcal{P}}_1$ is an anisotropic WO that we construct here following the VA formalism [2, 9] as a metrics inducing operator

$${\mathcal{P}}_1=\mathbf{M}=m^{\left(0\right)}\mathbf{I}+m^{\left(1\right)}{\nabla }^2$$

(9)

that specifies the metrics structure in the K-D solution/image space $B_{\left(K\right)}∍\mathbf{b}$ defined by the squared norm [2, 9]

$$\begin{array}{c}\mid \mathbf{b}{\mid }_{B_{\left(K\right)}}^2=\left[\mathbf{b},\mathbf{Mb}\right]=m^{\left(0\right)}\sum _{k_x,k_y=1}^{K_x,K_y}{\left(b\left(k_x,k_y\right)\right)}^2\\ +m^{\left(1\right)}\sum _{k_x,k_y=1}^{K_x,K_y}{\left(b\left(k_x,k_y\right)-\frac{1}{4}\left(\begin{array}{c}b\left(k_x-1,k_y\right)+b\left(k_x+1,k_y\right)\\ +b\left(k_x,k_y-1\right)+b\left(k,k_y+1\right)\end{array}\right)\right)}^2.\end{array}$$

(10)

The second sum on the right-hand side of (10) is recognized to be a 4-nearest-neighbors difference-form approximation of the Laplacian operator ${\nabla }_{\mathbf{r}}^2$ over the spatial coordinate r, while m⁽⁰⁾ and m⁽¹⁾ represent the nonnegative real-valued scalars that control the balance between two metrics measures defined by the first and the second sums at the right-hand side of (10). In the equibalanced case, m⁽⁰⁾ = m⁽¹⁾ = 1, the same importance is assigned to the both metrics measures, in which case (9) specifies the discrete-form approximation to the Sobolev metrics inducing operator $\mathcal{M}=m^{\left(0\right)}\mathcal{I}+m^{\left(1\right)}{\nabla }_{\mathbf{r}}^2$ in the relevant continuous-form solution space $B\left(R\right)∍b\left(\mathbf{r}\right)$, where $\mathcal{I}$ defines the identity operator [2]. Incorporating in (9) ${\mathcal{P}}_1$= $\mathcal{M}$ for the continuous model and ${\mathcal{P}}_1$ = M for the discrete-form image model, respectively, specifies the consistency-guaranteed anisotropic kernel-type windowing [2, 9] because it controls not only the SSP (image) discrepancy measure but also its gradient flow over the scene.

4.4. DEDR-VA-optimal dynamic SSP reconstruction

The transformation of (5) into the contractive iterative mapping format yields

$${\widehat{\mathbf{b}}}_{\left[i+1\right]}={\widehat{\mathbf{b}}}_{\left[i\right]}+\tau {\mathcal{P}}_{+}\left\{\mathbf{Mq}-\mathbf{M}{\mathbf{\Phi }}_{\mathbf{D}\left[i\right]}{\widehat{\mathbf{b}}}_{\left[i\right]}\right\};i=0,\mathsf{\text{ 1}},\mathsf{\text{ 2}},\dots$$

(11)

initialized by the conventional low-resolution MSF image

$${\widehat{\mathbf{b}}}_{\left[0\right]}=\mathbf{q}={\left\{\mathbf{Q}\right\}}_{\mathsf{\text{diag}}}={\left\{{\mathbf{S}}^{\mathsf{\text{ +}}}\mathbf{YS}\right\}}_{\mathsf{\text{diag}}}$$

(12)

with the relaxation parameter τ and the solution-depended point spread function (PSF) matrix operator

$${\mathbf{\Phi }}_{\mathbf{D}}={\mathbf{\Phi }}_{\mathbf{D}}\left(\widehat{\mathbf{b}}\right)=\left(\mathbf{\Psi }+N_{\Sigma }{\mathbf{D}}^{-1}\left(\widehat{\mathbf{b}}\right)\right)\bullet {\left(\mathbf{\Psi }+N_{\Sigma }{\mathbf{D}}^{-1}\left(\widehat{\mathbf{b}}\right)\right)}^{*}$$

(13)

Associating in (11) the iterations i+ 1→ t +Δt;i →t; τ→ Δt, with "evolution time", (Δt→ dt; t +Δt →t +dt) and considering the continuous 2-D rectangular scene frame R ∋ r= (x, y) with the corresponding initial MSF scene image q(r) = $\widehat{b}\left(\mathbf{r};0\right)$ and the "evolutionary"-enhanced SSP estimate $\widehat{b}\left(\mathbf{r};t\right)$, respectively, we proceed from (11) to the equivalent asymptotic dynamic scheme [2]

$$\frac{\partial \widehat{b}\left(\mathbf{r};t\right)}{\partial t}={\mathcal{P}}_{+}\left\{\mathcal{M}\left\{\left(q\left(\mathbf{r}\right)\right)\right\}-\mathcal{M}\left\{\underset{R}{\int }{\mathbf{\Phi }}_{\widehat{b}}\left(\mathbf{r},r^{\prime };t\right)\widehat{b}\left(r^{\prime };t\right)d^2{r}^{\prime }\right\}\right\},$$

(14)

where ${\Phi }_{\widehat{b}}\left(\mathbf{r},r^{\prime };t\right)$ represents the kernel PSF in evolution time t corresponding to the continuous-form dynamic generalization of the PSF matrix ${\mathbf{\Phi }}_{\mathbf{D}\left[i\right]}$ specified by (13), and $\mathcal{M}$ defines the metrics inducing operator. For the adopted $\mathcal{M}=m^{\left(0\right)}\mathcal{I}+m^{\left(1\right)}{\nabla }_{\mathbf{r}}^2$, the (14) is transformed into the VA dynamic process defined by the partial differential equation (PDE)

$$\begin{array}{c}\frac{\partial \widehat{b}(\mathbf{r};t)}{\partial t}={\mathcal{P}}_{+}\{c_0[q(\mathbf{r})-{\displaystyle \underset{R}{\int }\mathbf{\Phi }(\mathbf{r},{\mathbf{r}}^{\prime };t)\widehat{b}({\mathbf{r}}^{\prime };t)d{\mathbf{r}}^{\prime }}]\\ +c_1{\nabla }_{\mathbf{r}}^2\{q(\mathbf{r})\}-c_2{\nabla }_{\mathbf{r}}^2\{{\displaystyle \underset{R}{\int }\mathbf{\Phi }(\mathbf{r},{\mathbf{r}}^{\prime };t)\widehat{b}}({\mathbf{r}}^{\prime };t)d{\mathbf{r}}^{\prime }\}\}.\end{array}$$

(15)

For the purpose of generality, instead of relaxation parameter τ and balancing coefficients m⁽⁰⁾ and m⁽¹⁾ we incorporated into the PDE (15) three regularizing factors c₀, c₁, and c₂, respectively, to compete between noise smoothing and edge enhancement [2, 9]. These are viewed as additional VA-level user-controlled degrees of freedom.

4.5. Family of numerical DEDR-VA-related techniques for SSP reconstruction

The discrete-form approximation of the PDE (15) in "iterative time" {i = 0, 1, 2,...} yields the contractive mapping iterative numerical procedure [2]

$${\mathbf{b}}_{\left[i+1\right]}={\widehat{\mathbf{b}}}_{\left[i\right]}+{\mathcal{P}}_{+}\left\{c_0\left(\mathbf{q}-{\mathbf{\Phi }}_{\mathbf{D}\left[i\right]}{\widehat{\mathbf{b}}}_{\left[i\right]}\right)+c_1{\nabla }^2\left\{\mathbf{q}\right\}-c_2{\nabla }^2\left\{{\mathbf{\Phi }}_{\mathbf{D}\left[i\right]}{\widehat{\mathbf{b}}}_{\left[i\right]}\right\}\right\}$$

(16)

i = 0,1,2,... with the same MSF initialization (12). Different feasible assignments to the user-controlled degrees of freedom (i.e., balancing factors c₀, c₁, c₂) in (16) specify the family of corresponding DEDR-VA-related SSP reconstruction techniques that produce the relevant RS images. Extending the previous studies on the DEDR-VA topic [2, 9] herebeneath we exemplify the following ones.

• (i) The simplest case relates to the specifications: c ₀ = 0, c ₁ = 0, c ₂ = const = -c, c > 0, and Φ (r, r';t) = δ(r - r') with excluded projector ${\mathcal{P}}_{+}$. In this case, the PDE (15) reduces to the isotropic diffusion (so-called heat diffusion) equation $\partial \widehat{b}(\mathbf{r};t)/\partial t=c{\nabla }_{\mathbf{r}}^2\widehat{b}(\mathbf{r};t)$. We reject the isotropic diffusion because of its resolution deteriorating nature [1].

• (ii) The previous assignments but with the anisotropic conduction factor, -c ₂ = c(r; t) ≥ 0 specified as a monotonically decreasing function of the magnitude of the image gradient distribution [4], i.e., a function $c\left(\mathbf{r},\mid {\nabla }_{\mathbf{r}}\widehat{b}\left(\mathbf{r};t\right)\mid \right)$ ≥ 0, transforms the (15) into the anisotropic diffusion (AD) PDE, $\partial \widehat{b}(\mathbf{r};t)/\partial t=c(\mathbf{r};|{}_{\nabla }^{\mathbf{r}}\widehat{b}(\mathbf{r};t)|){\nabla }_{\mathbf{r}}^2\widehat{b}(\mathbf{r};t)$, which specifies the celebrated Perona-Malik AD method [4] that sharpens the edge map on the low-resolution MSF images.

• (iii) For the Lebesgue metrics specification c ₀ = 1 with c ₁ = c ₂ = 0, the PDE (15) involves only the first term at the right-hand side resulting in the locally selective robust adaptive spatial filtering (RASF) approach investigated in details in our previous studies [7, 9].

• (iv) The alternative assignments c ₀ = 0 with c ₁ = c ₂ = 1 combine the isotropic diffusion with the anisotropic gain controlled by the Laplacian edge map. This approach is addressed as a selective information fusion method [5] that manifests almost the same performances as the DEDR-related RASF method [7].

• (v) The aggregated approach that we address here as the unified DEDR-VA method involves all the three terms at the right-hand side of the PDE (15) with the equibalanced c ₀ = c ₁ = c ₂ = const (one for simplicity), hence, it combines the isotropic diffusion (specified by the second term at the right-hand side of (16)) with the composite anisotropic gain dependent both on the evolution of the synthesized SSP frame and its Laplacian edge map [2]. This produces a balanced compromise between the anisotropic reconstruction-fusion and locally selective image despeckling with adaptive anisotropic kernel windowing that preserves and even sharpen the image edge map [2].

All exemplified above techniques with different feasible specifications of the user-controllable degrees of freedom compose a family of the DEDR-VA-related iterative techniques for SSP reconstruction/enhancement. The general-form DEDR-VA framework is shown in Figure 1. It is noted that the progressive contractive mapping procedure (16) can be performed separately along the range (y) and azimuth (x) directions in a parallel fashion making an optimal use of the PSF sparseness properties of the real-world RS imaging systems. These features of the POCS-regularized DEDR-VA-related algorithms generalized by (16) result in the drastically decreased algorithmic computational complexity (e.g., up to ~10³ times for the typical large-scale 10³ × 10³ SAR pixel image formats [8]).

Figure 1

General framework of the unified POCS-regularized DEDR-VA method.

Next, several RS images formed by different sensor systems or applying different image formation techniques can be aggregated into an enhanced fused RS image employing the NN computational framework [10]. We are now ready to proceed with construction of such NN-adapted DEDR-VA-related techniques.

5. Radar/SAR image enhancement via sensor and method fusion

5.1. Fusion problem formulation

Consider the set of equations

$${\mathbf{q}}^{\left(p\right)}={\mathbf{\Phi }}^{\left(p\right)}b+{\mathbf{\nu }}^{\left(p\right)};p=\mathsf{\text{1}},\dots ,P,$$

(17)

which model the data {q^(p)} acquired by P RS imaging systems that employ the image formation methods from the DEDR-VA-related family specified in the previous section. In (17), b represents the original K-D image vector, {Φ^(p)} are the RS image formation operators referred to as the PSF operators of the corresponding DEDR-VA-related imaging systems (or methods) where we have omitted the sub index D for notational simplicity, and {ν^(p)} represent the system noise with further assumption that these are uncorrelated from system to system.

Define the discrepancies between the actually formed images {q^(p)} and the true original image b as the l₂ squired norms, J _p (b) = ||q^(p)- Φ^(p)b||²; p = 1,...,P. Let us next adopt the VA inspired proposition [10] that the smoothness properties of the desired image are controlled by the second-order Tikhonov stabilizer, J_P+1(b) = ${\mathbf{b}}^{\mathsf{\text{T}}}{\mathcal{P}}_1\mathbf{b}$, where ${\mathcal{P}}_1=\mathbf{M}=m^{\left(0\right)}\mathbf{I}+m^{\left(1\right)}{\nabla }^2$ is the VA-based metrics inducing (regularizing) operator specified previously by (9). We further define the image entropy as

$$H\left(\mathbf{b}\right)=-{\sum }_{k=1}^K{b}_k\mathsf{\text{ln}}{b}_{\mathsf{\text{k}}}.$$

(18)

Then, the contrivance for aggregating the imaging systems (methods), when solving the fusion problem, is the formation of the augmented objective (or augmented ME cost) function

$$E\left(\mathbf{b}\mid \lambda \right)=-H\left(\mathbf{b}\right)+\left(\mathsf{\text{1}}∕\mathsf{\text{2}}\right){\sum }_{p=1}^P{\lambda }_P{J}_P\left(\mathbf{b}\right)+\left(\mathsf{\text{1}}∕\mathsf{\text{2}}\right){\lambda }_{P+\mathsf{\text{1}}}{J}_{P+\mathsf{\text{1}}}\left(\mathbf{b}\right),$$

(19)

and seeking for a fused restored image $\widehat{\mathbf{b}}$ that minimizes the objective function (19), in which λ = (λ₁...λ _P , λ_P+1)^T represents the vector of weight parameters, commonly referred to as the fusion regularization parameters [10]. Hence, in the frame of the aggregate regularization approach to decentralized fusion [2, 6], the restored image is to be found as a solution of the convex optimization problem

$$\widehat{b}=\underset{\mathbf{b}}{\text{argmin}}E(\mathbf{b}|\mathbf{\lambda })$$

(20)

for the assigned values of the regularization parameters λ. A proper selection of λ is next associated with parametrical optimization [10] of such the aggregated fusion process.

5.2. NN-adapted fusion algorithm

The Hopfield-type dynamical NN, which we propose to employ to solve the fusion problem (20), is an expansion of the maximum entropy NN (MENN) proposed in our previous study [10]. We consider the multistate Hopfield-type (i.e., dynamic) NN [10, 11] with the K-D state vector x and K-D output vector z = sgn(Wx + θ), where W and θ are the matrix of synaptic weights and the vector of the corresponding bias inputs of the NN, respectively. The energy function of such the NN is expressed as [10]

$$\begin{array}{ll}\hfill E\left(\mathbf{x}\right)& =E\left(\mathbf{x};\mathbf{W},\theta \right)=-\left(\mathsf{\text{1}}∕\mathsf{\text{2}}\right){\mathbf{x}}^{\mathsf{\text{T}}}\mathbf{Wx}-{\theta }^{\mathsf{\text{T}}}\mathbf{x}\\ =-\left(\mathsf{\text{1}}∕\mathsf{\text{2}}\right){\sum }_{k=1}^K{\sum }_{i=1}^K{W}_{ki}{x}_k{x}_i-{\sum }_{k=1}^K{\theta }_k{x}_k.\end{array}$$

(21)

The proposed idea for solving the RS system/method fusion problem (20) using the dynamical NN is based on extension of the following cognitive processing proposition invoked from [10]. If the energy function of the NN represents the function of a mathematical minimization problem over a parameter space, then the state of the NN would represent the parameters and the stationary point of the network would represent a local minimum of the original minimization problem. Hence, utilizing the concept of the dynamical net, we may translate our image reconstruction/enhancement problem with RS system/method fusion to the correspondent problem of minimization of the energy function (21) of the related MENN. Therefore, we define the parameters of the MENN in such a fashion that to aggregate the corresponding parameters of the RS systems/methods to be fused, i.e.,

$$W_{ki}=–{\displaystyle {\sum }_{p=1}^P\text{[}}{\widehat{\lambda }}_p{\displaystyle {\sum }_{j=1}^K{\Phi }_{jk}^{\text{(}p\text{)}}{\Phi }_{ji}^{\text{(}p\text{)}}]–{\lambda }_P{}_{+\text{1}}{M}_{ki},}$$

(22)

$${\theta }_k=–\text{ln}{x}_k+{\displaystyle {\sum }_{p=1}^P\text{[}}{\widehat{\lambda }}_p{\displaystyle {\sum }_{j=1}^K{\Phi }_{jk}^{\text{(}p\text{)}}{q}_j^{\text{(}p\text{)}}}]$$

(23)

∀k, i = 1,...,K, where we redefined {x _k = b _k } and ignored the constant term E_const in E(x) that does not involve the state vector x. The regularization parameters {λ _p } in (22), (23) should be specified by an observer o pre-estimated invoking, for example, the VA inspired resolution-over-noise-suppression balancing method developed in [10, Section 3]. In the latter case, the result of the enhancement-fusion becomes a balanced tradeoff between the gained spatial resolution and noise suppression in the resulting fused enhanced image with the POCS-based regularizing stabilizer.

Next, we propose to find a minimum of the energy function (21) as follows. The states of the network should be updated as x '' = x ' + Δx using the properly designed update rule ℜ(z) for computing a change Δx of the state vector x, where the superscripts ' and '' correspond to the state values before and after network state updating (at each iteration), respectively. To simplify the design of such the state update rule, we assume that all x _k > > 1, which enables us to approximate the change of the energy function due to neuron k updating as [10]

$$\Delta E\approx -\left({\sum }_{i=1}^K{W}_{ki}{x^{\prime }}_i+{\theta }_k^{\prime }-\mathsf{\text{1}}\right)\Delta {\mathsf{\text{x}}}_{\mathsf{\text{k}}}-\left(\mathsf{\text{1}}∕\mathsf{\text{2}}\right){\mathsf{\text{W}}}_{\mathsf{\text{kk}}}{\left(\Delta {\mathsf{\text{x}}}_{\mathsf{\text{k}}}\right)}^2.$$

(24)

We now redefine the outputs of neurons as {z _k = sgn(${\sum }_{i=1}^K{W}_{ki}{x}_i^{\prime }+{\theta }_k^{\prime }$ - 1) ∀k = 1,...,K}. Using these definitions, and adopting the equibalanced fusion regularization weights, λ _p = 1 ∀p = 1,...,P, we next, design the desired state update rule ℜ(z) which guarantees nonpositive values of the energy changes ΔE at each updating step as follows,

$$\Delta x_k=\Re \left(z_k\right)=\left\{\begin{array}{ccc}\hfill 0\hfill & \hfill \text{if}\hfill & \hfill z_k=0\hfill \\ \hfill \Delta \hfill & \hfill \text{if}\hfill & \hfill z_k>0\hfill \\ \hfill -\Delta \hfill & \hfill \text{if}\hfill & \hfill z_k<0\hfill \end{array}\right.$$

(25)

where Δ is the pre-assigned step-size parameter. If no changes of ΔE(Δx) are examined while approaching to the stationary point of the network, then the step-size parameter Δ may be decreased, which enables us to monitor the updating process as it progresses setting a compromise between the desired accuracy of finding the NN's stationary point and computational complexity [10]. To satisfy the condition x _k > > 1 some constant x⁰ may be added to the gray level of every original image pixel and after restoration the same constant should be deducted from the gray level of every restored image pixel, hence, the selection of a particular value of x⁰ is not critical [10]. Consequently, the restored image $\widehat{\mathbf{b}}$ corresponds to the state vector $\widehat{\mathbf{x}}$ of the NN in its stationary point $\widehat{\mathbf{x}}$ as, $\widehat{\mathbf{b}}$= $\widehat{\mathbf{x}}$ - x⁰1, where 1 = (1 1...1)^T ∈ R^K is the K×1 vector composed with units. The computational structures of such the MENN and its single neuron are presented in Figures 2 and 3, respectively.

Figure 2

Computational structure of the multi-state MENN for sensor/image fusion.

Figure 3

Computational structure of a single neuron in the MENN.

6. Simulations

We simulated fractional side-looking imaging SAR operating in uncertain scenario [7]. We adopted a triangular shape of such imaging SAR range ambiguity function (AF) and a Gaussian shape of the corresponding azimuth AF [2, 12]. Simulation results are presented in Figures 4 and 5. The figure captions specify each particular simulated image formation/enhancement method (p = 1,...,P = 5). Aggregation of the locally selective robust spatial filtering (RSF) technique [5] with the DEDR-VA-optimal algorithm (16) was considered in the simulations of the NN-based fused enhancement mode. Next, Figure 6 reports the convergence rates for three most prominent VA-related enhanced RS imaging approaches: the APES [6], the DEDR, and the developed NN-adapted DEDR-VA-optimal method (16) implemented via the MENN technique (20-25).

Figure 4

Simulation results for the first uncertain fractional SAR imaging scenario for the large-scale (1024 × 1024 pixels) test scene and 5% random Gaussian perturbations in the SFO, < ||Δ||² >/||S||² = 5 × 10^-2. (a) degraded scene image formed applying the MSF method corrupted by composite noise (fractional SAR parameters: range PSF width (at 1/2 from the peak value) κ _r = 10 pixels, azimuth PSF width (at 1/2 from the peak value) κ _a = 30 pixels, composite SNR μ_SAR = 10 dB); (b) adaptively despeckled MSF image [8]; (c) image reconstructed applying the locally selective RSF method [5] after 30 performed iterations; (d) image reconstructed with the APES method [6] after 30 performed iterations; (e) image reconstructed applying the POCS-regularized RASF technique [7] after seven performed iterations and (f) image reconstructed applying the NN-fused RSF [5] and the DEDR-VA technique (16) after 7 performed iterations.

Figure 5

Simulation results for the second uncertain fractional SAR imaging scenario for the large-scale (1024 × 1024 pixels) test scene and 5% random Gaussian perturbations in the SFO, < ||Δ||² >/||S||² = 5 × 10^-2. (a) degraded scene image formed applying the MSF method corrupted by composite noise (fractional SAR parameters: range PSF width (at 1/2 from the peak value) κ _r = 7 pixels, azimuth PSF width (at 1/2 from the peak value) κ _a = 20 pixels, composite SNR μ_SAR = 15 dB); (b) adaptively despeckled MSF image [8]; (c) image enhanced using the AD technique [4] after 30 performed iterations; (d) image reconstructed applying the locally selective RSF method [5] after 30 performed iterations; (e) image reconstructed applying the POCS-regularized RASF technique [7] after seven performed iterations and (f) image reconstructed applying the NN-fused RSF [5] and the DEDR-VA technique (16) after 7 performed iterations.

Figure 6

Convergence rates evaluated via the IOSNR metric (26) versus the number of iterations evaluated for three most prominent high-resolution iterative enhanced RS imaging methods: DEDR-RASF method [7], APES--ML-optimal APES method [6], and the developed unified DEDR-VA-NN technique (16).

We employ two quality metrics for performance assessment of the reconstructive methods developed in this article. The traditional quantitative quality metric [7] for RS images is the so-called improvement in the output signal-to-noise ratio (IOSNR), which provides the metrics for performance gains attained with different employed estimators in dB scale

$$\mathsf{\text{IOSNR}}\left(\mathsf{\text{dB}}\right)=10\cdot {log}_{10}\left(\frac{{\sum }_{k=1}^K{\left(\mid {\widehat{b}}_k^{\left(\mathsf{\text{MSF}}\right)}-b_k\mid \right)}^2}{{\sum }_{k=1}^K{\left(\mid {\widehat{b}}_k^{\left(p\right)}-b_k\mid \right)}^2}\right)\mathsf{\text{,}}$$

(26)

where b _k represents the value of the k th element (pixel) of the original SSP, ${\widehat{b}}_k^{\left(\mathsf{\text{MSF}}\right)}$ represents the value of the k th element (pixel) of the rough SSP estimate formed applying the conventional low-resolution MSF technique (12), and ${\widehat{b}}_k^{\left(p\right)}$ represents the value of the k th element (pixel) of the enhanced SSP estimate formed applying the p th enhanced imaging method (p = 1,...,P), correspondingly. We consider and compare here five (i.e., P = 5) RS image enhancement/reconstruction methods, in which case p = 1 corresponds to the Lee's local statistics-based adaptive despeckling technique [2], p = 2 corresponds to the Perona-Malik AD method [5], p = 3 corresponds to the DEDR-related locally selective RASF technique [7], p = 4 corresponds to the APES method [6], and p = 5 corresponds to the NN-fused RSF and DEDR-VA methods, respectively.

The second employed quality metric is the l₁ total mean absolute error (MAE) metric [13]

$$\mathsf{\text{MAE}}=\frac{1}{K}{\sum }_{k=1}^K\mid {\widehat{b}}_k^{\left(p\right)}-b_k\mid ,p=1,...,P.$$

(27)

The quality metrics specified by (26) and (27) allow us to quantify the performance of the developed DEDR-VA-related high-resolution reconstructive methods (enumerated above by p = 1,...,P = 5) and, also, the NN fusion quality.

The quantitative measures of the image enhancement/reconstruction performance gains achieved with the particular employed DEDR-RSF method [7], the APES algorithm [6], and DEDR-VA-NN technique (16) for different SNRs evaluated with two different quality metrics (26), (27) are reported in Tables 1 and 2, respectively. The numerical simulations verify that the MENN implemented DEDR-VA method outperforms the most prominent existing competing high-resolution RS imaging techniques [1–7] (both without fusion and in the fused version) in the attainable resolution enhancement as well as in the convergence rates.

Table 1 IOSNR values provided with three methods, p = 3, 4, 5

Table 2 MAE values provided with three simulated methods, p = 3, 4, 5

7. Concluding remarks

The extended DEDR method combined with the dynamic VA regularization has been adapted to the NN computational framework for perceptually enhanced and considerably speeded up reconstruction of the RS imagery acquired with imaging array radar and/or fractional SAR imaging systems operating in an uncertain RS environment. Connections have been drawn between different types of enhanced RS imaging approaches, and it has been established that the convex optimization-based unified DEDR-VA-NN framework provides an indispensable toolbox for high-resolution RS imaging system design offering to observer a possibility to control the order, the type, and the amount of the employed two-level regularization (at the DEDR level and at the VA level, correspondingly). Algorithmically, this task is performed via construction of the proper POCS operators that unify the desirable image metrics properties in the convex image/solution sets with the employed radar/SAR motivated data processing considerations. The addressed family of the efficient contractive progressive mapping iterative DEDR-VA-related techniques has particularly been adapted for the NN computing mode with sensor systems/method fusion. The efficiency of the proposed fusion-based enhancement of the fractional SAR imagery has been verified for the two method fusion example in the reported simulation experiments. Our algorithmic developments and the simulations revealed that with the NN-adapted POCS-regularized DEDR-VA techniques, the overall RS imaging performances are improved if compared with those obtained using separately the most prominent in the literature despeckling, AD or locally selective RS image reconstruction methods that do not unify the DEDR, the VA and the NN-adapted method fusion considerations. Therefore, the developed unified DEDR-VA-NN framework puts in a single optimization frame, radar/SAR image formation, speckle reduction, and adaptive dynamic scene image enhancement/fusion performed in the rapidly convergent NN-adapted computational fashion.