Questions tagged [regularization]
Regularization, in mathematics and statistics and particularly in the fields of machine learning and inverse problems, refers to a process of introducing additional information in order to solve an ill-posed problem or to prevent overfitting. (Def: http://en.wikipedia.org/wiki/Regularization_(mathematics))
379 questions
0
votes
0
answers
45
views
Is Abel summability the same as contour integration?
I'm dealing with the following integral $$\int_{-\infty}^\infty \frac{ke^{ikx}}{\sqrt{k^2+m^2}}dk$$
where $m,x$ are some real positive fixed constants. I asked a question about the calculation of this ...
- 2,354
2
votes
1
answer
96
views
Capturing the Core: Approximating Integrals by Local Behavior
I want to tackle the following integral, which is inspired by Planck’s 1900 paper on radiation:
\begin{equation}
\label{1}
I = \int_{-\infty}^{\infty} f(x) \, g(x) \, dx
\end{equation}
where
$$
f(x) = ...
0
votes
0
answers
35
views
Identifiability & estimation: $d$ and $\underline c$ from $\lvert S \rvert = 6$
The scalar target $z$ is modeled as $$f(x,y) = \underline c^T \underline b, \qquad \underline b=\begin{bmatrix} 1 \\ x \\ ln(y+d) \\ x \cdot ln(y+d) \end{bmatrix},$$
with unknown parameter $d$ and ...
- 65
0
votes
0
answers
66
views
Divergent integral as a function of a parameter
I have the following integral:
\begin{equation}
\int_0^\infty -\frac{y^2\left(4-4 k^2+k^4+4 y^2\right) \beta \operatorname{Sech}\left[\frac{1}{4} \sqrt{4-4 k^2+k^4+4 y^2} \beta\right]^2}{12 k^2\left(-...
- 147
3
votes
0
answers
52
views
Condition for minimization of Tikhonov functional
Consider the following theorem where $X$ and $Y$ are Hilbert spaces:
I see why condition (16.2) is sufficient. But why is it necessary?
- 5,164
0
votes
0
answers
54
views
How does L1 regularization lead to sparse parameter matrix?
It is commonly said that L1 regularization on the parameters yields a sparse solution.
For which kind of problems is that true? Does that go only for linear problems (even though neuronal networks ...
- 164
5
votes
1
answer
324
views
Contour integral with real poles misconception
The following integral over the real line (principal value if you prefer)
$$I_1=\int_{-\infty}^{\infty}\mathrm d x\,\frac{e^{ix}}{x}=i\pi$$
can be calculated from any of these two contours about $0$:
...
- 469
1
vote
0
answers
113
views
Sparse least-squares regression and SVD
I am studying sparse partial least-squares (SPLS) regression, and I am interested in the mathematical foundations behind this method. The algorithm is proposed by Kim-Anh Lê Cao et al.$^\color{magenta}...
- 11
0
votes
0
answers
101
views
What is the subdifferential of complex $\ell_1$ norm?
I have a complex least-squares with $\ell_1$ regularization problem. Given the matrix $\mathbf{A}\in\mathbb{C}^{m\times n}$ and the vector $\mathbf{y}\in\mathbb{C}^{m}$,
$$ \arg\min_{\mathbf{x} \in \...
- 39
2
votes
1
answer
92
views
The proximal mapping of the projection onto the set of matrices of $\operatorname{rank} (X) \leq r$
Let $m \leq n$. Let
$$\mathsf{R}(r):= \left\{ X \in \mathbb{R}^{m \times n} : \operatorname{rank}(X) \leq r \right\}$$
be the set of (non-tall) $m \times n$ matrices of rank at most $r$. Given $\gamma ...
- 147
0
votes
0
answers
71
views
Integral using Hadamard regularization
I am not an expert on Hadamard regularization/Dimensional regularization, I am still learning. I am recovering some locally diverging integral, for a physical solution I need to use Hadamard part ...
1
vote
0
answers
72
views
Bregman distance (divergence) in a $2$ convex and $p$ smooth Banach space.
I have been reading the paper Convex regularization in statistical inverse problem and I can't understand something which the author mentions as "obvious".
Let $X$ be a Banach space over the ...
- 96
1
vote
0
answers
115
views
Sinkhorn Knopp algorithm - Bregman projection in update rule
I don't understand the updating rule for $u^{l+1}$ in the Sinkhorn algorithm. The below images contain all necessary definitions of the projection operators $A_1$ and $A_2$, which project a discrete ...
- 163
0
votes
0
answers
33
views
Title: Calculating the First Derivative with Respect to $\xi_j$ in a Mixture Model
Title: Calculating the First Derivative with Respect to $\xi_j$ in a Mixture Model
I'm currently reading the section on soft parameter sharing in Chapter 9 of Deep Learning by Christopher M. Bishop, ...
- 338
8
votes
2
answers
320
views
Value of $\lim_{\epsilon \rightarrow 0^+} \left(\frac{1}{(x- i \epsilon)^a} - \frac{1}{(x+ i \epsilon)^a}\right)$?
I am trying to obtain a relation which generalises
$$
\lim_{\epsilon \rightarrow 0^+}\left(\frac{1}{x-i \epsilon} - \frac{1}{x+i \epsilon}\right) = 2 \pi i \delta(x)
$$
for some generic power $a$ of ...
- 71