Questions tagged [bayesian-optimization]
Bayesian optimization is a family of global optimization methods which use information about previously-computed values of the function to make inference about which function values are plausibly optima. Its applications include computer experiments and hyper-parameter optimization in some machine learning models.
198 questions
0
votes
0
answers
57
views
An analytical approach to find $X_{5}= \left [ 0.4, 0.2, 0.4, 0 \right ]$ and $X_{6}= \left [ 0, 0.4, 0.2, 0.4 \right ]$ maximizing mutual information
Simpler case
In a binomial channel, inputs $X= \left [ x_{1}, x_{2} \right ]$ represent probabilities of $\left [ {\rm failure}, {\rm success} \right ]$ in $n$ trials, with output $Y\in\left\{ 0, 1, \...
- 51
0
votes
0
answers
22
views
Is there a better approach than Bayesian optimization to solve this problem? [duplicate]
I am using matlab to evaluate a time-consuming expression. Think a few minutes for each expression evaluation. The expression is a function of about 6 variables and I am seeking to extremize the ...
1
vote
0
answers
67
views
best approaches for multiple root finding when functions are not differentiable
I have a problem similar to one I posted about recently but sufficiently different to warrant its own discussion I think.
I have k functions, each of the same k-dimensional vector x, and I want to ...
- 2,542
1
vote
0
answers
78
views
Suggestions constrained optimization with noisy observations
For $N$ correlated Ornstein-Uhlenbeck processes, I want to find $N$ absorption boundaries, $\mathbf{A}\in\mathbb{R}^{N}$, such that expected value of the summed $N$ processes is maximized, while the ...
- 11
1
vote
1
answer
131
views
What Constrained Optimization method to use when my objective isn't strictly differentiable
I'm trying to find the vector of parameters x which gets me the optimal reward, subject to a couple of constraints like $f(x)=k$ and $g(x) \geq C $.
I have lower and upper bounds for each component of ...
- 2,542
0
votes
0
answers
52
views
Questions about calculating uncertainty and correlation matrix of model parameters from optimization
I am running a nonlinear earth system model to optimize 42 parameters p with 7 different kinds of observations $O_j$ where ...
- 213
1
vote
0
answers
44
views
Using limited labeling effort to estimate the proportion of positives
I have an (uncalibrated) binary image classifier. I want to use this classifier to estimate the proportion of positives $p_i$ in a dataset $D_i$. I have multiple datasets, each of which is drawn from ...
- 271
3
votes
1
answer
140
views
Standardizing data in Bayesian optimization
I am implementing a very basic Bayesian optimization algorithm in Matlab. It is generally recommended to standardize both the inputs (sampling points) and the outputs (black-box objective function ...
- 33
1
vote
0
answers
66
views
Best way to tackling SVM fine-tuning
I'm encountering a multiclass classification problem where I'm trying to predict 4 categories using SVM. I'm trying to fine-tuning its hyperparameter using Bayesian Optimization to speed up the ...
- 33
2
votes
1
answer
121
views
Posterior estimation using VAE
Using normalizing flows, we can model model's posteriors $p(\theta|D)$, by feeding Gaussian noise $z$ to the NF (parametrized with $\phi$), using the output of the NF $\theta$ as model parameters, and ...
- 1,589
1
vote
1
answer
127
views
How to determine the optimal exploitation-exploration trade off for a fixed number of objective function evaluations
In Bayesian optimization, we guess the next sampling point by finding $x = \textrm{argmax}_x \alpha(x)$, where $\alpha(x)$ is the acquisition function. For simplicity, let us consider the upper ...
- 11
0
votes
0
answers
72
views
Learning a probability distribution from samples drawn from unknown function
I am wanting to learn some probability distribution $p$ from data (using e.g., Kernel Density Estimation, a Normalizing Flow, whatever your favourite machine learning model is).
If I had a dataset $D =...
- 91
1
vote
0
answers
68
views
Understanding Bayesian Optimal Experiment Design
I read this tutorial on Bayesian Experimentation Design (https://pyro.ai/examples/working_memory.html) and I'm trying to wrap my head around it.
Suppose you have data (X,y).
You're thinking about ...
0
votes
1
answer
97
views
Maximum likelihood estimation and bayesian inference of variance given multiple datasets
I'm currently working on a problem were I have multiple normal distributed data sets $X_1, \dotsc,X_n$ with each data set having it's own mean $\bar x_i $ but all have the same variance $\sigma$. The ...
- 11
1
vote
1
answer
154
views
What exactly are we training across different iterations in the Gaussian Process Regression example in GPyTorch?
I am following this tutorial to implement a GP Regression using gPyTorch.
Based on my understanding of GP Regression, given the training data we can compute the posterior mean and covariance using the ...
- 169