9
$\begingroup$

Is it proper to view conditional probabilities, such as the forms:

P(a|c)

P(a|c,d)

P(a, b|c, d)

...and so forth, as being tensors?

If so, does anyone know of a decent introductory text (online tutorial, workshop paper, book, etc) which develops tensors in that sense for computer scientists/machine learning practitioners?

I have found a number of papers, but those written at an introductory level are written for physicists, and those written for computer scientists are rather advanced.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ sorry but what is Tensors? $\endgroup$ Commented May 17, 2013 at 23:31
  • 1
    $\begingroup$ Tensors are the multidimensional analogs of matrices. Matrices are two-dimensional tensors. $\endgroup$ Commented May 18, 2013 at 5:35
  • $\begingroup$ I would have said they are the multidimensional analogs of vectors, myself. But, in that light, they are geometric objects with special rules and operators that I'm not deeply familiar with. So the question is, can conditional probabilities be viewed as tensors in that sense, and if so how are the typical operators interpreted? $\endgroup$ Commented May 18, 2013 at 15:20
  • 1
    $\begingroup$ If you view $P(A=a) = \sum_b P(A=a|B=b) P(B=b)$ as a matrix times a vector, then you can find similar interpretations of other similar equations. Is that what you're after? $\endgroup$ Commented May 21, 2013 at 23:46
  • $\begingroup$ Yes, basically. With the host of questions that follow: Are those proper tensor operations? Is there an intuitive interpretation of covariance and contravariance in this framework? What about contraction operations? Raised and lowered indices, and so-called "index juggling"? $\endgroup$ Commented May 22, 2013 at 17:35

1 Answer 1

Reset to default
5
$\begingroup$

This is definitely possible, although the tensor has of course certain additional structure (constraints).

If you consider the following conditional defined for a categorical response $Y$ on categorical predictors $X_i$:

$P(Y|X_1,\ldots,X_n)$

this correspond to a conditional probability tensor of size $d_0 \cdot d_1 \cdot \ldots \cdot d_n$. Here $Y$ (as a categorical response) takes $d_0$ levels.

Foremost the elements of the tensor have each to be nonnegative.

You can find papers on this, e.g. "Bayesian Conditional Tensor Factorizations for High-Dimensional Classification" by Yang and Dunson (pdf, arxiv).

Of course, if the random variables concerned are not categorical variables, but either continuous random variables or taking an infinite number of values (or both), the mapping to finite tensors won't be so trivial. You'll need something like an infinite tensor (as an extension of an infinite matrix).

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.