Let's make the change of variables $x=e^{-\xi}$, $t=e^{-\tau}$ in the integral equation:
$$
\int_t^{1}f(x)f\!\left(\frac{t}{x}\right)dx=\int_0^{\tau}f(e^{-\xi})f(e^{-(\tau-\xi)})e^{-\xi}\,d\xi=e^{-\tau/2}. \tag{1}
$$
Defining $g(z):=f(e^{-z})$, we can rewrite $(1)$ as
$$
\int_0^{\tau}g(\tau-\xi)g(\xi)e^{-\xi}\,d\xi=e^{-\tau/2}. \tag{2}
$$
Taking the Laplace transform of both sides of $(2)$, we get
$$
G(s)G(s+1)=\frac{1}{s+\frac{1}{2}}, \tag{3}
$$
where $G(s):=\mathcal{L}\{g(\xi)\}(s)$.
The positive solution to the functional equation $(3)$ is (see the Appendix)
$$
G(s)=\frac{1}{\sqrt{2}}\frac{\Gamma\!\left(\frac{s}{2}+\frac{1}{4}\right)}{\Gamma\!\left(\frac{s}{2}+\frac{3}{4}\right)}. \tag{4}
$$
Finally, taking the inverse Laplace transform of $G(s)$, we obtain$^{(*)}$
$$
g(\xi)=\sqrt{\frac{2e^{-\xi}}{\pi\left(1-e^{-2\xi}\right)}}, \tag{5}
$$
hence
$$
f(x)=\sqrt{\frac{2x}{\pi\left(1-x^2\right)}}. \tag{6}
$$
Appendix: Solution of Eq. $(3)$
To solve Eq. $(3)$, let's rewrite $G(s)$ as
$$
G(s)=\frac{G(s)G(s+1)\cdots G(s+2n-1)}{G(s+1)G(s+2)\cdots G(s+2n)}\,G(s+2n)\qquad(n>1). \tag{A.1}
$$
Using Eq. $(3)$, we can rewrite $(\text{A.}1)$ as
\begin{align}
G(s)&=\frac{\left(s+\frac{3}{2}\right)\left(s+\frac{7}{2}\right)\cdots\left(s+2n-\frac{1}{2}\right)}
{\left(s+\frac{1}{2}\right)\left(s+\frac{5}{2}\right)\cdots\left(s+2n-\frac{3}{2}\right)}\,G(s+2n) \\
&=\frac{\left(\frac{s}{2}+\frac{3}{4}\right)\left(\frac{s}{2}+\frac{7}{4}\right)\cdots\left(\frac{s}{2}+n-\frac{1}{4}\right)}
{\left(\frac{s}{2}+\frac{1}{4}\right)\left(\frac{s}{2}+\frac{5}{4}\right)\cdots\left(\frac{s}{2}+n-\frac{3}{4}\right)}\,G(s+2n). \tag{A.2}
\end{align}
Now, using the identity
$x(x+1)\cdots(x+n-1)=\frac{\Gamma(x+n)}{\Gamma(x)}$,
we can rewrite $(\text{A}.2)$ as
$$
G(s)=\frac{\Gamma\!\left(\frac{s}{2}+\frac{1}{4}\right)
\Gamma\!\left(\frac{s}{2}+n+\frac{3}{4}\right)}
{\Gamma\!\left(\frac{s}{2}+\frac{3}{4}\right)
\Gamma\!\left(\frac{s}{2}+n+\frac{1}{4}\right)}\,G(s+2n). \tag{A.3}
$$
Since the RHS of $(\text{A}.3)$ does not depend on $n$, we can simplify it using the fact that Eq. $(3)$ implies $G(s+2n)\sim\frac{1}{\sqrt{s+2n}}$ as $n\to\infty$. Taking this limit with the help of WolframAlpha, we finally obtain
$$
G(s)=\lim_{n\to\infty}\frac{\Gamma\!\left(\frac{s}{2}+\frac{1}{4}\right)
\Gamma\!\left(\frac{s}{2}+n+\frac{3}{4}\right)}
{\Gamma\!\left(\frac{s}{2}+\frac{3}{4}\right)
\Gamma\!\left(\frac{s}{2}+n+\frac{1}{4}\right)}\,\frac{1}{\sqrt{n+2s}}
=\frac{1}{\sqrt{2}}\frac{\Gamma\!\left(\frac{s}{2}+\frac{1}{4}\right)}
{\Gamma\!\left(\frac{s}{2}+\frac{3}{4}\right)}.
\tag{A.4}
$$
Direct substitution shows that $(\text{A}.4)$ indeed satisfies Eq. $(3)$:
$$
G(s)G(s+1)=\frac{1}{2}\frac{\Gamma\!\left(\frac{s}{2}+\frac{1}{4}\right)\Gamma\!\left(\frac{s+1}{2}+\frac{1}{4}\right)}
{\Gamma\!\left(\frac{s}{2}+\frac{3}{4}\right)\Gamma\!\left(\frac{s+1}{2}+\frac{3}{4}\right)}
=\frac{\Gamma\!\left(\frac{s}{2}+\frac{1}{4}\right)}
{2\,\Gamma\!\left(\frac{s}{2}+\frac{5}{4}\right)}
=\frac{1}{s+\frac{1}{2}}. \tag{A.5}
$$
$^{(*)}$ A.P. Prudnikov, Yu.A. Brychkov, O.I. Marichev, Integrals and Series, Volume 5: Inverse Laplace Transforms, formula $3.1.1.1$.
