Good, simple reference for Riesz-Fischer Theorem.

Almost everyone uses the same idea, which sounds like your definition of rapidly-Cauchy sequence. Somehow you have to find a subsequence of a Cauchy sequence $\{ f_n \}$ that converges; once you know that a subsequence converges, then the sequence itself converges. However, when authors try to break the proof into a bunch of small results, I think the simplicity of the proof is lost.

By choosing $\{ n_k \}$ so that $\|f_{l}-f_{m} \| < 1/2^k$ whenever $l, m \ge n_k$, you have a candidate sequence, written as a telescoping sum $$ f_{n_k}=f_{n_1}+(f_{n_2}-f_{n_1})+(f_{n_3}-f_{n_2})+\cdots+(f_{n_{k}}-f_{n_{k-1}}) \tag{$\dagger$} $$ Why is this a good candidate? Because, by the triangle inequality for the norm, $$ \left(\int (|f_{n_1}|+|f_{n_2}-f_{n_1}|+|f_{n_3}-f_{n_2}|+\cdots+|f_{n_k}-f_{n_{k-1}}|)^p d\mu\right)^{1/p} \\ \le \|f_{n_1}\|_p+\|f_{n_2}-f_{n_1}\|_p+\|f_{n_{2}}-f_{n_{3}}\|_p+\cdots+\|f_{n_k}-f_{n_{k-1}}\|_p \le M, $$ where $M=\|f_{n_1}\|_p+\sum_{k=1}^{\infty}\frac{1}{2^k}$. Because the above holds for all $k$, the Monotone Convergence Theorem guarantees that the following non-negative, monotone sequence of functions converges to a finite limit a.e.: $$ g=|f_{n_1}|+\sum_{k=1}^{\infty}|f_{n_{k+1}}-f_{n_{k}}|. $$ Comparing with $(\dagger)$, you obtain the pointwise a.e. convergence of $\{ f_{n_k}\}_{k=1}^{\infty}$; and the dominated convergence theorem guarantees that the pointwise a.e. finite limit function $f$ is in $L^p$. Finally, $$ \|f-f_{n_{k}}\|_p \le \sum_{l=k}^{\infty}\|f_{n_{l+1}}-f_{n_{l}}\|_p \rightarrow 0 \mbox{ as } k\rightarrow\infty. $$ I don't think a proof of completeness can be much cleaner than that.