This problem is most easily done in polar coordinates. In polar coordinates, the Newtonian gravity is given by $$\mathbf{g} = -\frac{GM}{r^2} \mathbf{\hat{r}}$$

Applying Newton's Second Law in polar coordinates gives two differential equations: one radial and one angular: $$\ddot{r} - r \dot{\theta}^2 = -\frac{GM}{r^2} \\ r \ddot{\theta} + 2 \dot r \dot \theta = 0$$

By substituting $u = 1/r$ and re-expressing the above equations in terms of $u$, it can be shown that $$\frac{\text{d}^2u}{\text{d} \theta^2} + u= \frac{GM}{h^2}$$ where $h$ is the angular momentum per unit mass of the orbiting body. This can then be solved to obtain$$r=\frac{h^2}{GM (1+e \cos \theta)}$$ where $e$ is the eccentricity. 

This is the equation we were looking for: a conic section. $e =0$ gives a circle, $0 \lt e \lt 1$ gives an ellipse, $e=1$ gives a parabola, and $e \gt 1$ gives a hyperbola.