If the "burst copper" only leaks when the spigot is turned on, (which is how I read the wording of your question, and there's no obvious inside valve showing) you already have a frost-proof sillcock (the outside of which is the copper tube.)
That was evidently left full of water over the winter, rendering it "not frost-proof" as it was not drained in freezing weather (typically by leaving a hose on it) and it just needs to be replaced. The spigot and handle are outside the house, the valve stem is long, and the valve itself is inside the house.
You can make any other plumbing changes to want to make at the same time, of course - i.e. replacing the whole business with PEX up to the back of the sillcock. It makes no particular difference to the repair.
To avoid repeating this repair, always remove the hose from the spigot as soon as frost is in the forecast. If using the hose on a mild day in the fall/winter/spring, remove it as soon as you are done so the sillcock can drain. Only leave the hose connected when freezing is not in the forecast.