Why it was not possible to use 256-QAM in the past

Probably because it was really hard? As you move to bigger and bigger constellations, you need several things to improve:

• You need more dynamic range on your ADCs as you need to discern between smaller changes in amplitude.

• You need more linear mixers as non-linearity can move points areound the constellation or in some cases make different points overlap. Saturating your mixer makes the outer edge points move in towards the center

• You need more stable clock sources (i.e. phase noise as peter mentioned). Phase noise causes the points to dance around and can cause the constellation to get rotated.

• You need more processing power to decode the symbols. This is partly because you're trying to discern between more levels (goes hand-in-hand with higher ADc precision). You're also going to need to perform more error checking and more compensation.

• You need (most of the time) some way to compensate for things like multipath interference, fading, motion etc. These effects are almost always compensated for in software (partly because they are highly dynamic and require constant adjustments) this places a big load on your processor.

All kinds of things that were either less of an issue or not even relevant before, and it only gets worse as the constellations get bigger and the symbol rates get higher. There wasn't any one single invention that helped. It was a mix of steadily improving technology, better error correcting and signal path compensation algorithms, but I think the fact that as time went on, the fact that people required more and more bandwidth played a big role. Never underestimate market pressure as a driving factor in technological development.

(note: symbol rate and data rate are not the same thing).

Oh, it was perfectly possible, just you as a consumer wouldn't have been able to afford it. Market pressures mostly dictate what reaches the consumer market, and the market of telecom utilities providing the services.

As the constellation gets bigger, it takes more capable semiconductors to process it. These more capable semiconductors initially cost way more than the consumer market can afford.

It's the same reason people didn't play 3D games on a CRAY1 at home, even though it was perfectly technologically feasible, and you could get stunning graphics out of it, with a custom framebuffer and a custom monitor capable of displaying it. Whereas today, with much better semiconductors, you get >100x the computing power of CRAY-1, and 1000x its RAM, in a mobile phone.

A compelling "need" spurs introduction of advanced technology to widespread adoption. Your examples had been in use for years before being applied in end-customer equipment.
In the case of your examples, the last-mile problem spurred the introduction of advanced carrier modulation to increase available data rates into the home. Certainly, twisted pair telephone lines were designed to carry voice, not data.

Previous examples of modulation advances to accommodate a need:

• FM stereo previously mono, multiplexing applied to allow left + right channels.

• Colour TV previously black-and-white. Colour subcarrier modulation added.

One big challenge in these adaptations is maintaining compatibility with the original intent - allowing old equipment to still accept a modified signal. Adding modulation dimensions is a common way of providing the desired extras, where signals are well above the noise.