so we can conclude that the man has angular acceleration without any external torque, which is an apparent contradiction of the terms, so how do we reconcile the case with the concept?
We reconcile it with the law of conservation of angular momentum.
The angular velocity of the skater increases when drawing in the arms in order to conserve angular momentum. The angular momentum of the skater will not change unless an external torque is applied to the object. So converse to your thinking, the change in angular velocity is due to no external torque being applied to the skater in order to conserve angular momentum.
Conservation of Energy:
The increase in angular velocity can also be explained by conservation of rotational kinetic energy. Ignoring friction there is no external force that can cause a change in the skaters rotational kinetic energy = 1/2 I$a^2$ where I is the rotational moment of internia of the skater and $a$ is the angular velocity of the skater. When the skater pulls his/her arms in it reduces the rotational moment of inertia I. In order to conserve kinetic energy the skater’s angular velocity $a$ must increase. Note however you can say that an internal force is what enabled the skater to pull in his/her arms.
Hope this helps.