The loading property is ignored because the attack from Horde Breaker is not part of the Attack action
Features that grant additional attacks that are part of the Attack action explicitly state this (the examples I'll use are Extra Attack, the Samurai Fighter's Rapid Strike feature, and the Gloom Stalker Ranger's Stalker's FuryFlurry feature; the emphasis is mine):
Beginning at 5th level, you can attack twice, instead of once, whenever you take the Attack action on your turn.
If you take the Attack action on your turn and have advantage on an attack roll against one of the targets, you can forgo the advantage for that roll to make an additional weapon attack against that target, as part of the same action. You can do so no more than once per turn.
At 11th level, you learn to attack with such unexpected speed that you can turn a miss into another strike. Once on each of your turns when you miss with a weapon attack, you can make another weapon attack as part of the same action.
Because the Horde Breaker features does not state that it is part of the Attack action, it is not part of the Attack action and so it is not subject to the Loading property.
The Loading property only applies when you use your action, bonus action, or reaction to fire the weapon; here you are not using any of those. Is it clear that you are not using your bonus action or reaction as these would be explicitly stated as being required, and by comparison to other similar features, it is also clear that this attack is not part of the Attack action.