Matt.25:15, says the servants were given talents according to their abilities. Who determines the abilities of each or anyone?

The parable deals with "talents" as a unit of money not innate abilities. In Jesus' time a talent was about 100 denarii. "Talents" were not considered to be abilities until about around the 13th century. Thus the master in the parable declares:

28 Therefore take away the talent from him, and give it to the one who has the ten talents.’

No one would suggest that God takes away an ability from one person (who does not use it well) and give it to a more-talented person. In that sense the OP's question does not fit with the meaning of the parable. The master gave his slaves money to invest for him. He did not give them abilities to develop.

But if we presume the the OP's premise is correct - that the parable deals with abilities rather than investment monies - then the answer would be yes. God gives people certain talents. If they don't use the talent well, he takes that talent away from them and gives it to someone with more abilities. (I do not suggest that we think of God this way ;-)

Addendum: in terms of the actual meaning of the parable, I suggest that it is about righteousness, not abilities. Those who fail to act righteously will be harshly judged, but those who do good (or are judged to be righteous by faith) will be richly rewarded:

29 “For to everyone who has, more shall be given, and he will have an abundance; but from the one who does not have, even what he does have shall be taken away. 30 Throw out the worthless slave into the outer darkness; in that place there will be weeping and gnashing of teeth.

Let's review the Parable of the Talent/Parable of the Bags of Gold

Matthew 25:14-30 NIV

14 “Again, it will be like a man going on a journey, who called his servants and entrusted his wealth to them. 15 To one he gave five bags of gold, to another two bags, and to another one bag,[a] each according to his ability. Then he went on his journey. 16 The man who had received five bags of gold went at once and put his money to work and gained five bags more. 17 So also, the one with two bags of gold gained two more. 18 But the man who had received one bag went off, dug a hole in the ground and hid his master’s money.

19 “After a long time the master of those servants returned and settled accounts with them. 20 The man who had received five bags of gold brought the other five. ‘Master,’ he said, ‘you entrusted me with five bags of gold. See, I have gained five more.’

21 “His master replied, ‘Well done, good and faithful servant! You have been faithful with a few things; I will put you in charge of many things. Come and share your master’s happiness!’

22 “The man with two bags of gold also came. ‘Master,’ he said, ‘you entrusted me with two bags of gold; see, I have gained two more.’

23 “His master replied, ‘Well done, good and faithful servant! You have been faithful with a few things; I will put you in charge of many things. Come and share your master’s happiness!’

24 “Then the man who had received one bag of gold came. ‘Master,’ he said, ‘I knew that you are a hard man, harvesting where you have not sown and gathering where you have not scattered seed. 25 So I was afraid and went out and hid your gold in the ground. See, here is what belongs to you.’

26 “His master replied, ‘You wicked, lazy servant! So you knew that I harvest where I have not sown and gather where I have not scattered seed? 27 Well then, you should have put my money on deposit with the bankers, so that when I returned I would have received it back with interest.

28 “‘So take the bag of gold from him and give it to the one who has ten bags. 29 For whoever has will be given more, and they will have an abundance. Whoever does not have, even what they have will be taken from them. 30 And throw that worthless servant outside, into the darkness, where there will be weeping and gnashing of teeth.’

In this parable, the master represents the Lord, while the servants symbolize believers who follow Jesus. The talents or bags of gold represent the gifts, abilities and opportunities that God entrusts to His followers. Believers are called to use these blessings in service to God.

Not everyone is gifted equally. Those with greater abilities are expected to produce greater results. However, God does not measure success by quantity alone, rather, He values faithfulness and effort. He praises those who use their gifts according to their capability, but punishes those who squander His blessings and fail to act.

It is important not to misunderstand God's role in human inequality. While humanity differs in talents and opportunities, God watches over each person with fairness and justice. He expects more from those who are more capable and requests lighter demand on those who are vulnerable.

This principle of divine accountability is powerfully conveyed in Luke 12:48:

From everyone who has been given much, much will be demanded; and from the one who has been entrusted with much, much more will be asked.

The verse highlights God's expectation that individuals use their gifts, abilities and opportunities responsibly and faithfully. Those blessed with greater abilities are called to higher levels of stewardship and service.

The parables teaches that the judgment or justice of God is observed the same way we observe it in our daily business management or capitalism. Just as a business master knows how to judge the ability of his slaves, God being omniscient knows everyone's heart and soul. He shall judge each according to his responsibility or the talents given to him to produce profit for him. God invests in men, and judges them according to their profitability or fruits.

Etymology of the word talent: The word comes from the Greek τάλαντον (talanton), meaning "balance, pair of scales," and by extension "a weight" or "a sum of money." In classical times, a talent was a large unit of weight (and thus, by extension, a monetary value), used in Greece, Rome, and the ancient Near East. Medieval and Figurative Senses: In Medieval Latin and Old French, talent also acquired the figurative meaning of "inclination, will, desire," influenced by the biblical Parable of the Talents (Matthew 25:14–30), where "talent" is used as a metaphor for gifts or resources entrusted to a person. After 1600, it took the modern connotation of inborn ability or aptitude.

According to Liddell & Scott, as a monetary sum it was considered to consist of 6,000 drachmae, or, in Attica, 57.75 lbs. of silver. Also borrowed in other Germanic languages and Celtic. It is attested in Old English (talente) in the sense of "ancient unit of weight or monetary sum." The Medieval Latin and common Romanic "will, inclination, desire" sense developed from figurative use of the word in the sense of "money, wealth, riches."

The meaning "gift committed to one for use and improvement" developed by mid-15c., probably mostly from the parable of the talents in Matthew xxv.14-30. The notion is of something God has granted to one and for which one will render account at the Last Judgment. It may be also in part from or encouraged by the figurative sense of "wealth, treasures, riches."

The general sense of "special natural ability or aptitude" is by c. 1600. The meaning "persons of ability collectively" is attested from 1856. Talent scout is attested by 1936; talent agency is by 1956.

In Middle English, to have talent was "have resolve, be resolved, have a will or inclination;" talent of being was "survival instinct." To drink (one's) talent was to drink one's fill.