The answer to this question is found in Heb 11:17 where we read:

By faith Abraham, when he was tested, offered up Isaac, and the one who had received the promises was offering up his unique son

Now, as the OP correctly points out, Abraham had other son(s) such as Ishmael. (I will leave aside whether the sons born to him via Keturah were born before or after Isaac's sacrifice as no definite data is available.)

The text of Heb 11:17 says that Isaac was unique. That he was not Abraham's only son is obvious, however, Isaac was unique for several reasons:

• Isaac was the some of God's promise
• Isaac was a miraculous son of aged parents that were biologically incapable of producing progeny
• Isaac was, humanly speaking, became the progenitor of Christ.

All this made Isaac certainly unique among Abraham's children. This is further confirmed by Gen 25:5 which says:

Abraham left everything he owned to Isaac.

Thus, the importance and significance of Isaac, as Abraham's unique son validates the comments in Gen 22:

• V2 - “Take your son,” God said, “your only/unique son Isaac, whom you love ...
• V12 - ... for now I know that you fear God, since you have not withheld your only/unique son from me.
• V16 - you have not withheld your only/unique son