Are members of the US Congress entitled to abstain or vote "present" (as opposed to yea or nay)?

Yes, they can. According to the Congressional Institute:

When a bill or amendment is up for a vote, a Representative may vote “aye”, “no”, or “present”, which is a refusal to take sides. A “present” vote does not count towards or against the passage of a bill, but it contributes towards the quorum, which is the minimum number of Members required in attendance for the body to conduct business legally.

united-kingdom

Members of Parliament can vote, or not vote, however they like. If they don't toe the party line, then under the whip system, they might face the sanction of being suspended or expelled from their party. However, that doesn't make an abstention unlawful, and it doesn't prevent the MP from continuing to sit in Parliament as an independent.

There is no option in the UK to vote "present". You either vote "Aye", "No", neither, or both (the latter two options both being equivalent to an abstention). There's a related concept called "pairing" where an aye and a no voter both agree to abstain so that they cancel each other out. This is normally used when one can't attend parliament (e.g. because of illness).

israel

The Israeli Parliament (the Knesset) also allows an explicit "abstain" vote. While there's no requirement of quorum, and certain laws are known to have passed with a handful of MKs present, the abstain vote is important for a different reason. The Israeli government is based on a parliamentary coalition, and coalition members may be required to follow the guidance of the coalition coordinator (usually under the directive of the Prime Minister) on how to vote on laws that are important to the government. Those who vote against the guidance (either for the law the government opposes or against the law the government supports) risk disciplinary actions from their respective fraction leaders.

Often times, coalition MKs who don't want to vote with the government prefer to just "miss" the vote, but that too can lead to certain disciplinary actions (this time from the Knesset administrative leadership) if too many votes are missed.

Abstaining would not violate the coalition guidance, doesn't count as a missed attendance, and allows MKs to express their disagreement while avoiding the immediate negative repercussions.

Are members of congress entitled to abstain or vote "present" (as opposed to yea or nay)?

Yes.

Is there a way for members of congress to be reprimanded for absenteeism or chronic abstaining from voting?

Each house can discipline its own members according to its own rules. (Representative Adam Clayton Powell, Jr. was excluded from his seat for (among other reasons) excessive absenteeism, although the Supreme Court later ruled this to be unconstitutional for other reasons.) Censuring someone for their voting choices seems like something members would tend to avoid for political reasons, at least.

A contempt of congress type of charge or is abstaining to keep quorum from being attained considered valid strategy?

The houses of Congress typically "reprimand," "censure," or "expel" their members. Criminal contempt is off the table, as it would require a court trial, and the constitution immunizes members of Congress as noted in littleadv's answer. A house could, I suppose, find one of its own members in contempt using its inherent contempt power, but given the ability to discipline members under the rules, and the fact that inherent contempt is almost never used, this seems unlikely.

On the relationship between "present" votes and quorum determinations, see for example Voting and Quorum Procedures in the Senate from the Congressional Research Service. It begins by discussing voice votes:

Until a point of no quorum has been raised, the Senate operates on the assumption that a quorum is present, and even if only a few Senators are present, a measure may be passed or a nomination agreed to.... Voice votes may be taken on the passage of a bill and if no question of a quorum is raised, that action is final, even though a majority of the Senators did not participate; the Senate operates on the absolute assumption that a quorum is always present until a point of no quorum is made.

Then, on roll call votes:

Every Senator is expected to vote on each roll call unless, under paragraph 3 of Rule XII, "he believes that his voting on such a matter would be a conflict of interest." If a Senator declines to vote for any other reason, paragraph 2 of the same rule prescribes a procedure for the Senator to explain his or her reason and for the Senate to decide if that reason is sufficient; this procedure is very rarely invoked.

For a roll call vote to be constitutionally valid, a majority of Senators must vote, answer "Present," or announce that they have live pairs and refrain from voting for that reason. If less than a majority is present on a roll call vote, a quorum call usually ensues. Once a quorum is established, a new vote takes place on the question before the Senate (in other words, the question is put de novo).

On "pairs":

Pairing is a voluntary arrangement between individual Senators to offset their votes on a roll call vote so that, if one of the Senators needs to be absent, it is offset by another member who does not cast his vote. "When less than a quorum votes, but the addition of names of Senators present and paired and announcing votes made a quorum, the vote is valid." Riddick's Senate Procedure, p. 1431.

"A yea and nay vote by less than quorum is not valid unless a sufficient number is present and paired to make a quorum." Riddick's Senate Procedure, p. 1075. "Where less than a quorum votes, as disclosed by a yea and nay vote, the next business is for the presiding officer to direct a call of the roll to develop a quorum." Riddick's Senate Procedure, p. 1064. Alternatively, the Senate may vote to adjourn.