Since the replies are treated as answers by the backend (they count towards tag scorescount towards tag scores, they check for duplicates which blocks postingblocks posting, and you can add commentsadd comments on them and savesave them via non-obvious ways), please use the same visibility rules for deleted replies as used for deleted answers on "conventional" Q&A. In particular, a reply author should be able to see his/her own deleted replies and there should be a way to undelete the reply if the reply was self-deleted.
This can prevent dramas as in the comment section of this question.
In the long term, it may be necessary to think about what features on answers actually make sense for replies and block those that don't make sense for replies in a consistent way. In my opinion, the replies getting considered towards tag badges (and likely other answer-related badges as well) clearly makes no sense. Proper comments on replies make sense and are what I've been advocating for, and saving makes sense but currently there are no UIs to achieve that.