Trump would have a choice, either declassify all matters necessary to freely and fairly litigate his defamation case, including the classified documents, or have his lawsuit dismissed pursuant to the state secrets privilege (if the Epstein files are state secrets, which is doubtful).
While the state secrets privilege is described as an evidentiary privilege, in practice, it is basically a ground to dismiss any lawsuit in which any party would have to rely on state secrets to prove their case.
The state secrets privilege also prohibits covert operatives (or their heirs) from suing the federal government for compensation, and prohibits individuals on secret government missions from suing for personal injuries caused by a defective product used in the mission that causes them injuries or death during the mission.
At some point in a Trump lawsuit for defamation in which classified Epstein files would be an issue, Trump would be forced to elect between declassifying them (and they arguably aren't even state secrets since they don't pertain to national security or foreign affairs), or dismissing the lawsuit. Since Trump has the power to declassify these non-national security documents, he can waive the privilege and proceed, or he can dismiss the lawsuit.
(A small subset of classified documents, including those at issue in the Florida classified documents case against Trump before he was elected, like nuclear weapons secrets, can't be declassified, even by the President. They are automatically classified as a matter of statute.)
Trump might claim that the work product privilege (which is derivative of the attorney-client privilege) of the Department of Justice lawyers in the Epstein litigation, rather than the state secrets privilege, protects these files from disclosure. But again, since he has the power to waive the work product privilege as to these files, he would have to do so to proceed with his defamation case.
It also may be helpful to distinguish between confidentiality and privilege.
Something that is confidential can't be disclosed by a person with a duty to keep it confidential. For example, a lawyer or law clerk in the Department of Justice can't disclose any attorney work product without authorization to do so (even if that same document has already been printed on the front page of the New York Times). But if someone obtains the confidential documents anyway, and introduces them as evidence in court, the fact that the documents are confidential doesn't prohibit those documents from being used in court, unless the confidential documents are also privileged. The duty of lawyers to keep certain matters confidential arises from Rule of Professional Conduct 1.6 (the rules of professional conduct are established at the state/DC/territory level but the numbering system is uniform in every U.S. jurisdiction).
If evidence is privileged, then it can't be used in court, no matter how it was obtained, unless the holder of the privilege takes action to waive the privilege, or if the privilege is waived by someone authorized to waive the privilege on behalf of the holder of the privilege. In the case of the Epstein files, the holder of the privilege would be the United States government, and President Trump would be someone who is authorized to waive the privilege. In federal court, evidentiary privileges and the law of waiver arise from Federal Rules of Evidence 501 and 502 and the federal common law of evidentiary privileges. Some states codify their evidentiary privileges, while others follow the federal model.
One important part of the law of waiver of privilege is that someone who brings a lawsuit, as a matter of law, waives all of their evidentiary privileges related to the subject matter of their lawsuit, except their attorney client privilege between their attorney in that lawsuit, as opposed to prior separate lawsuits (see Federal Rule of Evidence 502 and the case law thereunder), when it is put in issue by the lawsuit filed. The plaintiff in a lawsuit must, as a matter of law, waive privilege with respect to any matter that is at issue in the lawsuit, and something is at issue in a lawsuit if either side asserts or alleges something in a claim or defense and the other side does not admit that assertion. It isn't always clear if evidence is "at issue" in a case until all complaints and answers and counterclaims and replies in the case are filed.
But, President Trump does not have the authority to declassify grand jury testimony without court approval, which has been sought and denied. So, President Trump could bring his lawsuit without disclosing grand jury testimony in the Epstein case.
(Trade secrets are not strictly speaking privileged, although they are entitled to certain protections in litigation, but the Epstein files aren't trade secrets in this context, because they don't belong to Trump and to the extent that they are trade secrets, the trade of human trafficking of children for prostitution is illegal and not entitled to legal trade secret protections.)