A general answer can be found, e.g., at Switching tocdepth in the middle of a document. However, because of using KOMA-Script there are some special features, which should be recognized and also provide alternative suggestions.
LaTeX kernel command \contentsline is the command \addcontentsline would write to the ToC file(s) using \addtocontents. It has four arguments. The first one is the symbolic level, e.g., chapter, section, figure etc. But you just want to write to a contents file, so you want to use \addtocontents with two arguments. The first one is the extension of the file and the second one the content. Note, that you need to protect several fragile commands in the second argument. You also don't need \global. Because \setcounter already works globally.
And because \setcounter works globally, using \setcounter inside the toc file could result in an empty List of Figures (if \listoffigures is somewhere after \tableofcontents). Same for other lists of floats like List of Table, List of Listings etc. Instead you can use
\addtocontents{toc}{\protect\value{tocdepth}=0}
or
\addtocontents{toc}{\csname c@tocdepth\endcsname=0}
Otherwise you would need to add an additional
\setcounter{tocdepth}{\figuretocdepth}
before \listoffigures.
But because the toc depth of the levels are configurable with KOMA-Script IMHO you also should not assume that chapter is level 0 but use \chaptertocdepth. And the extension is also configurable, so I would suggest to use \ext@toc instead of toc. And if only the appendix sections should not appear in the ToC you could bound the change to \appendix.
\documentclass{scrbook}
\setcounter{tocdepth}{\sectiontocdepth}
\makeatletter
\AddToHook{scrbook/appendix}{%
\addtocontents{\ext@toc}{\protect\c@tocdepth=\protect\chaptertocdepth}%
}
\makeatother
\begin{document}
\tableofcontents
\chapter{Chapter}
\section{Section a}
\section{Section b}
\appendix
\chapter{Appendix Chapter}
\section{Appendix Section a}
\section{Appendix Section b}
\end{document}
This would also make it easy to make a different ToC for the appendix:
\documentclass{scrbook}
\setcounter{tocdepth}{\sectiontocdepth}
\makeatletter
\AddToHook{scrbook/appendix}{%
\addtocontents{\ext@toc}{\protect\c@tocdepth=\protect\chaptertocdepth}%
\listofappendices
\NewCommandCopy{\AddToCEntryDefault}{\addtocentrydefault}%
\renewcommand*{\addtocentrydefault}[3]{%
\AddToCEntryDefault{#1}{#2}{#3}%
\begingroup
\renewcommand*{\ext@toc}{atoc}%
\AddToCEntryDefault{#1}{#2}{#3}%
\endgroup
}%
}
\DeclareNewTOC[
type=appendix,
types=appendices,
setup=totoc,
listname=Appendix Contents,
]{atoc}
\makeatother
\begin{document}
\tableofcontents
\chapter{Chapter}
\section{Section a}
\section{Section b}
\appendix
\chapter{Appendix Chapter}
\section{Appendix Section a}
\section{Appendix Section b}
\end{document}
BTW: An alternative to the first suggestion would be to simply prevent writing of the section entries to the ToC file:
\documentclass{scrbook}
\setcounter{tocdepth}{\sectiontocdepth}
\AddToHook{scrbook/appendix}{%
\renewcommand*{\addsectiontocentry}[2]{}%
\renewcommand*{\addsubsectiontocentry}[2]{}% not needed with tocdepth=\sectiontocdepth
\renewcommand*{\addsubsubsectiontocentry}[2]{}% not needed with tocdepth=\sectiontocdepth
\renewcommand*{\addparagraphtocentry}[2]{}% not needed with tocdepth=\sectiontocdepth
\renewcommand*{\addsubparagraphtocentry}[2]{}% not needed with tocdepth=\sectiontocdepth
}
\begin{document}
\tableofcontents
\chapter{Chapter}
\section{Section a}
\section{Section b}
\appendix
\chapter{Appendix Chapter}
\section{Appendix Section a}
\section{Appendix Section b}
\end{document}
Note, that most of the \renewcommands are added for higher values of tocdepth. In your case you would need only the first one.
