I believe the option you're looking for is g:copilot_filetypes:
g:copilot_filetypes A dictionary mapping file types to their enabled
status. Most file types are enabled by default, so
generally this is used for opting out.
>
let g:copilot_filetypes = {
\ 'xml': v:false,
\ }
<
Disabling all file types can be done by setting the
special key "*". File types can then be turned back
on individually.
>
let g:copilot_filetypes = {
\ '*': v:false,
\ 'python': v:true,
\ }
So add to your vimrc:
let g:copilot_filetypes = { '*': v:false }
And then enable it when needed.