Unfortunately, I don't think you'll get notifications unless Chrome is running. When you exit out of Chrome, practically all of the processes used to support it are stopped. That's a good thing; otherwise everyone would have hidden processes running in the background. It's the same for Outlook and other mail programs.

I have Gmail on my phone to give me those notifications, because on phones, apps never truly shut off all the way. Most apps still have processes that check for updates, including incoming messages. Windows 10 does have the mail app, which through some trickery can probably implement Gmail notifications, but it's a background app almost as heavy as Chrome; you may as well just keep Chrome open all the time.