I updated the DNS settings at my DNS provider for my domain. While all public DNS servers including the DNS resolver of my router have picked up the new settings after some hours, systemd-resolved still shows the previous (outdated) records. I already tried resolvectrl flush-cashes but it did not help.
How do I debug systemd-resolved where the outdated records come from?
systemd-resolved Configuration
myuser@desktop-pc ~ $ resolvectl status
Global
Protocols: +LLMNR -mDNS -DNSOverTLS DNSSEC=allow-downgrade/supported
resolv.conf mode: stub
Fallback DNS Servers: 1.1.1.1#cloudflare-dns.com 8.8.8.8#dns.google 1.0.0.1#cloudflare-dns.com 8.8.4.4#dns.google 2606:4700:4700::1111#cloudflare-dns.com 2001:4860:4860::8888#dns.google 2606:4700:4700::1001#cloudflare-dns.com
2001:4860:4860::8844#dns.google
Link 2 (enp6s0)
Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=allow-downgrade/supported
Current DNS Server: 192.168.178.1
DNS Servers: 192.168.178.1
DNS Domain: fritz.box
Link 3 (sit0)
Current Scopes: none
Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=allow-downgrade/supported
Querying "upstream" DNS servers returns correct response
myuser@desktop-pc ~ $ dig +nocmd +nocomments +nostats +noquestion @192.168.178.1 my-domain.tld. DNSKEY my-domain.tld. DS
my-domain.tld. 3600 IN DNSKEY 257 3 13 lwrfAkszf5Ntm0HOvMcU5Hy9mRdIcdJCePC5yiEdFzDvYP/d3/A1JfoT di4xDocD1rK7hzC3RLyC/u87Y6lRkQ==
my-domain.tld. 85456 IN DS 48469 13 2 B2744CEE8C59AE34191B6BED6C1710364C4857F59727FC155F53A575 EADAF835
myuser@desktop-pc ~ $ dig +nocmd +nocomments +nostats +noquestion @1.1.1.1 my-domain.tld. DNSKEY my-domain.tld. DS
my-domain.tld. 3600 IN DNSKEY 257 3 13 lwrfAkszf5Ntm0HOvMcU5Hy9mRdIcdJCePC5yiEdFzDvYP/d3/A1JfoT di4xDocD1rK7hzC3RLyC/u87Y6lRkQ==
my-domain.tld. 86400 IN DS 48469 13 2 B2744CEE8C59AE34191B6BED6C1710364C4857F59727FC155F53A575 EADAF835
myuser@desktop-pc ~ $ dig +nocmd +nocomments +nostats +noquestion @1.0.0.1 my-domain.tld. DNSKEY my-domain.tld. DS
my-domain.tld. 3600 IN DNSKEY 257 3 13 lwrfAkszf5Ntm0HOvMcU5Hy9mRdIcdJCePC5yiEdFzDvYP/d3/A1JfoT di4xDocD1rK7hzC3RLyC/u87Y6lRkQ==
my-domain.tld. 86400 IN DS 48469 13 2 B2744CEE8C59AE34191B6BED6C1710364C4857F59727FC155F53A575 EADAF835
All DNS servers (incl. my local Internet router 192.168.178.1) return the correct (new) DS record.
Local stub resolver returns outdated response
myuser@desktop-pc ~ $ dig +nocmd +nocomments +nostats +noquestion my-domain.tld. DNSKEY my-domain.tld. DS
my-domain.tld. 1627 IN DNSKEY 257 3 13 lwrfAkszf5Ntm0HOvMcU5Hy9mRdIcdJCePC5yiEdFzDvYP/d3/A1JfoT di4xDocD1rK7hzC3RLyC/u87Y6lRkQ==
my-domain.tld. 6644 IN DS 6769 8 2 61D117BD41CC280C4907804324B3F2B6E6810D881F1E1D1F4C0E8423 39976A70
The local stub resolver (127.0.0.53), which is built into systemd-resolved, returns the outdated DS record.
digfor the DS record, I see a DNS query for the DS record on the wire, my Internet home router replies with the correct response, and systemd-resolve shows me the old, outdated result again.systemd, making Windows 3.1 from 1992 looking more reliable onesystemdfailure after another.... Why aninitsystem had to take over and screw up DNS resolution is a mystery for the ages.systemd-resolvefinally returns the correct (new) DNS record. Unfortunately, I don't know what changed, probably some internal timer which finally elapsed.