Adding the route to the pfsense makes the traffic flow weird.
Traffic from the laptop bounces over the pfsense, back down to the nas and then to the container. Replies from the container however will go via the nas directly to the laptop on layer 2. No routing needed.
A consequence of that is that the firewall on the nas that only allow related packages get confused and invalidates the relation between the ssh packages and the first ssh connect packet.
My guess is that iptables conntrack on the nas is killing the session.
# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N DEFAULT_FORWARD
-N DOCKER
-N DOCKER-ISOLATION
-A FORWARD -j DEFAULT_FORWARD
-A DEFAULT_FORWARD -j DOCKER-ISOLATION
-A DEFAULT_FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DEFAULT_FORWARD -o docker0 -j DOCKER
-A DEFAULT_FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A DEFAULT_FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER-ISOLATION -j RETURN
Removing the route from the pfsense and configuring it directly on the laptop instead seem to work a lot better.
How to add route on macOS:
sudo route add 172.17.0.0/16 10.11.12.10