nftables deleting a rule without passing handle (similar to iptables delete)

It is a really hard task to remove rules from nftables in a reliable way and it looks like nobody is going to make this task easier. delete-by-name feature has been shelved. I've read mailing lists: some guys tried but failed.

Everyone is trying to get handles using grep, awk, etc but this way is not reliable. Something will change in the next 10 years, and all your software will be auto-converted to pumpkin.

Redhat is forcing everyone to move from iptables to nftables, so we have to deal with deletion somehow, no matter how complex this bicycle will be. Lets rock!

sudo nft "add rule ip nat POSTROUTING oifname \"wg0\" masquerade"

sudo nft --json --handle list ruleset |
  jq -r '
    .nftables |
    map(
      .rule |
      select(
        .family == "ip" and
        .table == "nat" and
        .chain == "POSTROUTING" and
        .expr != null and
        (
          .expr |
          map(
            select(
              .match.left.meta.key == "oifname" and
              .match.op == "==" and
              .match.right == "wg0"
            )
          ) | length > 0
        ) and
        (
          .expr |
          map(
            select(has("masquerade")
          ) | length > 0
        )
      )
    ) |
    .handle
  )[]' |
  xargs -I {} sudo nft "delete rule ip nat POSTROUTING handle {}"

PS I am reading Simple rule management, doing facepalm and crying at the same time. It was not possible for nftables developers to understand that release without delete-by-name feature is a pain for users. Low quality engineers from Redhat are trying to force nftables release in the following form.

One rationale I can think about is about having the same cost when deleting a rule as when adding a rule.

With iptables (legacy) this was also the same cost for the wrong reasons: adding or deleting a single rule forced to transmit the whole ruleset from kernel, edit it in userland (the iptables command) and put back again the whole ruleset in kernel. So adding one rule to a ruleset with 1000000 rules required processing 1000000+1 rules anyway. iptables-nft, while using the nftables kernel API as backend, has no choice but to be compatible so will have to retrieve more than needed when asked to delete a rule and do userland work.

With nftables, when adding a rule, only this single rule is sent to kernel which will then send back its handle (but using an index will probably require extra work). Adding 1 rule to a chain with 1000000 rules has no extra cost related to these 1000000 rules already present. To delete a rule by content would require to retrieve at least the whole chain and parse it until found, possibly having to parse 1000000 rules. While parsing is fine for display, this should be avoided whenever possible when performing operations that need to be fast. Having an unique handle is faster. I guess the decision was made to avoid this runtime cost at the cost of management methods.

Now depending on the problem there are multiple ways to not have to rely on deletion by content.

• Separate data from code (to be preferred)

  Using named sets, maps and vmaps allows to keep static rules that will use dynamic data: then there's no need to add or delete rules in typical runtime use, but instead only to add or delete elements. element lookup (during packet path traversal) and manipulation (during setup or dynamic use (eg: a tool blacklisting addresses in real-time such as fail2ban) ) are optimized for this use (with proper hashing etc. under the hood) and manipulation is not done by handle but by content.

  Likewise, iptables can leverage ipset for the same purpose, but nftables being more generic usually allows more interaction with these data structures than is possible with iptables and ipset.

• Store (during program or script execution) the handle immediately after it was created.

  nft --echo --handle ... will echo back then rule with its handle whenever added. With also --json for JSON output it can be easier to do something automated with it. The tool can then do what iptables was also doing in userland: find a matching rule to figure out the handle, but probably in an easier way since this would be a tailored use.

• Split rules into specific regular (user) chains

  Just as is possible with iptables, use dedicated chains whose role is to have data rather than code. Then to rebuild them, flush them first and populate them back from an external source of data. As nftables operations can be atomic (when using nft -f ...) there is no temporary misbehaving ruleset to be expected.

  Still it would be preferable to use a set/map/vmap instead: A packet will traverse linearly a chain with all its rules so traversing a chain having n rules attempting to match this packet has an average lookup time of O(n), while a set/map/vmap (called from a single rule) is hashed and gets a faster traversal: an average lookup time of O(1).