I'm not sure that this question can be totally answered, because as you point out, the line between hardware and software can be blurred in some areas, such as firmware or embedded systems. However, I would argue that something falls under hardware security if any of the following three conditions applies:

• A flaw can be mitigated through physical security measures

• A flaw exists in the actual hardware design, eg. side channel attacks against CPU cache in cloud environments are enabled by a shared CPU cache.

• A flaw cannot be mitigated through firmware/software upgrade.