The big difference between software and hardware is that w a hardware wallet is it is chip protected. So while I can have infinite time with a encrypted wallet.dat, a hardware wallet will prevent unlimited attack. If that level of physical security is required, I would think a chip protected cheap laptop would be a good solution.
Same issue with a standard USB encryption. You can attack infinity if you your possession, though they also make chip protected USB's as well. My theory is if your keys are stored such that the USB is exposed to physical attack (lost or stolen) then a chip could help. If it is physically protected (in a safe or otherwise in your possession) a chip doesn't add a reasonable layer of protection, but uneeded complexity.