The question is about a user service and the OP notes that "setting a nice value of -20 requires root permissions". The problem is a user service has no permission to set a negative niceness in the first place. Similarly it cannot freely switch to another user.
User=factorio in your service suggests it is a system service that has root permissions and thus is able to set niceness to -4 and to switch to another user. The OP has reasons not to create a system service. While I believe your service works and the niceness is -4, the answer does not address the OP's problem, it's not useful here.