There are ways to exclude from
NOPASSWD, but security-wise this is pointless: I believe one can run a shell command from vim; and there are other commands that can give the user an elevated shell, thus allowing any command without password. Exceptions to ALL just cannot work.