You run bind -x '"\C-o":vifm' in Bash. Then you hit Ctrl+o and Bash runs vifm; the shell is a parent, vifm is a child. Then you invoke :shell and vifm spawns its own child: a new Bash. Finally you hit Ctrl+o again.
The child of vifm (the shell which received your futile Ctrl+o) is not the parent of vifm (the shell which has been configured with bind …).
If you made it so each new interactive Bash runs bind -x '"\C-o":vifm' for itself automatically then Ctrl+o would work in the child shell. But it would run yet anothervifm in which you might be tempted to invoke :shell again an spawn yet another shell.
Stacking shells and vifms is not elegant.
From a child shell you can go back to the parent (i.e. old) vifm simply by exiting the shell (exitEnter or Ctrl+d when command line is empty).