I’ve had many frustrating moments trying to put GVimPortable on Windows systems. The biggest difference is that GVimPortable doesn’t follow the normal vimrc conventions for Windows.
Simply put your settings in GVimPortable/Data/settings/vimrc . Notice that there’s no underscore or dot as is the convention with the normal Windows binary. Also note that it ignores _vimrc files in your home directory, but it will happily display that file if you try and edit the settings via the GUI. All in all, this is extremely confusing and frustrating. The documentation does make this clear however, but I’d rather the normal conventions be followed.