May be a little late in replying
but I had a similar problem and think it happens when I zoom in using the mouse - I then can't use the pan view on the joystick.
The answer is to zoom back out again (move your head away from the windscreen (!) ) or use the 'Reset View' key/button if you have it mapped.
Seems to work for me.