and then I have to restart the game completely.

Any idea why this happens and a quicker fix than restarting the game?