Cannot change time speed when rotating camera (holding MMB)

Issue description

Whenever I rotate camera around using default key (middle mouse button), I cannot use my numbers under the esc key (numbers 1,2 and 3) to change game speed. Neither spacebar to pause works.
Whenever I release MMB, it then is possible to change the game speed.

Otherwise, thank you lads for an amazing update. Can't wait to get lost in the time while playing it :)

Reproduction steps

1. In a game, hold MMB
2. Press a key to fast forwards time (1,2,3 or spacebar)
3. See that the game speed is unaffected.

Game version and build number

v0.7.1a (b411)

Operating system

Windows