Thats how Mk2 Escort ventilation is designed...
You get some "natural" air flow coming through when moving.
All Mk2 Escorts do this... (I never knew this was an unusual setup for years, as my first 3 cars were Escorts... It is most un Japanese like.. and I do miss being able to get fresh air on Jap cars and even Falcons/Commodores.)