This effectively reverts commit c5d3df3 (#4160). It appears that rustup has some problems setting PATH properly for some users, so we _need_ this behaviour.