Or does it simply not matter, since if it's flipped upside-down it's still random?
The internet appears to believe there should only be two examples of each number 1-10 available, so it does matter.
According to the Countdown wiki, at least in the early days this has sometimes led to confusion:
Episode 79
Episode 79 was broadcast on 1 July 1983, and was the second semi-final
of Series 2.
... It appears that three 9's were selected for the Round 8 numbers
game; it is likely that one of the 9's was an upside down 6.
Episode 103
Episode 103 was broadcast on 26 October 1983, as part of Series 3.
...
It appears that three 6's were selected for the Round 4 numbers game;
it is likely that one of the 6's was an upside down 9.
I can't readily find later examples so I would speculate that a procedure was changed. That wiki is still active so its participants may know more.