An $n \times n \times n$ cube is divided into $n^3$ unit cubes, of which $n^2$ are black and the remaining ones are white. A move consists of selecting two parallel $1 \times 1 \times n$ columns (i.e., blocks of $n$ unit cubes aligned in the same direction) and swapping their positions without rotating them. Is it always possible, by a sequence of such moves, to arrange the unit cubes so that all black unit cubes lie on a single face of the big cube?

(I do not know the source of the problem.)

The statement is true for $n=2$, but checking bigger values of $n$, is hard. I tries multiple colorings to find something invariant, which would help me find a counterexample but I failed. I am not even sure whether the answer is "no".