Python flatten array inside numpy array

Question

I have a pretty stupid question, but for some reason, I just can't figure out what to do. I have a multi-dimensional numpy array, that should have the following shape:

(345138, 30, 300)

However, it actually has this shape:

(345138, 1)

inside the 1 element-array is the array containing the shape

(30, 300)

So how do I "move" the inside array, so that the shape is correct?

At the moment it looks like this:

[[ array([[0, 0, 0, ..., 0, 0, 0],
   [0, 0, 0, ..., 0, 0, 0],
   [0, 0, 0, ..., 0, 0, 0],
   ..., 
   [0, 0, 0, ..., 0, 0, 0],
   [0, 0, 0, ..., 0, 0, 0],
   [0, 0, 0, ..., 0, 0, 0]], dtype=int32)]
 [ array([[0, 0, 0, ..., 0, 0, 0],
   [0, 0, 0, ..., 0, 0, 0],
   [0, 0, 0, ..., 0, 0, 0],
   ...,

but I want this without the array(...), dtype=32 and move what is in there into the first array so that the shape is (345138, 30, 300) and looks like this:

[[ [0, 0, 0, ..., 0, 0, 0],
   [0, 0, 0, ..., 0, 0, 0],
   [0, 0, 0, ..., 0, 0, 0],
   ..., 
   [0, 0, 0, ..., 0, 0, 0],
   [0, 0, 0, ..., 0, 0, 0],
   [0, 0, 0, ..., 0, 0, 0]],
 [ [0, 0, 0, ..., 0, 0, 0],
   [0, 0, 0, ..., 0, 0, 0],
   [0, 0, 0, ..., 0, 0, 0],
   ...,

Any ideas?

Try: np.array(x) and if the shapes are compatible they will be squashed. — cs95
– cs95, Commented Aug 17, 2017 at 15:52
hmmm no, unfortunately, this didn't change anything the one-element array is ruining everything — SirTobi
– SirTobi, Commented Aug 17, 2017 at 15:56
How did you create this array in the first place? See if you can fix that. — Warren Weckesser
– Warren Weckesser, Commented Aug 17, 2017 at 15:58
Yeah you are probably right, I retrieve the array from a dataframe via df.as_matrix() so I am assuming it was incorrectly put into the dataframe. — SirTobi
– SirTobi, Commented Aug 17, 2017 at 16:09

hpaulj · Accepted Answer · 2017-08-17 16:07:30Z

Looks like you have a 2d array that contains 2d arrays (object dtype). I can construct one like that with:

In [972]: arr = np.empty(4,dtype=object)
In [973]: arr = np.empty((4,1),dtype=object)
In [974]: for i in range(4): arr[i,0]=np.ones((2,3),int)
In [975]: arr
Out[975]: 
array([[array([[1, 1, 1],
       [1, 1, 1]])],
       [array([[1, 1, 1],
       [1, 1, 1]])],
       [array([[1, 1, 1],
       [1, 1, 1]])],
       [array([[1, 1, 1],
       [1, 1, 1]])]], dtype=object)

Simply wrapping that in np.array does not work; not does applying tolist:

In [976]: np.array(arr)
Out[976]: 
array([[array([[1, 1, 1],
       [1, 1, 1]])],
       [array([[1, 1, 1],
       [1, 1, 1]])],
       [array([[1, 1, 1],
       [1, 1, 1]])],
       [array([[1, 1, 1],
       [1, 1, 1]])]], dtype=object)
In [977]: arr.tolist()
Out[977]: 
[[array([[1, 1, 1],
         [1, 1, 1]])], [array([[1, 1, 1],
         [1, 1, 1]])], [array([[1, 1, 1],
         [1, 1, 1]])], [array([[1, 1, 1],
         [1, 1, 1]])]]

One way of 'flattening' is to use some version of concatenate:

In [978]: np.stack(arr.ravel())
Out[978]: 
array([[[1, 1, 1],
        [1, 1, 1]],

       [[1, 1, 1],
        [1, 1, 1]],

       [[1, 1, 1],
        [1, 1, 1]],

       [[1, 1, 1],
        [1, 1, 1]]])
In [979]: _.shape
Out[979]: (4, 2, 3)

I used ravel to reduce the outer array to 1d, which stack can use as a list. stack acts like np.array in that it combines the elements on a new axis (which we can specify).

tolist and array can work together:

In [981]: np.array(arr.tolist())
Out[981]: 
array([[[[1, 1, 1],
         [1, 1, 1]]],

       [[[1, 1, 1],
         [1, 1, 1]]],


       [[[1, 1, 1],
         [1, 1, 1]]],


       [[[1, 1, 1],
         [1, 1, 1]]]])
In [982]: _.shape
Out[982]: (4, 1, 2, 3)

Or tolist plus squeeze (which is actually np.asarray(...).squeeze())

In [983]: np.squeeze(arr.tolist())
Out[983]: 
array([[[1, 1, 1],
        [1, 1, 1]],

       [[1, 1, 1],
        [1, 1, 1]],

       [[1, 1, 1],
        [1, 1, 1]],

       [[1, 1, 1],
        [1, 1, 1]]])
In [984]: _.shape
Out[984]: (4, 2, 3)

Hi, thank you so much, this worked! It also crashed my pc though :D You are awesome!

Collectives™ on Stack Overflow

Python flatten array inside numpy array

1 Answer 1

1 Comment

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Linked

Related