get a numpy array from a sequence of images in a folder

Question

I have a folder, say video1 with bunch of images in order frame_00.png, frame_01.png, ...

What I want is a 4D numpy array in the format (number of frames, w, h, 3)

This is what I did, but I think it is quite slow, is there any faster or more effecient method to achieve the same thing?

folder = "video1/"

import os
images = sorted(os.listdir(folder)) #["frame_00", "frame_01", "frame_02", ...]

from PIL import Image 
import numpy as np 

video_array = []
for image in images:
    im = Image.open(folder + image)
    video_array.append(np.asarray(im)) #.transpose(1, 0, 2))

video_array = np.array(video_array)
print(video_array.shape)
#(75, 50, 100, 3)

Answer 1

There's an older SO thread that goes into a great deal of detail (perhaps even a bit too much) on this very topic. Rather than vote to close this question as a dup, I'm going to give a quick rundown of that thread's top bullet points:

• The fastest commonly available image reading function is imread from the cv2 package.
• Reading the images in and then adding them to a plain Python list (as you are already doing) is the fastest approach for reading in a large number of images.
• However, given that you are eventually converting the list of images to an array of images, every possible method of building up an array of images is almost exactly as fast as any other
  • Although, interestingly enough, if you take the approach of assigning images directly to a preallocated array, it actually matters which indices (ie which dimension) you assign to in terms of getting optimal performance.

So basically, you're not going to be able to get much faster while working in pure, single-threaded Python. You might get a boost from switching to cv2.imread (in place of PIL.Image.open).

Answer 2

PNG is an extremely slow format, so if you can use almost anything else, you'll see a big speedup.

For example, here's an opencv version of your program that gets the filenames from command-line args:

#!/usr/bin/python3

import sys
import cv2
import numpy as np

video_array = []
for filename in sys.argv[1:]:
    im = cv2.imread(filename)
    video_array.append(np.asarray(im)) 

video_array = np.array(video_array)
print(video_array.shape)

I can run it like this:

$ mkdir sample
$ for i in {1..100}; do cp ~/pics/k2.png sample/$i.png; done
$ time ./readframes.py sample/*.png
(100, 2048, 1450, 3)

real    0m6.063s
user    0m5.758s
sys 0m0.839s

So 6s to read 100 PNG images. If I try with TIFF instead:

$ for i in {1..100}; do cp ~/pics/k2.tif sample/$i.tif; done
$ time ./readframes.py sample/*.tif
(100, 2048, 1450, 3)

real    0m1.532s
user    0m1.060s
sys 0m0.843s

1.5s, so four times faster.

You might get a small speedup with pyvips:

#!/usr/bin/python3

import sys
import pyvips
import numpy as np

# map vips formats to np dtypes
format_to_dtype = {
    'uchar': np.uint8,
    'char': np.int8,
    'ushort': np.uint16,
    'short': np.int16,
    'uint': np.uint32,
    'int': np.int32,
    'float': np.float32,
    'double': np.float64,
    'complex': np.complex64,
    'dpcomplex': np.complex128,
}   

# vips image to numpy array
def vips2numpy(vi):
    return np.ndarray(buffer=vi.write_to_memory(),
                      dtype=format_to_dtype[vi.format],
                      shape=[vi.height, vi.width, vi.bands])

video_array = []
for filename in sys.argv[1:]:
    vi = pyvips.Image.new_from_file(filename, access='sequential')
    video_array.append(vips2numpy(vi)) 

video_array = np.array(video_array)
print(video_array.shape)

I see:

$ time ./readframes.py sample/*.tif
(100, 2048, 1450, 3)

real    0m1.360s
user    0m1.629s
sys 0m2.153s

Another 10% or so.

Finally, as other posters have said, you could load frames in parallel. That wouldn't help TIFF much, but it would certainly boost PNG.