-1

In order to investigate whether the decisions of a CNN for image classification are predominantly driven by image regions related to relatively fine object features, I am looking for an image pertubation technique that allows for a local disturbance of groups of pixels while retaining the general structure of the displayed object.

In this regard I was thinking about using a moving kernel (similar to a mean filter) in which I randomly select the value of one pixel within the kernel and attribute it to the center pixel. By gradually increasing the kernel size I am aiming to introduce varying levels of disturbance.

How can I effectively randomly select a pixel in a kernel and attribute its value to the center pixel using Python?

New contributor
Adrian Straker is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
5
  • 1
    Asking for recommendations is off-topic as it will invite opinions rather than facts. I voted to close. We would need to know your specific expectations to be able to help. Please take the tour and read How to Ask to see how this site works and how it can help you to solve your programming-related problems. Commented 2 days ago
  • I disagree and don't think this is opinion based. It is asking for an efficient implementation of a filter taking a random sample out of a sliding kernel in numpy/python. This is how I read the question. However, please include what you tried? Do you have a sample image? What is the dimension of image and kernel, respectively? Commented 2 days ago
  • Choose random values x,y for pixel position, choose dx,dy < N where N is the size of the rectangular support region then swap those two pixels. I rather like the designer tweaked JPEGs that are to a human eye very obviously of a farm tractor but some AI classifies as "a cat". Things have improved a lot on that front recently. Commented 2 days ago
  • This is off-topic as its a research problem, not a programming one, how to attribute model outputs is a research problem and not a programming one. Commented 2 days ago
  • @Dr.Snoopy - Do I overread something? The only sentences with a question mark I see in OP's question is "How can I effectively randomly select a pixel in a kernel and attribute its value to the center pixel using Python?", which is highly similar to the headline. So why is this not about programming? Commented 2 hours ago

1 Answer 1

0

Let me provide a basic and a fully vectorized implementation that picks a random sample out of each kernel_size x kernel_size region sliding over the image.

Naive approach

def random_filter_naive(image: np.ndarray, kernel_size: int = 3, seed: int = None) -> np.ndarray:
    # Check input dimensions
    assert image.ndim == 2, "Input image must be a 2D array."
    assert kernel_size > 0, "Kernel size must be a positive integer."
    assert image.shape[0] >= kernel_size and image.shape[1] >= kernel_size, "Image dimensions must be at least as large as the kernel size."

    if seed is not None:
        np.random.seed(seed)

    # Pad the image to handle borders
    pad_size       = kernel_size // 2    
    padded_image   = np.pad(image, pad_size, mode='reflect')
    filtered_image = np.zeros_like(image)

    for i in range(image.shape[0]):
        for j in range(image.shape[1]):
            # Extract the neighborhood
            neighborhood = padded_image[i:i+kernel_size, j:j+kernel_size]
            # Flatten the neighborhood and choose a random element
            random_value = np.random.choice(neighborhood.flatten())
            filtered_image[i, j] = random_value

    return filtered_image

Time taken for with naive approach on image with size (512, 512) and kernel size 13: 1.4364 seconds

Optimized approach

def random_filter(image: np.ndarray, kernel_size: int = 3, seed: int = None) -> np.ndarray:
    # Check input dimensions
    assert image.ndim == 2, "Input image must be a 2D array."
    assert kernel_size > 0, "Kernel size must be a positive integer."
    assert image.shape[0] >= kernel_size and image.shape[1] >= kernel_size, "Image dimensions must be at least as large as the kernel size."

    if seed is not None:
        np.random.seed(seed)

    # Pad the image to handle borders
    pad_size     = kernel_size // 2
    padded_image = np.pad(image, pad_size, mode='reflect')
    rows, cols   = image.shape

    # Generate random row and column offsets for each pixel, respectively
    rand_row_offsets = np.random.randint(0, kernel_size, size=(rows, cols))
    rand_col_offsets = np.random.randint(0, kernel_size, size=(rows, cols))

    # Compute the indices of the random neighbors
    row_indices = np.arange(rows)[:, None] + rand_row_offsets
    col_indices = np.arange(cols)[None, :] + rand_col_offsets

    # Use advanced indexing to select random neighbors
    filtered_image = padded_image[row_indices, col_indices]

    return filtered_image

Time taken for fully vectorized approach on image with size (512, 512) and kernel size 13: 0.0050 seconds

original and filtered image Image courtesy of the Computing Sciences Unit, Tampere University

Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.