AttributeError: 'Tensor' object has no attribute 'numpy' while extending keras sequential model

Question

I am trying to compile a Keras Sequential model (in TF2) in the eager execution mode. Following is my custom layer:

class CustomLayer(Layer):
  def __init__(self, output_shape, **kwargs):
    self.output_shape = output_shape
    super(CustomLayer, self).__init__(**kwargs)

  def build(self, input_shape):
    assert len(input_shape) == 2
    input_dim = input_shape[1]

  def call(self, inputs, mask=None, **kwargs):
    y_pred = inputs.numpy() #<---- raise error
    return y_pred

I use this layer to extend another Sequential Model as follow:

encoder = Sequential(encoders)  # encoders is a bunch of Dense layers
encoder.compile(
    loss='mse',
    optimizer=SGD(lr=self.learning_rate, decay=0, momentum=0.9),
    run_eagerly=True
)
self.MyModel = Sequential([encoder, CustomLayer(output_shape=output_shape)])
self.MyModel.compile(
    loss='MSE',
    optimizer=sgd,
    run_eagerly=True
)

The original model is an autoencoder with denoising layers, and I'm adding a new layer after the bottleneck to make some customized predictions. I need to have access to the tensor values within the new layer. Doing so raises the following error:

    Traceback (most recent call last):
  File "main.py", line 185, in initialize
    self.DEC = Sequential([encoder, CustomingLayer(input_shape=input_shape,
  File ".virtualenvs/vision/lib/python3.8/site-packages/tensorflow/python/training/tracking/base.py", line 530, in _method_wrapper
    result = method(self, *args, **kwargs)
  File ".virtualenvs/vision/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File ".virtualenvs/vision/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py", line 699, in wrapper
    raise e.ag_error_metadata.to_exception(e)
AttributeError: Exception encountered when calling layer "custom_layer" (type CustomLayer).

in user code:

    File "custom_layer.py", line 25, in call  *
        y_pred = inputs.numpy()

    AttributeError: 'Tensor' object has no attribute 'numpy'


Call arguments received:
  • inputs=tf.Tensor(shape=(None, 10), dtype=float32)
  • mask=None
  • kwargs={'training': 'None'}

I need to pass the inputs to a third-party package to do some calculations, which only supports numpy as an input — Salman
– Salman, Commented Mar 1, 2022 at 12:56
Do you just want to use the output of your third-party function during inference or also during training? — AloneTogether
– AloneTogether, Commented Mar 1, 2022 at 13:33
this package helps me in calculating the loss. so I will call it during the training as well. — Salman
– Salman, Commented Mar 1, 2022 at 13:35
This function is the Kmeans from 'sklearn' package. New layer basically is a clustering layer and I've defined new loss function based on the result of the clusters — Salman
– Salman, Commented Mar 1, 2022 at 13:38

Salman · Accepted Answer · 2022-03-01 14:05:06Z

2

The direct using of this numpy function is impossible - as it's neither implemented in Tensorflow nor in Theano. Moreover, there is no direct correspondence between tensors and arrays. Tensors should be understood as algebraic variables whereas numpy arrays as numbers. Tensor is an abstract thing and applying a numpy function to it is usually impossible.

But you could still try to re-implement your function on your own using keras.backend. Then you'll use the valid tensor operations and no problem would be raised.

Another way to tackle your problem would be to use tf.numpy_function, see the documentation, this allows you to use numpy functions but there are some limitations.

edited Mar 1, 2022 at 14:05

Salman

1,0161 gold badge8 silver badges24 bronze badges

answered Mar 1, 2022 at 13:15

MD Mushfirat Mohaimin

2,0724 gold badges14 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Salman Over a year ago

Thanks for the answer. The problem is when I'm compiling the new model, it does not execute eagerly and I don't have access to tensors values. I have not used the tf.numpy_fuction already. but I'll take a look at it

Salman Over a year ago

Using tf.numpy_function worked for me

Collectives™ on Stack Overflow

AttributeError: 'Tensor' object has no attribute 'numpy' while extending keras sequential model

1 Answer 1

2 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Related