All Questions
48 questions
0
votes
1
answer
259
views
How to load individual .npz files as samples into a Tensorflow Dataset
I am having a problem with loading the data that make up the dataset. My previous (working) approach was to use a pandas DataFrame, but for larger datasets, the training process gets killed, as the ...
- 71
1
vote
1
answer
651
views
Use tf.data.Dataset direct form .npz file
I have 115GB .npz file.
It is possible to create tf.data.Dataset directly from this file to avoid 115GB RAM allocation?
According to:
https://www.tensorflow.org/tutorials/load_data/numpy
First step is ...
- 149
0
votes
0
answers
461
views
Load numpy array to a Tensorflow dataset
I am trying to do image colorization. I have 5000 images (256x256x3) and would like not to load all data in my program (for memory reason). I have found that it is possible to use ImageDataGenerator....
- 1
0
votes
0
answers
160
views
Replacing of weights with set_weights or any other method
I am using tensorflow federated with following imports.
import tensorflow as tf
import tensorflow_federated as tff
import collections
import os
import random
import math
import time
import numpy as np
...
- 81
2
votes
1
answer
237
views
Extract only a portion of a numpy array from tf.data
I have a NumPy array of shape 500,36,24,72. Now I want to create a data pipeline for a problem using tf.data. For every iteration, only a subset of the array is required, for example, first the model ...
- 53
0
votes
0
answers
1k
views
Problem mapping data (AttributeError: 'Tensor' object has no attribute 'numpy')
I try to map the data into the function that creates mfcc spectrogram. It says that
WARNING:tensorflow:Using a while_loop for converting IO>AudioResample
--------------------------------------------...
- 1
1
vote
1
answer
922
views
How to change the values of a tf.Dataset object in a specific index
The structure of my tf.data.Dataset object is as follow.
((3, 400, 1), (3, 400, 1))
I would like to divide the elements in the 3rd row, of each element by 10. My code is as follows. But it complains ...
- 6,568
1
vote
0
answers
51
views
Create TensorFlow Dataset using File Links
I have around 2.1 million multi-channel images stored individually as npy-file in a directory. The directory currently works as centralized storage for all images. The images with 25 Pixels x 25 ...
- 97
2
votes
1
answer
462
views
Prepare data input for tensorflow from numpy and scipy.sparse
How to prepare data for input into a tensorflow model (say a keras Sequential one) ?
I know how to prepare x_train, y_train, x_test and y_test using numpy and scipy (eventually pandas, sklearn style) ...
- 329
1
vote
1
answer
1k
views
Create a tensorflow dataset based on a "multi-input"
Problem
Create a tf.data.Dataset object from a numpy array that contains multiple X array.
Explaination
This is the model that I'm using, some layers eliminated for reduce the image:
As you can see, ...
- 3,065
1
vote
2
answers
7k
views
This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported
I encountered this error, how do I resolve it?
NotImplementedError: Cannot convert a symbolic Tensor (lstm_2/strided_slice:0) to a numpy array. This error may indicate that you're trying to pass a ...
- 43
1
vote
1
answer
398
views
Consistent extraction of data from tensorflow dataset
I want to extract the data from a tensorflow dataset consistently into numpy arrays/tensors. I'm loading pictures witn
data = keras.preprocessing.image_dataset_from_directory(
'./data',
labels='...
- 253
0
votes
2
answers
3k
views
Using Datasets from large numpy arrays in Tensorflow
I'm trying to load a dataset, stored in two .npy files (for features and ground truth) on my drive, and use it to train a neural network.
print("loading features...")
data = np.load("[.....
- 64
2
votes
1
answer
2k
views
Merging tensorflow dataset batches
Please consider the code below:
import tensorflow as tf
import numpy as np
simple_features = np.array([
[1, 1, 1],
[2, 2, 2],
[3, 3, 3],
[4, 4, 4],
[5, 5,...
- 391
0
votes
1
answer
398
views
Data augmentation in the data loading of tensorflow dataset, tfds, resulting in TypError or AttributeError
I am trying to do some data augmentation but i am not so familiar with tensors.
This is the code i started with:
def _random_apply(func, x, p):
return tf.cond(tf.less(tf.random.uniform([], minval=0, ...
- 251