421 questions
3
votes
1
answer
74
views
How to decode a tfrecord based on the available meta data
I have a three tfrecords (train,test valid) from the a deepmind github repo (https://github.com/google-deepmind/deepmind-research/tree/master/meshgraphnets) that I am trying to decode (at the minute ...
1
vote
1
answer
86
views
Incorrect header check error when trying to access a tf_record file
I've recently ran into an error when trying to evaluate my training model onto my testing files. The error is thrown at the batch_raw = raw_iter.get_next():
try:
while N_b<max_batches:
...
- 15
0
votes
1
answer
57
views
How can I create TFRecords with a dictionary of multiple 2D Tensors and read them back without parsing errors?
I have looked at other posts e.g. Tensorflow TFRecord: Can't parse serialized example to try and resolve the issue of parsing serialized data failures but cannot see where I'm misunderstanding the ...
- 91
0
votes
0
answers
38
views
Can TFRecordDataSetm start reading elements from a random spot in the file?
I have an application with 200GB data and I can't load it in RAM. Based on the data/problem, it will be extremely beneficial to have TFRecord files "sorted" in a certain way that makes them ...
0
votes
0
answers
58
views
How to convert polygon points in a XML file to TFRecord format?
I am currently using a Python script to convert XML files containing object annotations into TFRecord files, which are necessary for training object detection models like Faster R-CNN in TensorFlow. ...
1
vote
0
answers
48
views
Multi-Input network for image classification using TensorflowDataset gets none values not supported error
I have a dataset of tfrecords and mapped them to a Tensorflow dataset. The dataset structure has a dict structure as follows:
{ "image" : ImageDataTensor, "additional_features" : ...
- 23
0
votes
0
answers
73
views
How can I create sequences (windows) from data in a TFRecord?
I have around 1800 short videos. Each video is around 30 seconds. I trained a VAE to encode each frame into a latent vector (of size 200).
Using this VAE, then I created a TFrecord with an entry for ...
- 1,638
0
votes
1
answer
108
views
Excessive RAM usage for TFRecord file while feeding into any model
Problem
I have a *.tfrecords file that I want to feed in a ConvLSTM2D model, created using Tensorflow. Here is the model structure.
model = Sequential([
ConvLSTM2D(64, (3, 3), activation='relu', ...
- 85
0
votes
0
answers
153
views
Generate tfrecord.py : UnicodeDecodeError: ‘utf-8’ codec can’t decode byte
I converted the xml files I created in the label img file to csv and I want to convert them to a tf record file from there. I tried many ways and looked at many forums and videos, but I still cannot ...
user22776335
0
votes
1
answer
105
views
TFRecord - Diffrentiate between Example & SequenceExample
If I have a TFRecord file, how do I find if the underlying proto is Example vs SequenceExample?
- 124
2
votes
0
answers
30
views
2
votes
1
answer
202
views
`py_function` causes `ragged_batch()` not working in `tf.data`
I'm working on an object detection project, and use tf.data.Dataset input pipeline to load local data. Because object detection requires not only image but also annotations, and the different ...
- 726
0
votes
1
answer
110
views
time series in tensorflow with tfrecords
I am currently writing a tensorflow model to predict the next value of a label depending on a temporal sequence of items.
I created a series of tfrecords files, each corresponding to a single trial ...
- 57
0
votes
1
answer
288
views
Tensorflow TFRecord: Can't parse serialized example when loop through tensor
I am trying to read a tfrecord but keep getting this error
InvalidArgumentError: Key: sign. Can't parse serialized Example.
Here's what I did:
def decode_tfrec(record_bytes):
features = tf.io....
- 53
1
vote
1
answer
656
views
reading and decoding a .tfrecord file, and saving it to .npz file or reading it in numpy/pytorch
I'm currently working on implementing a pytorch version of this tensorflow project. They published a paper which can be found here.
Dataset formats and where to find them
The datasets available are ....
