For my job, I'm working on a project that involves working with robot sensor data. We've been using protobuf to serialize and transfer data. I'm trying to set up a new protobuf type but this data involves a large array (e.g. a million elements) of doubles.
So I have a protobuf file set up something like this
syntax = "proto2";
message Sensor {
optional string name = 1;
repeated double temperature = 2 [packed = true];
optional int32 humidity = 3;
}
Where "temperature" is meant to hold a large array of temperatures. But when it comes to actually creating the Sensor object and storing the temperatures the only way I can find to do this is through add_temperature
e.g.
Sensor sensor
sensor.add_temperature(23.4)
And so I'll need to call add_temperature a million times. Calling a function a million times strikes me as very inefficient way to build up an array of a million elements. Ideally I'd like to reserve an appropriate amount of memory and call one function that copies the data.
I see on the Overview page for Protobufs it's written that they are "less than maximally efficient" for large arrays of floating point numbers. Does this mean it's infeasible to use protobufs to serialize large arrays of floating points? Or is there a way I can make this work?
