Moving kafka log to another broker

Question

Is it possible to move the log on a kafka broker to a different broker?

I'm using kafka as an eventstore (as suggested on their site) so i don't erase the log ever and i am wondering if it is possible to move the log to another broker if my broker fills up (as i expand the cluster).

Also, when a new broker is added and partitions are assigned to it if i understand correctly the old data from the moved partitions stays on it's original broker - meaning if a new consumer is added who wants to consume from the beginning he might not get the events in the correct order - i could solve this if the log was movable.

I know partitions are movable, but if i understood correctly the old data is not moved.

Thank you.

Kulasangar · Accepted Answer · 2016-05-31 12:32:26Z

0

Did you try increasing the log.index.size.max.bytes value from the Broker Config? You might have a look at Broker Configs (consider changing the log.segment.bytes & log.retention.size as well).

I'm not sure of whether you'll be able to move a log file from one broker to another. But why do you have to do that, since log files are to refer what's going on.

Check all the log related configuration properties over here.

answered May 31, 2016 at 12:32

Kulasangar

9,5326 gold badges66 silver badges97 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Moving kafka log to another broker

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related