If you truly need the full data before processing then just read it all first before processing it... unless of course you have control of the data being sent, then the answer from u/candied_orange could be a better way to go.

But many scenarios where you think you need the full data can actually be converted to a streaming implementation. The real question to ask is: "What are the pros and cons of processing this in a streaming manner" where the common tradeoff for streaming is usually a lower memory footprint but giving up keeping a re-usable data-structure (for example SAX or StAX vs DOM XML processing). However, streaming processors could always still build those data-structures, so having a streaming implementation available with that ability is usually the best of both worlds.

If you truly need the full data before processing then just read it all first before processing it... unless of course you have control of the data being sent, then the answer from u/candied_orange could be a better way to go.

But many scenarios where you think you need the full data can actually be converted to a streaming implementation. The real question to ask is: "What are the pros and cons of processing this in a streaming manner" where the common tradeoff for streaming is usually a lower memory footprint but giving up keeping a re-usable data-structure (for example SAX or StAX vs DOM XML processing). However, streaming processors could always still build those data-structures, so having a streaming implementation available with that ability is usually the best of both worlds.