Skip to content

GH-3706: Mark writer aborted on end() failure to avoid flushing incomplete files - #3707

Open
arg3t wants to merge 1 commit into
apache:masterfrom
arg3t:footer-corruption-on-write-fix
Open

GH-3706: Mark writer aborted on end() failure to avoid flushing incomplete files#3707
arg3t wants to merge 1 commit into
apache:masterfrom
arg3t:footer-corruption-on-write-fix

Conversation

@arg3t

@arg3t arg3t commented Aug 4, 2026

Copy link
Copy Markdown

Rationale for this change

Makes ParquetFileWriter#end() mark the writer aborted before flushing, so an incomplete file is never committed on failure. Previously close() ran before the catch block in withAbortOnFailure and withAbortOnFailure only caught IOException, causing:

  • an incomplete file being flushed to storage when serializing indexes/stats/footer throws any exception
  • a RuntimeException (e.g. OOM) to be missed, leaving the writer un-aborted and still flushing

What changes are included in this PR?

  • withAbortOnFailure catches Throwable (not just IOException) and marks the writer aborted.
  • end() runs close() in an outer finally, so abort happens before close() and the flush is skipped on failure.

Are these changes tested?

Yes. 2 new tests in TestParquetFileWriter inject an IOException/RuntimeException during end() and assert the stream is never flushed.

Are there any user-facing changes?

No. Successful writes are unchanged; only the failure path is affected.

Closes #3706

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant