Skip to content

gh-116738: Make lzma module thread-safe - #142947

Merged
kumaraditya303 merged 6 commits into
python:mainfrom
yoney:ft_lzma_2
Jan 13, 2026
Merged

gh-116738: Make lzma module thread-safe#142947
kumaraditya303 merged 6 commits into
python:mainfrom
yoney:ft_lzma_2

Conversation

@yoney

@yoney yoney commented Dec 18, 2025

Copy link
Copy Markdown
Contributor

Make the attributes in lzma module thread-safe on the free-threading build. Attributes (check, eof, needs_input, unused_data) are now stored atomically or accessed via mutex-protected getters.

Even though the operations are protected by locks, the attributes exposed via PyMemberDef (check, eof, needs_input, unused_data) should still be updated atomically while holding the lock, or accessed via mutex‑protected getters, because they can be read without acquiring the lock.

This issue was not addressed in #140711. A small addition that repeatedly reads these attributes shows the issue under tsan build.

WARNING: ThreadSanitizer: data race (pid=2367860)
  Atomic read of size 1 at 0x7fdbc9506ee0 by thread T19:
    #0 _Py_atomic_load_char_relaxed /workspace/ft_lzma_tsan/./Include/cpython/pyatomic_gcc.h:311 (python+0x99e392) (BuildId: ce943d12d16f92ec125f1db1856a1617d3d75854)
    #1 PyMember_GetOne /workspace/ft_lzma_tsan/Python/structmember.c:37 (python+0x99e392)
    #2 member_get /workspace/ft_lzma_tsan/Objects/descrobject.c:180 (python+0x60f4d5) (BuildId: ce943d12d16f92ec125f1db1856a1617d3d75854)
...
...
...
Previous write of size 1 at 0x7fdbc9506ee0 by thread T11:
    #0 decompress /workspace/ft_lzma_tsan/./Modules/_lzmamodule.c:1067 (_lzma.cpython-315td-x86_64-linux-gnu.so+0x7256) (BuildId: 978a710f12e938f2fbc3233c01c40cf693105ef1)
    #1 _lzma_LZMADecompressor_decompress_impl /workspace/ft_lzma_tsan/./Modules/_lzmamodule.c:1139 (_lzma.cpython-315td-x86_64-linux-gnu.so+0x7256)
    #2 _lzma_LZMADecompressor_decompress /workspace/ft_lzma_tsan/./Modules/clinic/_lzmamodule.c.h:157 (_lzma.cpython-315td-x86_64-linux-gnu.so+0x7256)
...
...
...
SUMMARY: ThreadSanitizer: data race /workspace/ft_lzma_tsan/./Modules/_lzmamodule.c:1043 in decompress

cc: @mpage @colesbury

Make the attributes in lzma module thread-safe on the free-threading build.
Attributes (check, eof, needs_input, unused_data) are now stored atomically
or accessed via mutex-protected getters.
Comment thread Modules/_lzmamodule.c Outdated
@emmatyping

Copy link
Copy Markdown
Member

Looks like jobs failed due to a spurious GitHub outage. I've re-run the failed jobs.

Comment thread Modules/_lzmamodule.c Outdated
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

@colesbury colesbury left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@emmatyping @picnixz - do you have any further feedback on the PR?

Comment thread Modules/_lzmamodule.c
@kumaraditya303
kumaraditya303 enabled auto-merge (squash) January 13, 2026 13:36
@kumaraditya303
kumaraditya303 merged commit fca7fec into python:main Jan 13, 2026
47 checks passed
@yoney
yoney deleted the ft_lzma_2 branch January 27, 2026 17:07
thunder-coding pushed a commit to thunder-coding/cpython that referenced this pull request Feb 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment