Exploring Solutions to Tackle Low-Quality Contributions on GitHub

[moraesc]

Hey everyone,

I wanted to provide an update on a critical issue affecting the open source community: the increasing volume of low-quality contributions that is creating significant operational challenges for maintainers.

We’ve been hearing from you that you’re dedicating substantial time to reviewing contributions that do not meet project quality standards for a number of reasons - they fail to follow project guidelines, are frequently abandoned shortly after submission, and are often AI-generated. As AI continues to reshape software development workflows and the nature of open source collaboration, I want you to know that we are actively investigating this problem and developing both immediate and longer-term strategic solutions.

What we're exploring

We’ve spent time reviewing feedback from community members, working directly with maintainers to explore various solutions, and looking through open source repositories to understand the nature of these contributions. Below is an overview of the solutions we’re currently evaluating.

Short-term solutions:

• Repository-level PR controls
  • Disable PRs entirely - This has been a long-standing request (since 2016) and is a first step in giving maintainers more tools to customize their PR experience. This also addresses use cases where maintainers want to share code without accepting external contributions, such as mirror repositories. A repo-level setting would also eliminate the need for custom automations that various open source projects are building to automatically close external PRs.
  • Restrict PRs to collaborators - This provides more granular access control, allowing contributions exclusively from existing collaborators while blocking external contributors.
• The ability to delete a PR from the UI - This provides maintainers with the ability to remove spam or low-quality PRs directly from the interface to improve repository organization.

Long-term direction:

As AI adoption accelerates, we recognize the need to proactively address how it can potentially transform both contributor and maintainer workflows. We are exploring:

• Enhanced permission models - Provide maintainers with more tools so they can customize their PR experience:
  • More granular controls for determining who can create and reviewing PRs beyond blocking all users or restricting to collaborators only.
  • Ways for maintainers to define criteria that PRs must meet before they can be opened.
• Improved triage tools - Potentially leveraging AI to evaluate contributions against project guidelines and standards to identify which contributions maintainers should focus on reviewing.
• Transparency in AI-assisted contributions - Improved visibility and attribution when AI tools are used throughout the PR lifecycle.

Next Steps

These are some starting points, and we’re continuing to explore both immediate improvements and long-term solutions. Please share your feedback, questions, or concerns in this thread. Your input is crucial to making sure we’re building the right things and tackling this challenge effectively. As always, thank you for being part of this conversation. Looking forward to hearing your thoughts and working together to address this problem.

[xavidop]

such a great intiative

[xavidop]

I know this is a pretty ambitious idea and not trivial to implement, but it would be really powerful to have an AI-detection mechanism with a configurable threshold at the repository or organization level. That way, teams could decide what percentage of AI-generated code is acceptable in pull requests.

Another possible approach would be to define a set of rules or prompts and evaluate pull requests against them. PRs that don’t meet those rules could be automatically flagged or potentially even closed.

[moraesc]

Another possible approach would be to define a set of rules or prompts and evaluate pull requests against them. PRs that don’t meet those rules could be automatically flagged or potentially even closed.

This is definitely something we’re exploring. One idea is to leverage a repository’s CONTRIBUTING.md file as a source of truth for project guidelines and then validate PRs against any defined rules.

In regards to AI-generated code, have you seen cases where the code is AI-generated but still high-quality and actually solves the problem? Or is it always just something you want to close out immediately? I'm curious if an AI-detection mechanism would rule out PRs where AI is used constructively, but interested in investigating this more and understanding what sensible would thresholds look like.

[xavidop]

As of today, I would say that 1 out of 10 PRs created with AI is legitimate and meets the standards required to open that PR. On 28 Jan 2026, at 18:41, Camilla Moraes ***@***.***> wrote:  Another possible approach would be to define a set of rules or prompts and evaluate pull requests against them. PRs that don’t meet those rules could be automatically flagged or potentially even closed. This is definitely something we’re exploring. One idea is to leverage a repository’s CONTRIBUTING.md file as a source of truth for project guidelines and then validate PRs against any defined rules. In regards to AI-generated code, have you seen cases where the code is AI-generated but still high-quality and genuinely solves the problem? Or is it alwaays just something you want to close out immediately? I'm curious because I'm wondering if an AI-detection mechanism would rule out PRs where AI is used constructively, but that's where we'd want to test this thoroughly and understand what sensible thresholds look like. — Reply to this email directly, view it on GitHub<#185387 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABBWEYEKF6WLNDKE376L3GD4JDYFXAVCNFSM6AAAAACS7B7C7OVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTKNRTGEZTMMI>. You are receiving this because you commented.Message ID: ***@***.***>

[ThiefMaster]

Disable PRs entirely

PLEASE make this better than disabling issues: Imparticular, do not restrict access to previously opened PRs. Fine to hide the UI or even the ability to list them, but let people access them with a direct link. I'd also appreciate if that was done in repos where issues got disabled.

Context: Someone realizing that they no longer want to allow issues (or PRs) should NOT make any existing content disappear for good. Especially in cases where this content is referenced elsewhere and suddenly becomes a dead link...

The ability to delete a PR from the UI

Similar to my suggestion above, this should be limited in time. It's amazing to delete spam PRs. Hacktoberfest-cheating, Indian-Youtube-nonsense, and the likes. But if there is a controversy around a PR, which has happened numerous times before e.g. due to highly unpopular decisions by repo maintainers (Minio comes to my mind), then they should NOT be able to permanently delete at PR just because it's inconvenient or shows their true face.

My suggestion would be to have a (very) limited timeframe to delete a PR. Maybe a week in case of low activity (e.g. a spam PR the maintainers did not see), much less in case of high activity. Having an exception for "quiet" PRs (no meaningful amount of comments) and allowing to delete those for longer would be perfectly fine of course so someone who rarely checks their repo can delete spam even when they notice it late.

PS: Yes, I'm aware that deleting individual comments on PRs and issues is already possible. But that's a hassle compared to being able to nuke the whole thing with two clicks.

[moraesc]

PLEASE make this better than disabling issues: Imparticular, do not restrict access to previously opened PRs. Fine to hide the UI or even the ability to list them, but let people access them with a direct link. I'd also appreciate if that was done in repos where issues got disabled.

Context: Someone realizing that they no longer want to allow issues (or PRs) should NOT make any existing content disappear for good. Especially in cases where this content is referenced elsewhere and suddenly becomes a dead link...

Would you want to be able to navigate to the PR just for viewing purposes or to potentially keep making changes to it? I agree it makes sense for users to still be able to view their PRs - will see what we can do here.

Similar to my suggestion above, this should be limited in time. It's amazing to delete spam PRs. Hacktoberfest-cheating, Indian-Youtube-nonsense, and the likes. But if there is a controversy around a PR, which has happened numerous times before e.g. due to highly unpopular decisions by repo maintainers (Minio comes to my mind), then they should NOT be able to permanently delete at PR just because it's inconvenient or shows their true face.

My suggestion would be to have a (very) limited timeframe to delete a PR. Maybe a week in case of low activity (e.g. a spam PR the maintainers did not see), much less in case of high activity. Having an exception for "quiet" PRs (no meaningful amount of comments) and allowing to delete those for longer would be perfectly fine of course so someone who rarely checks their repo can delete spam even when they notice it late.

Thoughts on having this be an org-level or repo-level setting so admins can determine when deletion can be done? I know there are also certain repos/orgs where PRs can't be deleted for legal reasons.

[ThiefMaster]

I think just being able to view would be more than enough.

For private repos I would not limit this at all by default, or make it configurable. My comment was mostly about public repos where I don't think the maintainer should have the ability to make a PR and its associated discussion simply disappear for good, e.g. because they do not like how that discussion went.

[Mossaka]

Hey! I am from Azure Core Upstream and we have a lot of OSS maintainers who mainly maintain repositories on GitHub. We held an internal session to talk about copilot and there is a discussion on the topic where maintainers feel caught between today’s required review rigor (line-by-line understanding for anything shipped) and a future where agentic / AI-generated code makes that model increasingly unsustainable.

below are some key maintainer's pain points:

1. Review trust model is broken: reviewers can no longer assume authors understand or wrote the code they submit.
2. AI-generated PRs can look structurally “fine” but be logically wrong, unsafe, or interact with systems the reviewer doesn’t fully know.
3. Line-by-line review is still mandatory for shipped code, but does not scale with large AI-assisted or agentic PRs.
4. Maintainers are uncomfortable approving PRs they don’t fully understand, yet AI makes it easy to submit large changes without deep understanding.
5. Increased cognitive load: reviewers must now evaluate both the code and whether the author understands it.
6. Review burden is higher than pre-AI, not lower.

[moraesc]

Thanks for sharing @Mossaka ! Great to get more feedback. This looks along the lines of what we've been hearing from maintainers too, so validates what some of the key pain points are. Were there any feature requests mentioned during the session?

[Tijjani12345]

tijjaniyahay99@gmail.com
My flie

[Tibor17]

1. I have a problem with the following statement. It is very leftish. The GitHub has no rights to impose such limitations. You have to configure your repo and TTL of a PR:
   limited timeframe to delete a PR. Maybe a week in case of low activity

2. Regarding the next statement, you can delete a PR when you close it. How different is close and delete?
   Next question, why you mentioned low-quality PRs? I hope you do not want to employ AI to determine what low quality means! We, maintainers, have to determine it.
   The ability to delete a PR from the UI - This provides maintainers with the ability to remove spam or low-quality PRs directly from the interface to improve repository organization.

3. Who is an external contributor?
   Example, you mean a situation when XLibre developers are removed from Xorg?
   Restrict PRs to collaborators - This provides more granular access control, allowing contributions exclusively from existing collaborators while blocking external contributors.

4. This would segregate the maintainers and we don't want it. We have flags on the PRs to indicate what to do with them.
   If the AI is leveraged to this job, we have segregated the maintainers again in the Open Source. But the rule is that we are all equal, and this might be a problem if one is notified and the second guy is not. You may not totally trust the AI due to finally the result may change without the second guy or the quality would be bad without his opinion since he does not attend the review.
   Improved triage tools - Potentially leveraging AI to evaluate contributions against project guidelines and standards to identify which contributions maintainers should focus on reviewing.

[ThiefMaster]

It is very leftish. The GitHub has no rights to impose such limitations. Y

Disagree. For the sake of openness and transparency, completely nuking a PR is not a good thing. If there are good reasons one could always contact support...

Also WTF does it have to do with "leftish"?

[sirosen]

An option to limit new contributors to one open PR would be nice.

Just today I had to batch-close several AI generated PRs which were all submitted around the same time.

For this protection, defining "new contributor" is probably not possible to do perfectly. But anyone who has no interactions with a project prior to the last 48 hours seems like a good heuristic. The point is to catch such a user at submission time and limit the amount of maintainer attention they can take up.

For a different type of problem, I'd like to be able to choose PRs as "abandoned", similar to the issue close statuses. It's a clear UI signal to the contributor that their work isn't being rejected but I'm not going to finish it for them. Several of the low quality contributions I have handled, dating back to before the Slop Era but getting worse, are simply incomplete and need follow through.

[dgoldman0]

For the long term horizon: Implement a reviewer LLM that first does an initial scoring of the PRs? Critique is far easier than creation of a correct result. That automated pre-moderation should give the edge needed to handle. Depending on whether you just use rich prompting or fine-tuning, you can even start building an "oracle vox" for your project, which acts as a reasonably informed, reasonably on point virtual representative for the project/organization.

[ThiefMaster]

No thanks. I don't want AI reviews in my projects.

[dgoldman0]

Your loss. Just means that you'll be limited to smaller scope projects, your project will require more labor, and most likely your projects will be left behind as other builders adopt this technology, refine it, and learn how to leverage it effectively.

[ThiefMaster]

I read your comment in a perfect snake oil salesman voice :)

[dgoldman0]

That is your own delusions and paranoia speaking.

[sirosen]

I think you do not understand why LLM generated PRs are harmful to maintainers.

Review is time consuming labor that generally requires a higher level of expertise than contributing code. When I see a 10K SLOC commit which purports to fix 12 outstanding issues, it's unreviewable. I can't responsibly accept it even if every line is correct -- because I just have no way to assess it. Turning up the volume of content I need to review by adding a public chat between the contributor and some GitHub-run agent makes the information overload that much worse.

And yes, I would need to read that whole chat, because I guarantee that there will be hallucinations in there and misinformation I need to correct. Adding Mr. Flattery The Always Wrong Bot to every conversation is not going to help.

I don't want more. I want less, but better.