[policy] Good-faith agentic contributions and LLM use, and avoiding death by a thousand AI slops

[webknjaz]

I want to research including an "ai policy" across projects I'm involved in and I'm tracking some references to relevant documents and discussions @ https://gist.github.com/webknjaz/10f1106d0a3fd489745c2bc656400d1f. I've fed that gist into Claude and let it traverse like 357 sources out of that.

Here's what it came up with: https://claude.ai/public/artifacts/de2ffa40-98dc-4458-b1b5-f3e645bf7840. That document has some summaries and hightlights, principles/observations distilled, a policy draft, integration suggestions and guidelines for coping with spam.

I think it did a good job illuminating what's beeing observed across FOSS and showing how different parties react, why a clear/transparent document is even necessary. Let's talk about putting one up for pip-tools.

cc @sirosen @hugovk

[sirosen]

I have an overall much more negative posture towards the LLM tools than you do. I feel no hostility towards the users, but the technology is not being developed or promoted in a responsible way and we're seeing some of the consequences.
I have two separate areas to address below. First, as background info and sharing, why I think a ban is not viable. Second, what I think our policy should be.

My post here will barely touch on the ethical or legal issues. I have genuine concerns on these fronts, but I think any attempt to discuss them here will derail us.

Banning the tools is incorrect (for us)

I'm not concerned that a ban might drive some contributors away. Bluntly put, anyone who can't contribute without the help of an LLM probably shouldn't be trying to make changes in pip-tools. Even with a decent level of domain expertise, I've made some mistakes, and I don't think the problem space is in any way welcoming for a new developer.

But I don't believe that a policy banning LLM-assisted contributions (as, e.g., Gentoo has done) is appropriate. As much as I admire the clarity of this posture, I think this turns the project into a place I have to police for "bad contributions" which I then block/ban. I don't want to engage with the project in this way, and we haven't had a severe enough flood of slop on our project¹ to drive me towards supporting such a ban.

I think we'll still see some level of LLM-generated issue reports and PRs even with a ban. It then turns any further interaction negative, since the appropriate action is to close the issue without further evaluation.

I'm also not that keen on the idea of taking options away from contributors. If someone finds a useful way to leverage local models (which reduce many of the ethical concerns) to do their work, and their contributions are of high quality, why should I care to forbid them from using their preferred toolchain?²

Proposed policy: you are responsible for your work

I'd like us to put together a policy which is generic -- spans LLMs and any other current or future assistive tools -- and emphasizes the following two main aspects:

1. Disclosure. If you use a tool to do something, tell us what you did. If you wrote a libcst script to refactor some things, I'd expect you to share the script. By a similar line of thinking, "I wrote this issue in Turkish and used an LLM to translate it." at the end of an issue would be not only okay but good! We need to know what we are reviewing.

2. Ownership. You, not the LLM, are the owner of your code contribution. Maintainers should be able to ask you about any given line you are changing and you must be prepared to discuss it. You don't need mastery of the codebase, and it's okay if a part of your changeset is a guess at what's correct. But as the contributor you are responsible for your work.

The above is not quite a draft of the exact policy, but that's the main thrust of it. What do you think about it as a starting point? Are there major, important elements of good contributions which this misses?

I saw Claude suggested some bullet points aligned with the Ownership element. We might adapt some such list of affirmations into our PR template, but I don't think that's the core issue. I also see that it also used the word Disclosure, but defined it narrowly to LLMs. I'd rather we define it broadly -- partly, I think this sends a good social signal that LLM users aren't being singled out and penalized. Contributors who might be scared off by an overtly hostile policy will ideally find such a definition more welcoming.

---

I find it funny and somewhat topical that while skimming that document, I spotted an error which @webknjaz would be very unlikely to make:

From curl's Daniel Stenberg: [quotation follows]
From SymPy's Oscar Benjamin: [quotation follows]
From Python's Seth Larson: [quotation follows]
From Flask's Miguel Grinberg: [quotation follows]

This makes it sound like Miguel Grinberg is a Flask maintainer. Miguel's tutorials and books have done a lot to advance Flask, but he's not (and as far as I know, has never been) a maintainer. (I also find "Python's Seth Larson" to be odd phrasing for his role, but 🤷 on that)

Even the highest quality tool (Claude is well regarded) in the hands of a domain expert produces errors. I'm sure a deeper reading would reveal other errors.

Footnotes

1. 🤞 we've actually had hardly any 🤞 ↩

2. Possible answer: legal/copyright issues. I do not find this persuasive at all. If you hold that a large model and its outputs are a "derived work", then every output is GPL licensed (at the very least). It makes perfect sense if you look at licensing and legal systems as a programmer, "programming" in the legal system. Anyone who favors use of GPL/LGPL takes this view to some degree. But to actually enforce such copyright on all consumers of LLM outputs requires a level of collective political will which I cannot fathom. ↩

[webknjaz]

Agreed, I'm also against a ban. It's not enforceable and not 100% detectable anyway. Plus we have better things to do than hunting down offenders..

---

I also liked the disclosure and ownership bits. Especially the ownership bit — one should be able to reason about their patch coherently regardless of the tooling in use or not.

Regarding disclosure, I agree as well -- showing the prompts and stuff is exactly like showing "this is how I googled it / found on stackoverflow / reached the conclusion step-by-step". I've been thinking for a while that the issue forms should ask about the LLM in use and the prompts. Haven't implemented it anywhere, though. I know that some people suggest using https://aiattribution.github.io but that seems a bit too formal so I wouldn't insist. A generic disclosure sounds good. I also saw a mention that might've not made it into the document that one is responsible for ensuring their work doesn't violate copyright.

And those "Don't let AI speak for you" / "Don't let AI think for you" seemed relatable.

---

I didn't challange the quote author titles because that's a lot of work and I was mostly interested in the ideas at the end. The maintainer claims sounded weird but I don't know who davidism works with these days. Seth is mentioned in connection to PSF earlier but then it just "forgot" a part of the abbreviation, it seems. I also think that Claude's parser got rate-limited at some point and was unable to fetch the CPython's policy so it ended up claiming that there's none.

Note that this is the initial document generated in response to the very first prompt. I didn't iterate on it so there's no mark of a "domain expert" in it whatsoever ­— only this prompt (plus initial prompts in Claude account + project + crawling the web) contributed to the output.

---

I'm also interested in those maintainer-focused suggestions — quick red flag identification + response templates. Plus the contrib checklist looks reasonable.

[webknjaz]

Zulip shared (https://github.com/community/maintainers/discussions/646#discussioncomment-15065464) that they've also added an ask to verify accuracy: zulip/zulip#36744

[sirosen]

I'm not able to access github.com/community/maintainers (I just requested access; I expect it will be approved), but I see the link on the zulip project.

---

Regarding any linkage from issue forms or the PR template, I think let's get the contributing doc updated first, and then we can decide on how we want to link it. Similar to how, per the thread below, we may add AGENTS.md/CLAUDE.md after we get the policy set.

---

At this stage I have read enough that I'm getting diminishing returns from reading more of our options and ways that others are handling this. I think that the next step is to try to draft a policy, and we can discuss how to modify it. I can put some time into this in the next few days (although I want to finish reviewing the CI changes first! 😁 ). Would you like to do that in a thread here, or in a draft PR?

[webknjaz]

A draft PR sounds good, thanks!

[sirosen]

I'm not sure how to incorporate this, but I was recently shown this project which has a hard no-AI policy:
https://github.com/grishka/Smithereen/blob/49fdc3fb1653e4259eb66742dfcef8bcbc500c09/CLAUDE.md

Claude will apparently respect this, and refuses to help a user circumvent this instruction.

Per above, I don't think such a policy is appropriate for us, but it shows that a pretty short prompt can strongly influence the behavior (perhaps specifically if it's in the branded CLAUDE.md?).
I'd rather we stick to one of the evolving standards like llms.txt if we do something like this, but I think we should think about how we prompt agents to look at our policy doc in addition to the policy itself.

[webknjaz]

I'm not sure how to incorporate this, but I was recently shown this project which has a hard no-AI policy: https://github.com/grishka/Smithereen/blob/49fdc3fb1653e4259eb66742dfcef8bcbc500c09/CLAUDE.md

Interesting, although, I don't like the source, obviously.

Claude will apparently respect this, and refuses to help a user circumvent this instruction.

Maybe to an extent. One could probably hack this with a user-global memory file, I imagine.

Per above, I don't think such a policy is appropriate for us, but it shows that a pretty short prompt can strongly influence the behavior (perhaps specifically if it's in the branded CLAUDE.md?).

Yes, this is the default memory file that claude uses: https://code.claude.com/docs/en/memory. There's also CLAUDE.local.md that's beeing deprecated in favor of sourcing other files thought @-syntax. This one would be gitignored, normally. And there's ~/.claude/CLAUDE.md living outside the projects.

I'd rather we stick to one of the evolving standards like llms.txt if we do something like this

That thing is for the docs. We should stick https://sphinx-llms-txt.rtfd.io into our Sphinx env. But there's not much to do otherwise. Would be useful for someone scanning our docs with LLMs or training LLMs.

I think we should think about how we prompt agents to look at our policy doc in addition to the policy itself.

Yes, we should link it. There's AGENTS.md (although, their frontpage lies about the adoption stats) that's meant to be more universal than per-tool configs. Many products support it out of the box. At Ansible, we've gone for a trick of putting main instructions in that one and soucring it from CLAUDE.md: https://github.com/ansible/ansible/pull/85841/files#diff-6ebdb617a8104a7756d0cf36578ab01103dc9f07e4dc6feb751296b9c402faf7.

We've actually implemented tree-local and project-global files through sourcing.
Basically, ~/.claude/ansible.md may contain instructions that individuals would have, for personalized workflows. But also, there can be a CLAUDE.local.md in-tree, that's gitignored. I suggested it so that when there's multiple git worktrees for various efforts, it'd be possible to store some context in those files locally, per-PR/branch.

We should have something like this here but I wanted to address the policy question first.

[sirosen]

Interesting, although, I don't like the source, obviously.

Ah, I hadn't looked closely before, sorry! ❤️

---

You're right, I mixed up llms.txt and AGENTS.md. Maybe we want to add an llms.txt, but I was thinking more about the AGENTS.md part of the story, which seemed like a separate thread from "what should our policy be".

We can evolve it more, but I think we can start an initial AGENTS.md which reads to the effect of...

This is a project which has concerns about the impact of LLM-driven contributions. The programmer, not the LLM, is responsible for their work, and pip-tools has a clear policy regarding this. Please make sure to read and direct contributors to <this policy doc>.

Happy to iterate more on the wording. 😄

Also, I agree that actually setting our policy takes priority.

[sirosen]

I haven't given this much thought lately, but we just got a batch of PRs which are almost certainly slop (#2311, #2312, #2313, #2314, #2315, #2316). I bulk closed them with a fixed comment asking the contributor to select one topic to work on, rather than slamming us with PRs.

We've also been getting a few slop-y docs PRs which I've been coaching through to being acceptable, though I don't really like it.

Overall, I think the urgency of this issue is rising. I'll try to refocus and make an initial effort on it.

[webknjaz]

Looks like I got a positive example of a coachable contributor with an LLM in their hands, FTR:

• doc: fix ~75 typos and grammar issues across documentation pytest-dev/pytest#14179
• docs: fix typos and grammar in README and SECURITY pypa/gh-action-pypi-publish#395
• doc: indent envvar bodies and add src-layout link pytest-dev/pytest#14181
• doc: indent envvar directive bodies in reference.rst pytest-dev/pytest#14183
• doc: add seealso link to src-layout vs flat-layout discussion pytest-dev/pytest#14184
• doc: add example for the envvar directive sphinx-doc/sphinx#14295
• Style semantic markup roles pradyunsg/furo#924 (<-- Style the `:command:` role and similar pradyunsg/furo#890 (comment))

[sirosen]

Interesting idea from the JS ecosystem: asking contributors to add an emoji at the end of their PRs to indicate that they really did read the contrib doc.
See: https://typescript-eslint.io/contributing/pull-requests/#asking-questions

This lets the maintainers really quickly see whether or not the contributor did the expected reading.

I've been worried with this whole effort that few people -- especially new contributors -- read the contributing guidelines.
I'm not sure I'd do it with an emoji, but maybe it's good because it's low effort for contributors.

[webknjaz]

I think an emoji idea is nice. We could also ask the contributors to include a motivation paragraph in their first contrib, telling us why they've decided to contribute and how they envision being involved continously / over time. I'd also suggest people to self-advertise and show how they've been involved in the ecosystem / other projects.

[webknjaz]

Looks like somebody took a stab at visualizing the policy landscape: https://redmonk.com/kholterhoff/2026/02/26/generative-ai-policy-landscape-in-open-source/

[webknjaz]

Do we need to address these? https://simonwillison.net/2026/Mar/5/chardet/ / chardet/chardet#334 / chardet/chardet#327 (comment) / https://www.copyright.gov/ai/ai_policy_guidance.pdf

People on the internet seem concerned:

[sirosen]

I am genuinely concerned about copyright and copyright laundering, as a topic. I don't know what the various court systems will decide, which complicates matters, but I would like to formulate a principled stance as distinct from "what US law says".

With chardet, it's LGPL and was almost definitely part of the training set for the models used. I assume that the author acted in good faith, but my current opinion is that the rewrite is a derived work and can't be sufficiently strongly proven otherwise.

Right now, for a little while at least, I'm not that worried about my various projects and their contributors. If we got a massive vibe coded PR where I felt that copyright were a serious question, that might be grounds for rejecting the work, but I don't (yet) feel the need to act on this.

Eventually, every project might need a CLA that contributors sign. That feels like a pain but it's less ambiguous.

[webknjaz]

Yeah, that was my thinking. Even Richard Fontana said we might have to perform analysis (as it's still unclear how it affects downstream).

As for the CLA, it's not liked in FOSS because it's mostly used by the big corps that have it to be able to relicense contribs on a whim, it appears... Aside from corner cases like the PSF's needs, if course.

We may want to consider DCO though, right?

[sirosen]

I'm more familiar with the CLA process -- you go out and sign a document separate from your contribution.

Is the norm with DCO that you include a verbatim copy of it with your patches? It seems like it would be very annoying to put a copy of it in every commit message. I need to read up on this to learn how it works in practice.

[webknjaz]

The Signed-Off-By trailer in commits — there's a toggle for GH to paste one when you're making commits from the web. And there's

-s, --[no-]signoff    add a Signed-off-by trailer

in the git commit command that adds it for you. I think there should be a config setting too.

[webknjaz]

https://wiki.linuxfoundation.org/dco

[webknjaz]

I don't remember if I documented this but I've seen a couple of mentions that the "AI" tooling tends to lead to the complexity increase.

• https://hbr.org/2026/02/ai-doesnt-reduce-work-it-intensifies-it
• https://arxiv.org/pdf/2511.04427

I imagine this would translate into increased cognitive load. So we should emphasize that patches should simplify the code base and be review-optimized, right? Is there a better way to phrase this? The "less is more" note we have may be difficult to understand for less experienced folks, but maybe that's fine? Should we maybe have a separate review process document? I'd also like to get to files that different tools load as instructions at some point. Those would have to be extremely verbose, I think.

[sirosen]

Is there a better way to phrase this? The "less is more" note we have may be difficult to understand for less experienced folks, but maybe that's fine?

I think it's fine. Or maybe more accurately, I don't think it's a problem which can be fixed with more explanatory text.

New developers should not be expected to have good taste. For example, to know instinctively when "copy paste replace" is the best solution to a problem and when it's the worst one. I don't think this is an insult to new programmers and it's not meant to be gate-keeping; it's just the truth that years of experience can actually make you better at the craft and it's hard to distill down into a style guide or contrib doc.

We could try to jot down something. My first attempt, off the cuff:

Remember! Complexity for maintainers is bad. And complexity for users is even worse! Try to make things simple to understand, which usually means having fewer distinct codepaths, branch points, and special-case behavior. There should be explainable principles and abstractions which underpin specific tool behaviors.

But I don't really expect anyone inexperienced to be able to interpret and make good use of such advice.

An LLM will happily treat each bug report as a new special case, and pile up spaghetti code fixes (each of which, in isolation, is probably fine) with no underlying organizing principles.
I don't expect anyone without some experience to be able to spot the absence of important abstractions -- it's hard to see what's missing when you don't have a lot of practice yet.

---

I'm not very enthusiastic about trying to guide new people through this kind of decision making in the contrib docs because I don't think it will really work. They'll either get confused and not bother trying, or try but not be able to follow the advice anyway. I think there may be more productive ways to mentor people.

One thing I'm considering is adding an explicit note that if an issue has ambiguities, requires research, or has been open a very long time, a PR might not be the appropriate next step. Contributors should consider that discussing issues and working out the solution with maintainers in issues/discussions can be of much greater value than a PR -- and doing so can setup a future PR for success.

Having discussions prior to PRs being opened offers some good opportunities to coach people, share knowledge in both directions, and set expectations.

[webknjaz]

I've been noticing a bunch of spam posted by https://github.com/artvepa80 as issues (not PRs) where they'd run a tool with "ai" in the name, post a link to it (advertisement purposes?) and then admit they didn't verify the generated results. Notabely, the spam text claims that "ai"/LLM wasn't used for that. So far, all of their yesterday's issues have been closed either by themselves, or my the maintainers of the respective projects (in ansible, by our bot, in aiohttp by Sam, in flask by David, langchain has their own template checker).

@sirosen I wanted to document one of the automated auto-close solutions for future exploration: https://github.com/langchain-ai/langchain/blob/8182d6302dc81bc62849f9aa88ff698489b0e665/.github/workflows/close_unchecked_issues.yml

[webknjaz]

Example of somebody admitting to posting ai things prematurely but still working to right the wrongs, performing authentic analysis: llvm/llvm-project#192275 (comment). I think this is a positive contributor example FTR.

[webknjaz]

Can't remember where I posted this already, but I'd like to try out https://github.com/marketplace/actions/good-egg-trust-scoring-prs FTR.

[webknjaz]

Langchain has an interesting approach where they require issues to be assigned to external contributors and linked to PRs: langchain-ai/langchain#36757 (comment).
I imagine this makes sure people actually talk to the maintainers and agree on things before starting.

[webknjaz]

@MaxGraey shared this today: AssemblyScript/assemblyscript@c91247d

[webknjaz]

TODO: port aio-libs/.github#49 back into our repo.

[webknjaz]

Recent related post from the Fedora land: https://jwheel.org/blog/2026/05/fedora-ai-assisted-contributions-policy/

[webknjaz]

Just heard of https://docs.kernel.org/process/coding-assistants.html#attribution from @MaxymVlasov.

[MaxymVlasov]

Example usage: MaxymVlasov/dive-action#133

AGENTS.md

## AI attribution in git

- **Detect model** from the system prompt — it is stated as "You are powered by the model named X". Use X as `<model-name>`. NEVER hardcode or copy a model name from an example.
- **Detect agent name** from your own identity as declared in the system prompt (e.g. "Your designated identity for this session is Sisyphus"). Use that identity as `<agent-name>`. NEVER use "Claude" — that is the underlying AI, not the agent identity. If no identity is declared, fall back to "Claude".
- **Commit messages**: include `Assisted-by: <agent-name>:<model-name> <cli>` trailer
- **PR descriptions**: include `Assisted-by: <agent-name>:<model-name> <cli>` at the end of the body
- **PR review responses**: prepend every reply with `<sup>Generated by <agent-name>:<model-name> <cli></sup>`
- **Issues created** (Jira or GitHub): append `\n\n---\n_Generated by <agent-name>:<model-name> <cli>_` at the end of the issue body
- **Comments posted** (Jira or GitHub): prepend every comment with `<sup>Generated by <agent-name>:<model-name> <cli></sup>\n\n`
- **Detect CLI** from env vars before writing attribution:
  - `OPENCODE=1` → `opencode`
  - `CLAUDE_CODE_SSE_PORT` set → `claude`
  - None matched → omit the `<cli>` part

[webknjaz]

Ansible's AI policy discussion: ansible/ansible-documentation#3588 / https://docs.ansible.com/projects/ansible/latest/community/ai_policy.html

[webknjaz]

Hilarious sloperator metaphor shared by @MaxGraey just today: https://not-an-llm.bearblog.dev/meat-based-llm-proxies/

[webknjaz]

New CPython policy PR: https://github.com/python/devguide/pull/1778/changes/BASE..2aad5605297ee70e0f9f0efde3cfb5f801cfff13#diff-8907985fff9f18341e220efa9d9d1e6b76b34ca4e84bf59702b85b5f23197d4e (in progress).

Now live: https://devguide.python.org/getting-started/ai-tools/

[sirosen]

This revision is pretty interesting to me. Especially the rephrasing of why slop PRs may be closed without discussion -- that it's about any low quality PRs now. I'll try to keep an eye on this as it progresses.

[webknjaz]

Yeah, and it's live already.

[webknjaz]

https://nesbitt.io/2026/05/21/rfc-artificial-contributors-to-open-source.html

[webknjaz]

The explanatory websites are starting to appear: https://dontquotetheai.com

[webknjaz]

Another good read on the eternal sloptember: https://geohot.github.io/blog/jekyll/update/2026/05/24/the-eternal-sloptember.html

[webknjaz]

TIL new term: Slopware

[webknjaz]

Today's ethics gems:

• Question: intent of JqwikExecutor.printMessageForCodingAgents() — visible to agents, invisible to humans (1.10.0) jqwik-team/jqwik#708
• https://blog.johanneslink.net/2025/11/04/to-gen-or-not-to-gen/

[webknjaz]

@sirosen I feel like a concept of a house rules document might be interesting/useful to implement, check this out: https://github.com/tox-dev/turbohtml/pull/482/changes#diff-1e7b15f526491367007c2999868eabb31f33c6f1e26dbc1ac8b1bfd470e12c35