Relax AI policy - #641
Conversation
AdrianVovk
left a comment
AdrianVovk
left a comment
There was a problem hiding this comment.
Suggestion to make it clearer that the reviewers have ultimate authority to determine what is slop, and that there's no room to argue with them
travier
left a comment
travier
left a comment
There was a problem hiding this comment.
I agree with Adrian's suggestions as well
|
cc @CodedOre as github doesn't want me to mark you as a reviewer |
|
So I see this is draft yet a bunch of people commenting on it. Is this going to be merged soon? If so what is the hurry here? As I see it, it reintroduces room for disagreements and would waste our time again and the slop submissions fest is still ongoing, so seems too soon to me. I said in the issue to wait. It's also problematic if we change a policy every month. |
|
There's no hurry but the direct AI ban caused downstream waves across communities dependent on Flathub. So yes, I would like to arrive at something we're fine with this month.
We never formally announced the change in the first place, unless my Mastodon account counts, so I don't think it's a big deal. |
Where is that? It'd be useful to know who has been blocked from what and which downstream has been affected in what way to know the motivation. |
|
No one was blocked from anything – but some people in GNOME circles are unhappy as they claim the policy to be too explicit (even if the interlinear tension states the intent is "no slop, legit projects with some LLM usage probably fine"), affecting their Circles program, with similar concerns raised within Ublue, questioning if explicit ban isn't pushing flatpak/flathub into irrelevance. All discussions happening in the hallway track. |
Yes that would be what I want to know. What is something that got blocked by this? I'd like to know if that would have been accepted otherwise or would have failed to clear the bar anyways.
The submission queue disagrees with that clearly and I don't think changing policy to (and/or) fight to stay relevant is a correct motivation for this. I think what would work is if there was something legit that got blocked. |
A potential Circle submission is believed not to be free from AI-generated code, and it has to be submitted to Flathub for Circle inclusion. That's as specific I can get.
It's also about what didn't get submitted at all because authors assumed their LLM usage will prevent publishing. Random examples: https://lemonade-sdk.github.io/flatpak/ or whatever appears developer-approved in https://flatpark.org/. We don't have a magic wand to say who else skipped Flathub. |
As far as I can see they don't allow AI generated code. The "other indications of AI-generated output" line falls under "having AI-generated code" to me.
Lemonade doesn't work as an example as Claude is the number one committer in that repo surpassing humans at ~130 commits and ~90k lines changed.
That's 5 out of 8 developer approved apps who will not benefit from the policy in this PR as they are very significantly AI generated and don't fall under "minor uses". Ironically what would have helped it was the original text I had last year or having nothing at all. I am struggling to find a proper justification here. The last time it was meant to help the reviewers. Now it's getting partially undone within a month so not helping the reviewers fully any more. Is the current wording meant to help any of the above? If so then it doesn't reflect that and #641 (comment) holds. |
|
I don't think this is about significantly AI generated apps, but more about applications where the maintainer may have accepted some AI generated code and does not feel confident saying in a review that there is no AI generated code in their application, thus they don't submit it. I am not concerned about massively AI generated apps going to another remote. Good for them. But for applications where AI generated content is marginal or unspecified, a complete ban hurts. To be clear, I was initially fully on-board with the general ban, but I've changed my mind following the discussions. I still think (as this updated policy still says) that all submission PRs and all issue & PR interactions should NOT be AI generated at all. |
The only thing I can think of from my inbox was aaif-goose/goose#6602 (comment) but I don't have made up my mind if we would want that app |
You asked for examples so I provided them. Are you expecting me to spend time asking around social media to learn who decided not to submit their app at all? I'm only proving the point of you not knowing what could have been submitted.
The goal, whatever the final wording will be, is to keep the door open for LLM-assisted apps while giving reviewers a simple way of rejecting obvious slop. It's not what the pre-ban version accomplished either. |
Deploying documentation with
|
| Latest commit: |
e892c27
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://deeac4d5.documentation-962.pages.dev |
| Branch Preview URL: | https://ai-disclosure.documentation-962.pages.dev |
CodedOre
left a comment
CodedOre
left a comment
There was a problem hiding this comment.
I have my issue with the following sentence:
The use of generative AI, including substantial use, is not itself a policy violation.
This is way too much an invitation for slop creators to argue why their slop should be fine under that policy.
| This policy applies to both the application being submitted to Flathub and the | ||
| Flathub submission itself, including the manifest, metadata, patches, build | ||
| scripts, pull request description, and review interactions. For the purpose of | ||
| this policy, "applications" include Flatpak apps, BaseApps, extensions, | ||
| runtimes and any other artifacts that can be produced by flatpak-builder. | ||
|
|
||
| Submitters must disclose any AI-generated code, documentation, or other content | ||
| they know is included in the application. The disclosure must say which parts | ||
| are affected and roughly how much was generated. | ||
|
|
||
| Using AI for research, discussion or debugging does not need to be disclosed if | ||
| no generated code or content was added to the application. | ||
| Minor uses such as grammar or formatting fixes and small amounts of common | ||
| boilerplate are generally acceptable. AI use is substantial when it creates | ||
| core features, determines the main design of the application, or accounts for | ||
| a large part of the project's code or development history. | ||
|
|
||
| The Flathub reviewers may request additional information and will decide | ||
| whether the disclosed use is acceptable. |
There was a problem hiding this comment.
Just an idea for rewording this:
| This policy applies to both the application being submitted to Flathub and the | |
| Flathub submission itself, including the manifest, metadata, patches, build | |
| scripts, pull request description, and review interactions. For the purpose of | |
| this policy, "applications" include Flatpak apps, BaseApps, extensions, | |
| runtimes and any other artifacts that can be produced by flatpak-builder. | |
| Submitters must disclose any AI-generated code, documentation, or other content | |
| they know is included in the application. The disclosure must say which parts | |
| are affected and roughly how much was generated. | |
| Using AI for research, discussion or debugging does not need to be disclosed if | |
| no generated code or content was added to the application. | |
| Minor uses such as grammar or formatting fixes and small amounts of common | |
| boilerplate are generally acceptable. AI use is substantial when it creates | |
| core features, determines the main design of the application, or accounts for | |
| a large part of the project's code or development history. | |
| The Flathub reviewers may request additional information and will decide | |
| whether the disclosed use is acceptable. | |
| The use of AI for research, discussion or | |
| debugging does not need to be disclosed if | |
| no generated code or content was added to | |
| the application. | |
| Minor uses, such as grammar or formatting | |
| fixes and small amounts of common boilerplate | |
| are generally acceptable. | |
| Larger uses, such as generating large parts | |
| of the code or using AI in the software design | |
| process are considered substantial use and are | |
| subject to an case-by-case determination by | |
| the reviewer, who may or may not decide the | |
| use to be acceptable. | |
| "Vibe-coded" application which were mostly or | |
| fully created by AI will not be accepted. | |
| The Flathub reviewers may request additional | |
| information and will decide whether the | |
| disclosed use is acceptable. |
There was a problem hiding this comment.
With this one every vibecode dev will say "mine is not, i reviewed it" and argue.
it also goes the relaxing from "no AI" to "It's possibly ok we need to check", which may as well be a giant "slop welcome" sign
There was a problem hiding this comment.
AI can do good contributions if the instructions are clear. I use AI and members of flathub does as well.
It was already discussed and the consensus is that a simple ban to anything that was made with help of AI is unfair.
There was a problem hiding this comment.
even if it did technically the best contributions ever, completely allowing it is not acceptable to communities.
it was not already discussed. A few people gave their two cents once on a matrix channel.
The current draft is fine to me. Blanket allowing AI is not.
What would be good here is a middle ground, and this is it. Do not push things in a binary yes or no.
cassidyjames
left a comment
cassidyjames
left a comment
There was a problem hiding this comment.
Strong approve from me; it feels like a realistic and well-worded update. Regardless of how I personally feel about LLMs, blanket banning all LLM use leads to either:
- Developers lying about their submissions anyway, and/or
- Reviewers somewhat arbitrarily determining what is AI and what isn't
(In the worst case, it could push developers towards proprietary licenses/private repos to avoid disclosing their source code and commit history…)
I don't think that's the outcome anyone wants, so I prefer this reframing it as disclosure versus banning; it's similar to our approach with proprietary licenses as well where we have clear preferences towards FOSS as a community and project, but don't strictly blanket disallow non-free software, either—we require accurate licensing details and can take action based on that.
👍🏻
9 participants
see #620 for earlier discussion
This also implies a new checkbox in the submission form for AI usage disclosure. Long term, we're going to switch to a separate submission web service with canned responses and less human touch, as I don't find the current approach sustainable for anyone.