Repository navigation
Add local webapp and content-repurposer directories - #3
Merged
Merged
Conversation
Contributor
Author
|
Add local webapp and content-repurposer directories |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adding a perturbation class
A class needs four things. All four, or it doesn't go in.
A function in
src/ssb/perturb.py, registered with@register(name, family, description). It takes(text: str, seed: int)and returns a list ofvariant strings. It must be a pure function of its inputs: no clocks, no
randomness that isn't derived from
seedvia_seeded(). Reproducibility isthe point of the project.
A family assignment.
BENIGNfor variation that arises innocently,ADVERSARIALfor variation introduced deliberately to defeat matching,DEGRADEDfor variation introduced by an upstream system. This is the mostimportant decision you'll make, because it determines how a low score on your
class should be read.
A test in
tests/test_ssb.py. At minimum: that it never returns the inputunchanged, and that it produces what you claim for a representative input.
A paragraph in
docs/METHODOLOGY.mdexplaining what real-world phenomenonit models. Not what it does, what it models. A class nobody can justify is
noise in every scorecard that includes it.
Return an empty list rather than raising when a class doesn't apply to an input.
Most classes are conditional.
Where help is most needed
Non-Latin script coverage. Currently the benchmark generates Latin-script
variants of names that are themselves already romanised. Screening against native
script is a separate and harder problem and is not represented at all.
Entity and vessel names. The perturbation classes were designed for personal
names. Corporate names have their own variation grammar: legal-form suffixes,
abbreviation, translation, transliteration of the trading name but not the legal
name. None of it is modelled.
Alias structure. OFAC alias records carry type information (a.k.a., f.k.a.,
strong versus weak) that could sharpen both positive generation and scoring. It is
currently ignored.
Additional list sources. EU consolidated, UN, UK OFSI. The parsing layer in
src/ssb/sources.pyis structured to take them; nobody has written them.Better negative controls. The three synthetic classes are a floor, not a
ceiling. A more realistic negative distribution would materially improve what the
precision figure means.
Things that will get a change rejected
means anyone can run this on a locked-down corporate machine, which is exactly
where screening people work.
docs/LICENSING.md.baseline, and an honest one is the only kind worth having.
Reporting a result
If you run the benchmark against a commercial engine, you're welcome to open an
issue with the class-wise profile. Please do not name the vendor unless your
agreement with them clearly permits it. An anonymised profile, for example
"Engine A, commercial, deployed at a mid-size US bank", is more useful than
nothing and avoids putting you in a difficult position.
Tests
python3 -m unittest discover -s tests -t .Everything runs offline against the synthetic fixture. No network, no accounts, no
API keys.