Update docs

Author: julesjacobsen

docs/advanced_analysis.rst

@@ -22,7 +22,7 @@ These fields define all the input data, analysis steps and output requirements f
 be used as a record of how a sample was analysed. Alternatively the various sections can all be defined at runtime using
 the CLI arguments as explained in the :ref:`inputandoptions` section.
 
-which defines run mode, :ref:`filters<variantfilters>` and :ref:`prioritisers<prioritisers>`, and the :ref:`output options<outputoptions>` section that defines the output format,
+The :ref:`preset<preset>` or :ref:`analysis<analysis>` section define the run mode, :ref:`filters<variantfilters>` and :ref:`prioritisers<prioritisers>`, and the :ref:`output options<outputoptions>` section that defines the output format,
 output file and number of results that should be returned. Each of these sections can be defined independently on the
 command line or provided as a single file as shown in the `job`_ section.
 
@@ -68,9 +68,18 @@ Preset
 
 .. code-block:: yaml
 
-    # one of EXOME or GENOME. GENOME will require REMM to be available. Default is EXOME.
+    # one of EXOME, GENOME or PHENOTYPE_ONLY. GENOME will require REMM to be available. Default is EXOME.
     preset: EXOME
 
+The presets are also provided as a set of analysis YAML files in the ``examples/`` directory:
+
+.. csv-table:: Preset option YAML file equivalents
+    :header: "Preset", "YAML file"
+    :widths: auto
+
+    EXOME          ,  examples/preset-exome-analysis.yml
+    GENOME         ,  examples/preset-genome-analysis.yml
+    PHENOTYPE_ONLY , examples/preset-phenotype-only-analysis.yml
 
 
 .. _analysis:
@@ -278,12 +287,12 @@ the population frequency for the ACMG assignments, even if used in the filtering
 pathogenicitySources:
 ---------------------
 Possible pathogenicitySources: ``POLYPHEN``, ``MUTATION_TASTER``, ``SIFT``, ``REVEL``, ``MVP``, ``ALPHA_MISSENSE``,
-``SPLICE_AI`` (derived from gnomAD 4.0, so only available for hg38),  ``CADD``, ``REMM``. ``REMM`` is trained on
+``SPLICE_AI``,  ``CADD``, ``REMM``. ``REMM`` is trained on
 non-coding regulatory regions. **WARNING** if you enable ``CADD``, ensure that you have downloaded and installed the CADD
-tabix files and updated their location in the ``application.properties`` (see :ref:`cadd-install`). Exomiser will not run
-without this.
+tabix files and updated their location in the ``application.properties`` (see :ref:`cadd-install`). Similarly, REMM
+requires an optional data download (see :ref:`remm`). Exomiser will not run without these if specified.
 
-We recommend using either  ``[REVEL, MVP, ALPHA_MISSENSE]`` **OR** ``[POLYPHEN, MUTATION_TASTER, SIFT]`` as REVEL, MVP
+We recommend using either ``[REVEL, MVP, ALPHA_MISSENSE]`` **OR** ``[POLYPHEN, MUTATION_TASTER, SIFT]`` as REVEL, MVP
 and ALPHA_MISSENSE are newer predictors which have been shown to have better performance and are more nuanced. Mixing
 them with the Polyphen2, MutationTaster or SIFT will give worse performance. Testing on GEL solved cases with
 AlphaMissense slightly increased performance when combined with MVP. We advise testing on local cohorts for assessing
@@ -349,31 +358,78 @@ Removes variants with VCF `QUAL` scores lower than the given `minQuality`.
     qualityFilter: {minQuality: 50.0}
 
 
+alleleBalanceFilter:
+....................
+
+The allele balance AB, also known as the Variant Allele Frequency VAF, filter uses the ratio of reads at a variant locus
+supporting the alternate allele. This is a variant quality metric used, along with the genotype quality score (GQ) and
+allele depth (DP) found in the VCF file. Based on the `Undiagnosed Diseases Network (UDN)
+paper on optimising Exomiser and Genomiser <https://doi.org/10.1186/s13073-025-01546-1>`_, we implemented their
+recommendation of 15%>=VAF<=85% for heterozygous variants and GQ>=20. Additionally we placed a requirement for a DP>=20
+and for mitochondrial variants we used an AB cutoff of >= 0.05 in line with the `Genomics England Tiering pipeline <https://pipeline-rd-help.genomicsengland.co.uk/Lyra/variant-prioritisation-approaches/small-variant-tiering/segregation-with-disease/#mitochondrialgenome>`_.
+
+This filter is set to run as a default, from version 15.0.0, yet users may wish to disable it to perform their own
+pre-filtering as this is a very stringent filter and may remove some low-quality diagnostic variants.
+
+.. code-block:: yaml
+
+    alleleBalanceFilter: {}
+
+
+Refs:
+ 1. `Effective variant filtering and expected candidate variant yield in studies of rare human disease DOI:10.1038/s41525-021-00227-3 <https://doi.org/10.1038/s41525-021-00227-3>`_
+ 2. `Gene prioritisation for enhancing molecular diagnosis in rare skeletal muscle disease cohort DOI:10.1136/jmg-2024-110212 <https://doi.org/10.1136/jmg-2024-110212>`_
+ 3. `An optimized variant prioritization process for rare disease diagnostics: recommendations for Exomiser and Genomiser DOI:10.1186/s13073-025-01546-1 <https://doi.org/10.1186/s13073-025-01546-1>`_
+
+
 intervalFilter:
 ...............
-Defines an interval of interest. Only variants within this interval will be passed. Currently only single intervals are
-possible.
+Defines an interval(s) of interest. Only variants within an interval will be passed.
 
 .. code-block:: yaml
 
     intervalFilter: {interval: 'chr10:123256200-123256300'}
+    # or for multiple intervals:
+    intervalFilter: {intervals: ['chr10:123256200-123256300', 'chr10:123256290-123256350']}
 
 
-geneIdFilter:
+genePanelFilter:
 .............
-You can define `entrez-gene-ids <http://www.ncbi.nlm.nih.gov/gene/>`_ for genes of interest. Only variants associated with
+You can define `HGNC <https://genenames.org>`_ gene symbols for genes of interest. Only variants associated with
 these genes will be analyzed.
 
 .. code-block:: yaml
 
-    geneIdFilter: {geneIds: [12345, 34567, 98765]}
+    genePanelFilter: {geneSymbols: ["FGFR1", "FGFR2"]}
+
+
+geneBlacklistFilter:
+....................
+
+Removes variants from the analysis which are assigned to a set of blacklisted genes.
+
+Blacklisted genes are removed to reduce the noise level. These are 56 pseudogenes, HLA genes, and others that have a
+high degree of variants called in healthy individuals:
+
+.. parsed-literal::
+
+    COL4A2-AS2, CRIPAK, FCGBP, GOLGA6L2, GOLGA8N, HLA-A, HLA-B, HLA-C, HLA-DMA, HLA-DMB, HLA-DOA, HLA-DOB, HLA-DPA1,
+    HLA-DPB1, HLA-DPB2, HLA-DQA1, HLA-DQA2, HLA-DQB1, HLA-DQB1-AS1, HLA-DQB2, HLA-DRA, HLA-DRB1, HLA-DRB5, HLA-DRB6,
+    HLA-E, HLA-F, HLA-F-AS1, HLA-G, HLA-H, HLA-J, HLA-L, KRTAP4-7, KRTAP4-8, KRTAP9-6, LILRA6, LILRB3, LINC02081,
+    LRRC37A2, MUC12, MUC16, MUC17, MUC19, MUC2, MUC20, MUC21, MUC3A, MUC4, MUC6, PDE4DIP, PRAMEF2, PRAMEF9, SIRPA, TBC1D3I,
+    UGT1A7, USP17L1
+
+
+.. code-block:: yaml
+
+    geneBlacklistFilter: {}
 
 
 variantEffectFilter:
 ....................
 If you are interested only in specific functional classes of variants you can define a set of classes you want to remove
-from the output. Variant effects are generated by `Jannovar <http://charite.github.io/jannovar/>`_. Jannovar uses
-`Sequence Ontology (SO) <http://www.sequenceontology.org/>`_ terms and are listed in their `manual <http://jannovar.readthedocs.io/en/master/var_effects.html>`_.
+from the output. Variant effects are generated by `Jannovar <http://charite.github.io/jannovar/>`_ and reported as
+`Sequence Ontology (SO) <http://www.sequenceontology.org/>`_ terms.
 
 .. code-block:: yaml
 
@@ -436,7 +492,13 @@ pathogenicityFilter:
 Will apply the pathogenicity scores defined in the :ref:`pathogenicitySources<pathogenicitysources>` section to variants.
 If the ``keepNonPathogenic`` field is set to ``true`` then all variants will be kept. Setting this to ``false`` will set
 the filter to fail non-missense variants with pathogenicity scores lower than a score cutoff of 0.5.
-This filter is meant to be quite permissive and we recommend it be set to true.
+
+This filter is meant to be quite permissive and we recommend it be set to ``true`` unless running genomiser
+(i.e. including REMM/CADD/SPLICE_AI and all non-coding regions), in which case setting ``keepNonPathogenic: false`` is
+recommended as Exomiser will use a stringent cutoff to remove variants with a `CADD <https://cadd.bihealth.org/info>`_
+raw score <= 15.0 a `REMM score <https://doi.org/10.1093/gigascience/giad024>`_ <= 0.914
+or a `SpliceAI <https://doi.org/10.1016/j.cell.2018.12.015>`_ <= 0.1 These thresholds were chosen based on the
+recommendations of the authors of the scores.
 
 .. code-block:: yaml
 
@@ -538,6 +600,14 @@ protein-protein interaction proximities ``ppi``. e.g. only using human and mouse
 
     hiPhivePrioritiser: {runParams: 'human,mouse'}
 
+For a clinical diagnostic pipeline we advise running HiPhive in human-only mode as this will only prioritise genes with
+known disease associations but there will be no potential for strong mouse model hits if a causative variant lies in a
+novel disease causing gene.
+
+.. code-block:: yaml
+
+    hiPhivePrioritiser: {runParams: 'human'}
+
 
 phenixPrioritiser:
 ..................

docs/conf.py

@@ -55,13 +55,13 @@
 # -- Project information -----------------------------------------------------
 
 project = u'exomiser'
-copyright = u'2024, Jules Jacobsen, Damian Smedley, Peter Robinson'
+copyright = u'2026, Jules Jacobsen, Damian Smedley, Peter Robinson'
 author = u'Jules Jacobsen, Damian Smedley, Peter Robinson'
 
 # The short X.Y version
 version = u'15.0.0'
-genome_data_version = u'2502'
-phenotype_data_version = u'2502'
+genome_data_version = u'2512'
+phenotype_data_version = u'2512'
 # The full version, including alpha/beta/rc tags
 release = version
 

docs/input_files_and_options.rst

@@ -99,7 +99,7 @@ analysis
 
 .. important::
 
-    The exome and genome analyses found in the `test-analysis-exome.yml` and `test-analysis-genome.yml` files are
+    The exome and genome analyses found in the `examples/preset-exome-analysis.yml` and `examples/preset-genome-analysis.yml` files are
     recommended for use in most situations, and removing steps from the analysis is likely to negatively impact
     performance. It is *strongly* recommended to test any changes against the standard setup on the example samples and
     your own solved cases to check the impact of any changes you might want to make. If you want to score all variants
@@ -126,19 +126,20 @@ output
 
 By default Exomiser will write out any result files to the exomiser-cli-|version|/results sub-directory of the
 Exomiser installation directory. Unless specified in the `output.yml` or `outputOptions` section of the analysis YAML
-file, Exomiser will write out a `.json` and a `.html` file. These are for machine (JSON) and human (HTML) use. The
+file, Exomiser will write out a `.jsonl`, a `.parquet` and a `.html` file. These are for machine (JSON) and human (HTML) use. The
 filenames will match the input VCF filename. For example
 
 .. parsed-literal::
 
     java -jar exomiser-cli-|version|.jar analyse --sample examples/pfeiffer-phenopacket.yml --vcf path/to/manuel.vcf.gz --assembly hg19
 
-Would result in two files being output with the filename 'manuel_exomiser' and the '.json' and '.html' extensions:
+Would result in three files being output with the filename 'manuel_exomiser' and the '.jsonl', `.parquet` and '.html' extensions:
 
 .. parsed-literal::
 
   exomiser-cli-|version|/results/manuel_exomiser.html
   exomiser-cli-|version|/results/manuel_exomiser.json
+  exomiser-cli-|version|/results/manuel_exomiser.parquet
 
 
 Users requiring more control over their output can use either the ``outputOptions`` section of an analysis file or a
@@ -156,8 +157,8 @@ file:
     outputDirectory: results/
     # outputFileName: (optional) (default: 'input-vcf-name-exomiser')
 	outputFileName: NA12345-exomiser-results
-    # out-format options: HTML, JSON, TSV_GENE, TSV_VARIANT, VCF (default: HTML)
-    outputFormats: [HTML, JSON, TSV_GENE]
+    # out-format options: HTML, JSON, PARQUET, TSV_GENE, TSV_VARIANT, VCF (default: HTML)
+    outputFormats: [HTML, PARQUET]
 
 
 This file is passed to Exomiser using the ``--output`` switch:

docs/installation.rst

@@ -207,13 +207,23 @@ This will install the Exomiser program. You will still need to download the
 data files separately (see the :ref:`linux-install` instructions above for the
 list of files to download from https://data.monarchinitiative.org/exomiser/latest).
 
-After downloading and extracting the data files, configure
-``application.properties`` as described in the Linux section above.
+When installed via Homebrew, the first time it is run Exomiser will automatically create the following folders in your
+home directory:
+
+.. code-block:: bash
+
+    $ tree ~/.exomiser
+    /home/user/.exomiser
+    ├── application.properties
+    └── data
+
+If they are not present run `exomiser --version` and this should create them. By default, Exomiser will expect the data
+to be in the `~.exomiser/data` directory but this can be changed in the `~.exomiser/application.properties` file to suit
+your needs (see :ref:`data-directory`). For the most part, when using Exomiser installed via Homebrew, once configured,
+you can replace the `java -jar exomiser-cli-|version|.jar` incantation used in this manual with just `exomiser` and it
+should just work, however you will need to specify explicitly where to write the results as they will default to being
+written to the current working directory you are calling the `exomiser` command from (see :ref:`outputdirectory`)
 
-Given the nature of how Homebrew manages the application, you might want to consider
-installing the data in a separate data directory (see :ref:`data-directory`) and
-providing the variables needed in the ``application.properties`` via environment
-variables (see :ref:`overriding-settings`).
 
 .. _data-directory:
 
@@ -224,7 +234,7 @@ By default, Exomiser expects its data files to be in a ``data`` subfolder
 inside the Exomiser program folder. If you store your data elsewhere, you
 must tell Exomiser where to find it.
 
-Open ``application.properties`` and set the ``exomiser.data-directory`` to
+Edit the ``application.properties`` file and set the ``exomiser.data-directory`` to
 the full path of your data folder. For example:
 
 .. code-block:: properties
@@ -398,8 +408,46 @@ and try again.
 
 **Wrong Java version**
 
-Run ``java -version``. If the version shown is below 21, install a newer
-version from https://adoptium.net.
+Exomiser requires Java 21 or higher. This can be checked by running:
+
+.. code-block:: console
+
+    $ java -version
+
+You should see something like this in response:
+
+.. code-block:: console
+
+    openjdk 21.0.6 2025-01-21
+    OpenJDK Runtime Environment (build 21.0.6+7-Ubuntu-124.04.1)
+    OpenJDK 64-Bit Server VM (build 21.0.6+7-Ubuntu-124.04.1, mixed mode, sharing)
+
+
+Versions lower than 21 (e.g. 1.5, 1.6, 1.7, 1.8, 9, 10...) will not run exomiser.
+You can install the latest version from https://adoptium.net or https://jdk.java.net/ for
+example. On linux your distribution should provide a packaged version. MacOS users might prefer to use a homebrew version
+via ``brew install openjdk``
+
+If you get the following error message:
+
+.. code-block:: console
+
+    Exception in thread "main" java.lang.UnsupportedClassVersionError:
+    org/monarchinitiative/exomiser/cli/Main : Unsupported major.minor version
+
+
+or
+
+.. code-block:: console
+
+    Error: A JNI error has occurred, please check your installation and try again
+    Exception in thread "main" java.lang.UnsupportedClassVersionError: org/monarchinitiative/exomiser/cli/Main has been
+    compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime
+    only recognizes class file versions up to 52.0
+
+
+You are running an older unsupported version of Java, update your java installation.
+
 
 **"Permission denied" errors (Linux/macOS)**