Subcommand: correlation

Calculate the Edge Correlation of samples and metadata features.

Usage: gappa analyze correlation [options]

Options

Input
--jplace-path	Required. TEXT:PATH(existing)=[] ... List of jplace files or directories to process. For directories, only files with the extension `.jplace[.gz]` are processed.
Settings
--mass-norm	Required. TEXT:{absolute,relative}=absolute Set the per-sample normalization method. 'absolute' does not change the masses, while 'relative' normalizes them to a total mass of 1 per input jplace sample.
--point-mass	Treat every pquery as a point mass concentrated on the highest-weight placement.
--ignore-multiplicities	Set the multiplicity of each pquery to 1.
--edge-values	TEXT:{both,imbalances,masses}=both Values per edge used to calculate the correlation.
--method	TEXT:{all,pearson,spearman}=all Method of correlation.
Metadata Table Input
--metadata-table-file	Required. TEXT:FILE Tabular char-separated input file.
--metadata-separator-char	TEXT:{comma,tab,space,semicolon}=comma Separator char for tabular data.
--metadata-select-columns	TEXT Excludes: --metadata-ignore-columns Set the columns to select, by their name in the first (header) line of the table. All others columns are ignored. The options expects either a file with one column name per line, or an actual list of column names separated by --metadata-separator-char
--metadata-ignore-columns	TEXT Excludes: --metadata-select-columns Set the columns to ignore, by their name in the first (header) line of the table. All others columns are selected. The options expects either a file with one column name per line, or an actual list of column names separated by --metadata-separator-char
Color
--color-list	TEXT=spectral List of colors to use for the palette. Can either be the name of a color list, a file containing one color per line, or an actual list of colors.
--reverse-color-list	If set, the --color-list is reversed.
--mask-color	TEXT=#ffff00 Color used to indicate masked values.
Tree Output
--write-newick-tree	If set, the tree is written to a Newick file.
--write-nexus-tree	If set, the tree is written to a Nexus file.
--write-phyloxml-tree	If set, the tree is written to a Phyloxml file.
--write-svg-tree	If set, the tree is written to a Svg file.
Svg Tree Output
--svg-tree-shape	TEXT:{circular,rectangular}=circular Shape of the tree.
--svg-tree-type	TEXT:{cladogram,phylogram}=cladogram Type of the tree.
--svg-tree-stroke-width	FLOAT=5 Svg stroke width for the branches of the tree.
--svg-tree-ladderize	If set, the tree is ladderized.
Output
--out-dir	TEXT=. Directory to write files to
--tree-file-prefix	TEXT=correlation_ File prefix for tree files
Global Options
--allow-file-overwriting	Allow to overwrite existing output files instead of aborting the command.
--verbose	Produce more verbose output.
--threads	UINT Number of threads to use for calculations.
--log-file	TEXT Write all output to a log file, in addition to standard output to the terminal.

Description

The command takes a set of jplace files (called samples), as well as a table containing metadata features for each sample. It then calculates and visualizes the Edge Correlation with the metadata features per edge of the reference tree. The files need to have the same reference tree.

Edge Correlation is explained and evaluated in detail in our article (in preparation). The following figure and its caption are an example adapted from this article:

All subfigures show red edges or red paths at the clade on the left hand side of the tree. This indicates that presence of placements in this clade is anti-correlated with the used metadata feature. On the other hand, blue and green edges, which indicate positive correlation, are spread throughout the tree the same way in all subfigures. The extent of correlation is larger for Spearman’s Coefficient, indicating that the correlation is monotonic, but not strictly linear.

Details

By default, the command creates correlation trees for all valid metadata features, using all variants of the method. In the following, we first explain how to specify the metadata, and then how to change the default behavior.

Metadata Features (--metadata-file)

The metadata features are specified in a comma separated table file (.csv). The first row needs to contain the feature names, which are used as file names for the output files. The first column needs to contain the file names of the jplace files (samples) without extension.

Example:

File,Temperature,Salinity Sensor,Oxygen Sensor
ERR562588,19.85,36.32,221.47
ERR562558,23.83,37.49,n/a
ERR562591,26.23,36.62,199.94
ERR562643,21.44,37.89,207.79
ERR562637,26.64,35.36,189.81

This table specifies three types of metadata for five files ERR562588.jplace, ERR562558.jplace, etc. Note the n/a value in the last column. Any non-numerical value is interpreted as missing data, and is simply left out when calculating the correlation. That is, the last column only uses four data points.

Features Selection (--metadata-fields)

When specifying a comma-separated list of column headers of the meatadata table, only these features are used. Otherwise, all numerical columns are used, and trees for all for all of them are created.

Example: In order to only use the first two features of the above table, specify --metadata-fields "Temperature,Salinity Sensor" with the command. Note the double quotes, which are necessary here, as one of the feature names contains a space.

Edge Masses and Imbalances (--edge-values)

Controls whether to use masses or imbalances. By default, trees using both of them are crated. Using masses highlights the correlation of single edges, while using imbalances considers whole clades. See the article for details on the differences between these two variants.

Correlation Method (--method)

Controls which method of correlation is used for the visualization. By default, Pearsons and Spearmans are used, that is, trees for each of them are created.

Normalization (--mass-norm)

As the command is meant to show differences in a set of jplace samples files, it is important how those are normalized. Thus, the option is required.

If using --mass-norm relative, each sample (that is, each input jplace file) is normalized to unit mass 1.0, so that they all contribute equally to the result. Hence, the correlation is measured relatively. That is, a branch exhibits a high correlation with a metadata feature depending on the relative amount of placements on that branch (or in the clade, for imbalances) compared to the other placements in that sample.

On the other hand, if --mass-norm absolute is specified, the samples are not normalized. Thus, correlation is measured absolutely. Branches then exhibit a high correlation (or anti-correlation) with a metadata feature depending on the absolute number of placements on that branch (or clade). This can vastly differ from the normalized result, as the values then depends on the total number of pqueries in each sample - which in turn depend on things like amplification bias, rarefaction, and other factors that can change the total number of sequences per sample.

The decision whether to use relative or absolute abundances depends on the use case and what each sample represents. See our article for details.

Citation

When using this method, please do not forget to cite

Lucas Czech, Pierre Barbera, Alexandros Stamatakis. Genesis and Gappa: Processing, Analyzing and Visualizing Phylogenetic (Placement) Data. Bioinformatics, 2020. doi:10.1093/bioinformatics/btaa070

Lucas Czech, Alexandros Stamatakis. Scalable Methods for Analyzing and Visualizing Phylogenetic Placement of Metagenomic Samples. PLOS ONE, 2019. doi:10.1371/journal.pone.0217050