hpcviewer.tex

% -*- Mode: latex; -*-
% $HeadURL$
% $Id$


\HPCToolkit{} provides the \hpcviewer{}~\cite{Adhianto-MC-Ta:2010:PSTI-hpcviewer,Tallent-MC-etal:2011:ICS-hpctoolkit-scalable-tracing} performance presentation tool for interactive examination of performance databases.
\hpcviewer{}  presents a heterogeneous
calling context tree that spans both CPU and GPU contexts, annotated
with measured or derived metrics to help users assess code performance
and identify bottlenecks.

The database generated by \hpcprof{} consists of 4 dimensions: \emph{execution context} (also called \emph{execution profile}), \emph{time}, \emph{tree node context}, and \emph{metric}.
We employ the term \emph{execution context} to include any logical threads (such as OpenMP, pthread and C++ threads), and also MPI processes and GPU streams.
The \emph{time} dimension represents the timeline of the program's execution, and \emph{tree node context} depicts a top-down path in a calling-context tree.
This time dimension is only available if the application is profiled with traces enabled (\hpcrun{} \texttt{-t} option).
Finally, the \emph{metric} dimension constitutes program measurements performed by \textit{hpcrun} such as cycles, number of instructions, stall percentages and also some derived metrics such as ratio of idleness.

To simplify performance data visualization, \hpcviewer{} restricts
display two dimensions at a time: the \emph{Profile view} (Section~\ref{sec:profile}) displays
pairs of $\langle$tree node context, metric$\rangle$ or $\langle$execution context,
metric$\rangle$ dimensions; and the \emph{Trace viewer} (Section~\ref{sec:trace}) visualizes the
behavior of execution contexts over time.
Table below summarizes views supportted by hpcviewer.

\begin{center}
  \begin{tabular}{|l|p{1.3in}|p{3in}|}\hline\hline
        View & Dimension & Note \\\hline
        Profile view - Table  & Tree node context x Metrics  & display the tree and its associated metrics \\
        Profile view - Thread & Tree node context x Metrics  & display the tree and its metrics for a set of execution contexts \\
        Profile view - Graph  & Execution contexts x Metrics & display a metric of a specific tree node for all execution contexts \\
        Trace view - Main     & Execution contexts x Time    & display execution context behavior over time \\
        Trace view - Depth    & Tree node context  x Time    & display call stacks over time of an execution context \\\hline\hline
  \end{tabular}
\end{center}



Note that in the Profile view, GPU stream execution contexts are not shown in this view; metrics for a GPU operation are associated with the calling context in the thread that initiated the GPU operation (Section~\ref{sec:hpcviewer:thread-level}).
In the Trace view, GPU streams have their own trace lines independently from their host so that it allows to separate traces between hosts and devices.

% ===========================================================================
% ===========================================================================

\section{Launching}

Requirements to launch \hpcviewer:
\begin{itemize}
 \item On all platforms: Java 11 or newer (up to Java 17). Java 20 or newer are not supported yet.
 \item On Linux: GTK 3.20 or newer.
\end{itemize}

\hpcviewer{} can either be launched from a command line (Linux platforms) or by clicking the \hpcviewer{} icon (for Windows, Mac OS X and Linux platforms).
The command line syntax is as follows:
\begin{quote}
\begin{verbatim}
  hpcviewer [options] [<hpctoolkit-database>]
\end{verbatim}
\end{quote}
Here, \texttt{<hpctoolkit-database>} is an optional argument to load a database automatically.
Without this argument, \hpcviewer{} will prompt for the location of a database. Possible options for \hpcviewer{}  are shown in the table below:\\

\begin{centering}
\begin{tabular}{|l|p{4.1in}|}\hline\hline
 \verb|-h, --help|   &
 	Print a help message. \\\hline
 \verb|-jh, --java-heap| {\em size}  &
  	Set the JVM maximum heap size for this execution of  \hpcviewer{}. The value of {\em size} must be
	in megabytes (M) or gigabytes (G). For example, one can specify a {\em size}  of 3 gigabytes as either
	3076M or 3G.\\\hline
 \verb|-v, --version|   &
	Print the current version\\\hline\hline
 \end{tabular}
 \end{centering}
 \vspace{2ex}

On Linux, when  \hpcviewer{}  is installed using its \verb|install.sh| script, which chooses a default maximum size for the Java heap on the current platform. When analyzing measurements for large and complex applications, it may be necessary to use the \verb|--java-heap| option to specify a larger heap size for  \hpcviewer{}  to accommodate many metrics for many contexts.

On MacOs and Windows the value of JVM maximum heap size is stored in \texttt{hpcviewer.ini} file, specified with \texttt{-Xmx} option.
On MacOS, this file is located at \texttt{hpcviewer.app/Contents/Eclipse/hpcviewer.ini}.

% ===========================================================================
% ===========================================================================

\section{Profile View}
\label{sec:profile}

\begin{figure}[t]
\centering{\includegraphics[width=\textwidth]{fig/hpcviewer-legend.png}}
\caption{An annotated screenshot of \hpcviewer{}'s interface.}
\label{fig:hpcviewer-legend}
\end{figure}

This view is the default view and displays pairs of $\langle$context, metric$\rangle$ dimensions.
It interactively presents context-sensitive performance metrics correlated to program structure and mapped to a program's source code, if available.
It can present an arbitrary collection of performance metrics gathered during one or more runs or compute derived metrics.

Figure~\ref{fig:hpcviewer-legend} shows an annotated screenshot of \hpcviewer{}'s user interface presenting a call path profile.
The annotations highlight \hpcviewer{}'s principal window panes and key controls.
The browser window is divided into three panes.
The Source pane (top) displays program source code.
The Navigation and Metric panes (bottom) associate a table of performance metrics with static or dynamic program structure.
These panes are discussed in more detail in Section~\ref{sec:hpcviewer:panes}.

\hpcviewer{} displays calling-context-sensitive performance data in three different views: a top-down \emph{Top-down View}, a bottom-up \emph{Bottom-up View}, and a \emph{Flat View}.
One selects the desired view by clicking on the corresponding view control tab.
We briefly describe the three views and their corresponding purposes.

\begin{itemize}
\item \textbf{Top-down View}.
  This top-down view shows the dynamic calling contexts (call paths) in which costs were incurred.
  Using this view, one can explore performance measurements of an application in a top-down fashion to understand the costs incurred by calls to a procedure in a particular calling context.
  We use the term \emph{cost} rather than simply \emph{time} since \hpcviewer{} can present a multiplicity of metrics such as cycles, or cache misses) or derived metrics (\eg{} cache miss rates or bandwidth consumed) that that are other indicators of execution cost.

  A calling context for a procedure \fnnm{f} consists of the stack of procedure frames active when the call was made to \fnnm{f}.
  Using this view, one can readily see how much of the application's cost was incurred by \fnnm{f} when called from a particular calling context.
  If finer detail is of interest, one can explore how the costs incurred by a call to \fnnm{f} in a particular context are divided between \fnnm{f} itself and the procedures it calls.
  \HPCToolkit{}'s call path profiler \hpcrun{} and the \hpcviewer{} user interface distinguish calling context precisely by individual call sites; this means that if a procedure \fnnm{g} contains calls to procedure \fnnm{f} in different places, these represent separate calling contexts.

\item \textbf{Bottom-up View}.
  This bottom-up view enables one to look upward along call paths.
  The view apportions a procedure's costs to its callers and, more generally, its calling contexts.
  This view is particularly useful for understanding the performance of software components or procedures that are used in more than one context.
  For instance, a message-passing program may call \mytt{MPI_Wait} in many different calling contexts.
  The cost of any particular call will depend upon the structure of the parallelization in which the call is made.
  Serialization or load imbalance may cause long waits in some calling contexts while other parts of the program may have short waits because computation is balanced and communication is overlapped with computation.

  When several levels of the Bottom-up View are expanded, saying that the Bottom-up View apportions metrics of a callee on behalf of its callers can be confusing. More precisely, the Bottom-up View apportions the metrics of a procedure on behalf of the various \emph{calling contexts} that reach it.

\item \textbf{Flat View}.
  This view organizes performance measurement data according to the static structure of an application.
  All costs incurred in any calling context by a procedure are aggregated together in the Flat View.
  This complements the Top-down View, in which the costs incurred by a particular procedure are represented separately for each call to the procedure from a different calling context.

\end{itemize}


% ===========================================================================
% ===========================================================================

\section{Panes}
\label{sec:hpcviewer:panes}

\hpcviewer{}'s browser window is divided into three panes: the \emph{Navigation pane}, \emph{Source pane}, and the \emph{Metrics pane}.
We briefly describe the role of each pane.

% ==========================================================
% ==========================================================

\subsection{Source Pane}
\label{sec:pane-source}

The source pane displays the source code associated with the current entity selected in the navigation pane.
When a performance database is first opened with \hpcviewer{}, the source pane is initially blank because no entity has been selected in the navigation pane.
Selecting any entity in the navigation pane will cause the source pane to load the corresponding file, scroll to and highlight the line corresponding to the selection.
Switching the source pane to view to a different source file is accomplished by making another selection in the navigation pane.

% ==========================================================
% ==========================================================

\subsection{Navigation Pane}

The navigation pane presents a hierarchical tree-based structure that is used to organize the presentation of an applications's performance data.
Entities that occur in the navigation pane's tree include load modules, files, procedures, procedure activations, inlined code, loops, and source lines.
Selecting any of these entities will cause its corresponding source code (if any) to be displayed in the source pane.
One can reveal or conceal children in this hierarchy by `opening' or `closing' any non-leaf (\ie{}, individual source line) entry in this view.

The nature of the entities in the navigation pane's tree structure depends upon whether one is exploring the Top-down View, the Bottom-up View, or the Flat View of the performance data.
\begin{itemize}
\item In the \textbf{Top-down View}, entities in the navigation tree represent procedure activations, inlined code, loops, and source lines.
  While most entities link to a single location in source code, procedure activations link to two: the call site from which a procedure was called and the procedure itself.

\item In the \textbf{Bottom-up View}, entities in the navigation tree are procedure activations.
  Unlike procedure activations in the top-down view in which call sites are paired with the called procedure, in the bottom-up view, call sites are paired with the calling procedure to facilitate attribution of costs for a called procedure to multiple different call sites and callers.

\item In the \textbf{Flat View}, entities in the navigation tree correspond to source files, procedure call sites (which are rendered the same way as procedure activations), loops, and source lines.
\end{itemize}

\subsubsection{Navigation Control}

The header above the navigation pane contains some controls for the navigation and metric view.
In Figure~\ref{fig:hpcviewer-legend}, they are labeled as ``navigation/metric control.''
\begin{itemize}

\item \textbf{Flatten} \includegraphics[scale=.8]{fig/hpcviewer-button-flatten.png} /
      \textbf{Unflatten} \includegraphics[scale=.8]{fig/hpcviewer-button-unflatten.png}
      (only available for the Flat View):

Enabling to flatten and unflatten the navigation hierarchy.
Clicking on the flatten button (the icon that shows a tree node with a slash through it) will replace each top-level scope shown with its children.
If a scope has no children (\ie{}, it is a leaf ), the node will remain in the view.
This flattening operation is useful for relaxing the strict hierarchical view so that peers at the same level in the tree can be viewed and ranked together.
For instance, this can be used to hide procedures in the Flat View so that outer loops can be ranked and compared to one another.
The inverse of the flatten operation is the unflatten operation, which causes an elided node in the tree to be made visible once again.

\item \textbf{Zoom-in} \includegraphics[scale=.8]{fig/hpcviewer-button-zoomin.png} /
      \textbf{Zoom-out} \includegraphics[scale=.8]{fig/hpcviewer-button-zoomout.png} :

Depressing the up arrow button will zoom in to show only information for the selected line and its descendants.
One can zoom out (reversing a prior zoom operation) by depressing the down arrow button.

\item \textbf{Hot call path} \includegraphics[scale=.8]{fig/hpcviewer-button-hotpath.png} :

This button is used to automatically reveal and traverse the hot call path rooted at the selected node in the navigation pane with respect to the selected metric column. Let $n$ be the node initially selected in the navigation pane.
A hot path from $n$  is traversed by comparing the values of the selected metric for $n$ and its children. If one child accounts for T\% or more (where T is the threshold value for a hot call path) of the cost at $n$,  then that child becomes $n$ and the process repeats recursively.


\item \textbf{Add derived metric} \includegraphics[scale=.8]{fig/hpcviewer-button-derivedmetric.png} :

Create a new metric by specifying a mathematical formula.
See Section~\ref{sec:hpcviewer:derived-metrics} for more details.

\item \textbf{Hide/show metrics} \includegraphics[scale=.8]{fig/hpcviewer-button-checkcolumns.png} :

Show or hide metric columns.
A dialog box will appear and the user can select which metric columns should be shown.
See Section~\ref{sec:hpcviewer:metric-pane} section for more details.

\item \textbf{Resizing metric columns} \includegraphics[scale=.8]{fig/hpcviewer-button-fitData.png}/
				       \includegraphics[scale=.8]{fig/hpcviewer-button-fitBoth.png} :

Resize the metric columns based on either the width of the data, or the width of both of the data and the column's label.

\item \textbf{Export into a CSV format file} \includegraphics[scale=.8]{fig/hpcviewer-button-csv.png} :

Export the current metric table into a comma separated value (CSV) format file.
This feature only exports all metrics that are currently shown.
Metrics that are not shown in the view (whose scopes are not expanded) will not be exported (we assume these metrics are not significant).

\item \textbf{Increase font size} \includegraphics[scale=.8]{fig/hpcviewer-button-fontplus.png} /
      \textbf{Decrease font size} \includegraphics[scale=.8]{fig/hpcviewer-button-fontminus.png} :

Increase or decrease the size of the navigation and metric panes.

\item \textbf{Show a graph of metric values} \includegraphics[scale=.8]{fig/hpcviewer-button-graph.png} :

Show a graph (a plot, a sorted plot or a histogram) of metric values associated with the selected node in CCT for all processes or threads (Section~\ref{sec:hpcviewer:plots}).

\item \textbf{Show the metrics of a set of threads} \includegraphics[scale=.8]{fig/hpcviewer-thread.png}:

Show the CCT and the metrics of a seletected threads  (Section~\ref{sec:thread-level-table}).

\end{itemize}


\subsubsection{Context menus}
Navigation control also provides several context menus by clicking the right-button of the mouse.
\begin{itemize}
 \item \textbf{Copy}: Copy into clipboard the selected line in navigation pane which includes the name of the node in the tree, and the values of visible metrics in metric pane (Section ~\ref{sec:pane-metric}). The values of hidden metrics will not be copied.
 \item \textbf{Find}: Display the Find window to allow the user to search a text within the Scope column of the current table. The window has several options such as case sensitivity, whole word search and using regular expressions.
\end{itemize}


% ==========================================================
% ==========================================================

\subsection{Metric Pane}
\label{sec:pane-metric}

The metric pane displays one or more performance metrics associated with entities to the left in the navigation pane.
Entities in the tree view of the navigation pane are sorted at each level of the hierarchy by the metric in the selected column.
When \hpcviewer{} is launched, the leftmost metric column is the default selection and the navigation pane is sorted according to the values of that metric in descending order.
One can change the selected metric by clicking on a column header.
Clicking on the header of the selected column toggles the sort order between descending and ascending.

During analysis, one often wants to consider the relationship between two metrics.
This is easier when the metrics of interest are in adjacent columns of the metric pane.
One can change the order of columns in the metric pane by selecting the column header for a metric and then dragging it left or right to its desired position.
The metric pane also includes scroll bars for horizontal scrolling (to reveal other metrics) and vertical scrolling (to reveal other scopes).
Vertical scrolling of the metric and navigation panes is synchronized.




% ===========================================================================
% ===========================================================================

\section{Understanding Metrics}

\hpcviewer{} can present an arbitrary collection of performance metrics gathered during one or more runs, or compute derived metrics expressed as formulae. A derived metric may be specified with a formula that typically uses one or more existing metrics as terms in an expression.

For any given scope in \hpcviewer{}'s three views, \hpcviewer{} computes both \emph{inclusive} and \emph{exclusive} metric values.
First, consider the Top-down View.
Inclusive metrics reflect costs for the entire subtree rooted at that scope.
Exclusive metrics are of two flavors, depending on the scope.
For a procedure, exclusive metrics reflect all costs within that procedure but excluding callees.
In other words, for a procedure, costs are exclusive with respect to dynamic call chains.
For all other scopes, exclusive metrics reflect costs for the scope itself; \ie{}, costs are exclusive with respect to static structure.
The Bottom-up and Flat Views contain inclusive and exclusive metric values that are relative to the Top-down View.
This means, \eg{}, that inclusive metrics for a particular scope in the Bottom-up or Flat View are with respect to that scope's subtree in the Top-down View.


% ==========================================================
% ==========================================================

\subsection{How Metrics are Computed}

Call path profile measurements collected by \hpcrun{} correspond directly to the Top-down View.
\hpcviewer{} derives all other views from exclusive metric costs in the Top-down View.
For the Bottom-up View, \hpcviewer{} collects the cost of all samples in each function and attribute that to a top-level entry in the Bottom-up View.
Under each top-level function, \hpcviewer{} can look up the call chain at all of the context in which the function is called.
For each function, \hpcviewer{} apportions its costs among each of the calling contexts in which they were incurred.
\hpcviewer{} computes the Flat View by traversing the calling context tree and attributing all costs for a scope to the scope within its static source code structure.
The Flat View presents a hierarchy of nested scopes for load modules, files, procedures, loops, inlined code and statements.


% ==========================================================
% ==========================================================

\subsection{Example}

\begin{figure}[t]
\centering{%
\begin{tabular}{|p{2.7in}|p{2.7in}|}
\hline
\textbf{file1.c} & \textbf{file2.c} \\
\hline
\begin{verbatim}
f () {
  g ();
}

// m is the main routine
m () {
  f ();
  g ();
}
\end{verbatim}%
  & %
\begin{verbatim}
// g can be a recursive function
g () {
  if ( . . ) g ();
  if ( . . ) h ();
}

h () {
}
\end{verbatim}%
  \\
\hline
\end{tabular}%
}
\caption{A sample program divided into two source files.}
\label{fig:source-files}
\end{figure}



\begin{figure}[t]
\centering{\includegraphics[scale=0.5]{fig/metrics-cct.png}}
\caption{Top-down View. Each node of the tree has three boxes: the left-most is the name of the node (or in this case the name of the routine, the center is the inclusive value, and on the right is the exclusive value.}
\label{fig:cct}
\end{figure}

\begin{figure}
\centering{\includegraphics[scale=0.5]{fig/metrics-callers.png}}
\caption{Bottom-up View}
\label{fig:metrics-callers}
\end{figure}

\begin{figure}
\centering{\includegraphics[scale=0.5]{fig/metrics-flat.png}}
\caption{Flat View}
\label{fig:metrics-flat}
\end{figure}


Figure~\ref{fig:source-files} shows an example of a recursive program separated into two files, \texttt{file1.c} and \texttt{file2.c}.
In this figure, we use numerical subscripts to distinguish between different instances of the same procedure.
In the other parts of this figure, we use alphabetic subscripts.
We use different labels because there is no natural one-to-one correspondence between the instances in the different views.

Routine \texttt{g} can behave as a recursive function depending on the value of the condition branch (lines 3--4).
Figure~\ref{fig:cct} shows an example of the call chain execution of the program annotated with both inclusive and exclusive costs.
Computation of inclusive costs from exclusive costs in the Top-down View involves simply summing up all of the costs in the subtree below.

In this figure, we can see that on the right path of the routine \texttt{m}, routine \texttt{g} (instantiated in the diagram as \texttt{g$_1$}) performed a recursive call (\texttt{g$_2$}) before calling routine \texttt{h}.
Although \texttt{g$_1$}, \texttt{g$_2$} and \texttt{g$_3$} are all instances from the same routine (\ie{}, \texttt{g}), we attribute a different cost for each instance.
This separation of cost can be critical to identify which instance has a performance problem.

Figure~\ref{fig:metrics-callers} shows the corresponding scope structure for the Bottom-up View and the costs we compute for this recursive program.
The procedure \texttt{g} noted as \texttt{g$_a$} (which is a root node in the diagram), has different cost to \texttt{g} as a callsite as noted as \texttt{g$_b$}, \texttt{g$_c$} and \texttt{g$_d$}.
For instance, on the first tree of this figure, the inclusive cost of \texttt{g$_a$} is \texttt{9}, which is the sum of the highest cost for each path in the calling context tree shown in Figure~\ref{fig:cct} that includes \texttt{g}: the inclusive cost of \texttt{g$_3$} (which is \texttt{3}) and \texttt{g$_1$} (which is \texttt{6}).
We do not attribute the cost of \texttt{g$_2$} here since it is a descendant of \texttt{g$_1$} (in other term, the cost of \texttt{g$_2$} is included in \texttt{g$_1$}).

Inclusive costs need to be computed similarly in the Flat View.
The inclusive cost of a recursive routine is the sum of the highest cost for each branch in calling context tree.
For instance, in Figure~\ref{fig:metrics-flat}, The inclusive cost of \texttt{g$_x$}, defined as the total cost of all instances of \texttt{g}, is \texttt{9}, and this is consistently the same as the cost in the bottom-up tree.
The advantage of attributing different costs for each instance of \texttt{g} is that it enables a user to identify which instance of the call to \texttt{g} is responsible for performance losses.


% ===========================================================================
% ===========================================================================

\section{Derived Metrics}
\label{sec:hpcviewer:derived-metrics}

Frequently, the data become useful only when combined with other information such as the number of instructions executed or the total number of cache accesses.
While users don't mind a bit of mental arithmetic and frequently compare values in different columns to see how they relate for a scope, doing this for many scopes is exhausting.
To address this problem, \hpcviewer{} provides a mechanism for defining metrics.
A user-defined metric is called a ``derived metric.''
A derived metric is defined by specifying a spreadsheet-like mathematical formula that refers to data in other columns in the metric table by using \texttt{\$n} to refer to the value in the \texttt{n}\textsuperscript{th} column.

% ==========================================================
% ==========================================================

\subsection{Formulae}

The formula syntax supported by \hpcviewer{} is inspired by spreadsheet-like in-fix mathematical formulae.
Operators have standard algebraic precedence.
%% The in-fix syntax is very simple, for example:
%% \begin{quote}
%% \begin{verbatim}
%% <expression>     ::= <binary_op> | <function>
%% <binary_op>      ::= <expression> <binary_operand> <expression>
%% <binary_operand> ::= + | - | * | /
%% \end{verbatim}
%% \end{quote}
%
%\paragraph{Intrinsic Functions}
%The list of intrinsic functions supported can be found in \href{functions.html}{here}.
%Creating a new intrinsic function requires adjusting the source code.
%If you want to do it yourself, source code for the viewer is available at https://outreach.scidac.gov.
%Otherwise, you can contact the \HPCToolkit{} team at hpc@rice.edu.


% ==========================================================
% ==========================================================

\subsection{Examples}

Suppose the database contains information from five executions, where the same two metrics were recorded for each:
\begin{enumerate}
\item Metric 0, 2, 4, 6 and 8: total number of cycles
\item Metric 1, 3, 5, 7 and 9: total number of floating point operations
\end{enumerate}
To compute the average number of cycles per floating point operation across all of the executions, we can define a formula as follows:
\begin{quote}
\begin{verbatim}
avg($0, $2, $4. $6. $8) / avg($1, $3, $5, $7, $9)
\end{verbatim}
\end{quote}

% ==========================================================
% ==========================================================

\subsection{Creating Derived Metrics}

\begin{figure}[t]
\centering{\includegraphics[width=0.8\textwidth]{fig/hpcviewer-dialog-derived-metric.png}}
\caption{Derived metric dialog box}
\label{fig:hpcviewer-derived-dialog-box}
\end{figure}

A derived metric can be created by clicking the \textbf{Derived metric} tool item in the navigation/control pane.
A derived metric window will then appear as shown in Figure~\ref{fig:hpcviewer-derived-dialog-box}.


The window has two main parts:
\begin{itemize}

\item \textbf{Derived metric definition}, which consists of:

\begin{itemize}

\item \textit{New name for the derived metric}.
  Supply a string that will be used as the column header for the derived metric.
  If you don't supply one, the metric will have no name.

\item \textit{Formula definition field}.
  In this field the user can define a formula with spreadsheet-like mathematical formula.
  This field must be filled. A user can type a formula into this field, or use the buttons in the Assistance pane below below to help insert metric terms or function templates.

\item \textit{Metrics}.
  This is used to find the \textit{ID} of a metric.
  For instance, in this snapshot, the metric \mysf{WALLCLOCK} has the ID \texttt{2}.
  By clicking the button \textbf{Insert metric}, the metric ID will be inserted in formula definition field. A metric may refer to the value at an individual node in the calling context tree (point-wise) or the value at the root of the calling context tree (aggregate).

\item \textit{Functions}.
  This is to guide the user who wants to insert functions in the formula definition field.
  Some functions require only one metric as the argument, but some can have two or more arguments.
  For instance, the function \texttt{avg()} which computes the average of some metrics, needs at least two arguments.
\end{itemize}

\item \textbf{Advanced options}:
\begin{itemize}

\item \textit{Augment metric value display with a percentage relative to column total}.
  When this box is checked, each scope's derived metric value will be augmented with a percentage value, which for scope \textit{s} is computed as the 100 * (\textit{s}'s derived metric value) / (the derived metric value computed by applying the metric formula to the aggregate values of the input metrics for the entire execution).
  Such a computation can lead to nonsensical results for some derived metric formulae.
  For instance, if the derived metric is computed as a ratio of two other metrics, the aforementioned computation that compares the scope's ratio with the ratio for the entire program won't yield a meaningful result.
  To avoid a confusing metric display, think before you use this button to annotate a metric with its percent of total.

\item \textit{Default format}. This option will display the metric value using scientific notation with three digits of precision, which is the default format.

\item \textit{Display metric value as percent}. This option will display the metric value formatted as a percent with two decimal digits. For instance, if the metric has a value 12.3415678, with this option, it will be displayed as 12.34\%.

\item \textit{Custom format}. This option will present the metric value with your customized format. The format is equivalent to Java's Formatter class, or similar to C's printf format. For example, the format "\texttt{\%6.2f}" will display six digit floating-points with two digits to the right of the decimal point.

\end{itemize}

\end{itemize}

Note that the entered formula and the metric name will be stored automatically.
One can then review again the formula (or metric name) by clicking the small triangle of the combo box.


% ===========================================================================
% ===========================================================================

\section{Metrics in Execution-context level}
\label{sec:hpcviewer:thread-level}

\textbf{Execution context} is an abstract concept of a measurable code execution.
For example, in a pure MPI application, an execution context is an MPI rank, 
while an execution context of an OpenMP application is an OpenMP thread,
and an execution context of GPU applications can be a GPU stream.
For hybrid MPI+OpenMP applications, its execution context is its MPI rank and its OpenMP master and worker threads.

There are two types of execution context: \textbf{physical} such as NODE and CORE, and \textbf{logical} like RANK, THREAD, GPUCONTEXT and GPUSTREAM.
NODE is the id of the compute node, RANK is the rank of the process (like MPI), 
CORE is the CPU core where the application thread is bound to, 
THREAD is the application CPU thread (such as OpenMP thread),
GPUCONTEXT is a context used to access a GPU (like GPU device), and
GPUSTREAM is a stream or queue used to push work to a GPU.

\subsection{Plot Graphs}
\label{sec:hpcviewer:plots}
\begin{figure}[t]
\centering{\includegraphics[width=0.8\textwidth]{fig/hpcviewer-view-rawmetrics.png}}
\caption{Plot graph view of a procedure in GAMESS MPI+OpenMP application showing a imbalance where a group of execution contexts have much higher GPU operations than others.}
\label{fig:hpcviewer-view-scatterplot}
\end{figure}

\HPCToolkit{} Experiment databases that have been generated by \hpcprof{} can be used by \hpcviewer{} to plot graphs of metric values for each execution context.
This is particularly useful for quickly assessing load imbalance \emph{in context} across the several threads or processes of an execution.
Figure~\ref{fig:hpcviewer-view-scatterplot} shows \hpcviewer{} rendering such a plot.
The horizontal axis shows application execution context sorted by index (in this case it's MPI rank and OpenMP thread).
The vertical axis shows metric values for each execution context.
Because \hpcviewer{} can generate scatter plots for any node in the Top-down View, these graphs are calling-context sensitive.

To create a graph, first select a scope in the Top-down View; in the Figure~\ref{fig:hpcviewer-view-scatterplot}, the procedure \texttt{gpu\_tdhf\_apb\_j06\_pppp\_} is selected.
Then, click the graph button \includegraphics[scale=.8]{fig/hpcviewer-button-graph.png} to show the associated sub-menus.
At the bottom of the sub-menu is a list of metrics that \hpcviewer{} can graph.
Each metric contains a sub-menu that lists the three different types of graphs \hpcviewer{} can plot.

\begin{itemize}
\item \textbf{Plot graph}.
  This standard graph plots metric values by ordered by their execution context.

\item \textbf{Sorted plot graph}.
  This graph plots metric values in ascending order.

\item \textbf{Histogram graph}.
  This graph is a histogram of metric values.
  It divides the range of metric values into a small number of sub-ranges.
  The graph plots the frequency that a metric value falls into a particular sub-range.
\end{itemize}

Note that the plot graph's execution context have the following notation:
\begin{verbatim}
  <process_id> . <thread_id>
\end{verbatim}
Hence, if the ranks are 0.0, 0.1, \dots{} 31.0, 31.1 it means MPI process 0 has two threads: thread 0 and thread 1 (similarly with MPI process 31).

Currently, it is only possible to generate scatter plots for metrics directly collected by \hpcrun{}, which excludes derived metrics created within \hpcviewer{}.


% ===========================================================================
\subsection{Thread View}
\label{sec:thread-level-table}

\hpcviewer{} also provides a feature to view the metrics of a certain execution contexts (threads and/or processes) named Thread View.

\begin{figure}[t]
\centering{\includegraphics[width=0.8\textwidth]{fig/hpcviewer-dialog-thread.png}}
\caption{A snapshot of a thread filter dialog. Users can refine the list of threads using regular expression by selecting the Regular expression checkbox.}
\label{fig:hpcviewer-dialog-thread-select}
\end{figure}

\begin{figure}[t]
\centering{\includegraphics[width=0.8\textwidth]{fig/hpcviewer-thread-table.png}}
\caption{Example of a Thread View which display thread-level metrics of a set of threads. The first column is a CCT equivalent to the CCT in the Top-down View, the second and third columns represent the metrics of the selected threads (in this case they are the sum of metrics from threads 0.1, to 7.1)}
\label{fig:hpcviewer-view-thread-level}
\end{figure}

\hpcviewer{} also provides a feature to view the metrics of a certain threads (or processes) named Thread View.
To select a thread or group of threads, you need to use the thread selection window by clicking \includegraphics[scale=.6]{fig/hpcviewer-thread.png} button from the calling-context view.
On the thread selection window (Figure~\ref{fig:hpcviewer-dialog-thread-select}), you need to select the checkbox of the threads of interest.
To  narrow the list, one can specify the thread name on the filter part of the window.
Hence, to specify just a main thread (thread zero), one can type:
\begin{verbatim}
     THREAD 0     
\end{verbatim}
on the filter, and the view only lists all threads 0 (such as RANK 1 THREAD 0, RANK 2 THREAD 0, RANK 3 THREAD 0 \dots).

Once threads have been selected, you can click \textbf{OK}, and the Thread view (Figure~\ref{fig:hpcviewer-view-thread-level}) will be activated.
The tree of the view is the same as the tree from the top-down view, with the metrics only from the selected execution contexts.
If there are more than one selected execution contexts, the metrics are the sum of the metric values.


% ===========================================================================
% ===========================================================================
\section{Filtering Tree Nodes}
\label{sec:filter}
Occasionally, It is useful to omit uninterested nodes of the tree to enable to focus on important parts.
For instance, you may want to hide all nodes associated with OpenMP runtime and just show all nodes and metrics from the application.
For this purpose, \hpcviewer{} provides \emph{filtering} to elide nodes that match a filter pattern.
\hpcviewer{} allows  users to define multiple filters, and each filter is associated with a glob pattern\footnote{A glob pattern specifies which name to be removed by using wildcard characters such as *, ? and +} and a type.
There are three types of filter: ``\emph{self only}'' to omit matched nodes,
``\emph{descendants only}'' to exclude only the subtree of the matched nodes, and ``\emph{self and descendants}'' to
remove matched nodes and its descendants.

\begin{figure}
  \centering
  \begin{subfigure}{.45\textwidth}
    \centering
    \includegraphics[scale=.8]{fig/hpcviewer-filter-cct.png}
    \caption{The original CCT tree.}
    \label{fig:filter-cct}
  \end{subfigure}%
  \qquad
  \begin{subfigure}{.45\textwidth}
    \centering
    \includegraphics[scale=.8]{fig/hpcviewer-filter-self.png}
    \caption{The result of applying \emph{self only} filter on node \texttt{C}. % against the CCT from Figure~\ref{fig:filter-cct}.
Node \texttt{C} is elided and its children (nodes \texttt{D} and \texttt{E}) are augmented to the parent of node \texttt{C}.
The exclusive cost of node \texttt{C} is also augmented to node \texttt{A}.}
    \label{fig:filter-self}
  \end{subfigure}
  \begin{subfigure}{.45\textwidth}
    \centering
    \includegraphics[scale=.8]{fig/hpcviewer-filter-children.png}
    \caption{The result of applying \emph{Descendants only} filter on node \texttt{C}. % against the CCT from Figure~\ref{fig:filter-cct}.
All the children of node \texttt{C} (nodes \texttt{D} and \texttt{E}) are elided, and the total of their exclusive cost is added to node \texttt{C}. }
    \label{fig:filter-children}
  \end{subfigure}%
  \qquad
  \begin{subfigure}{.45\textwidth}
    \centering
    \includegraphics[scale=.8]{fig/hpcviewer-filter-self_children.png}
    \caption{The result of applying \emph{self and descendants} filter on node \texttt{C}. %against the CCT from Figure~\ref{fig:filter-cct}. }
Nodes \texttt{C} and its descendants are elided, and their exclusive cost is augmented to node \texttt{A} which is the parent of node \texttt{C}.}
    \label{fig:filter-selfchildren}
  \end{subfigure}

  \caption{Different results of filtering on node \texttt{C} from Figure~\ref{fig:filter-cct} (the original CCT). Figure~\ref{fig:filter-self} shows the result of \emph{self only} filter,
	   Figure~\ref{fig:filter-children} shows the result of \emph{descendants only} filter, and Figure~\ref{fig:filter-selfchildren} shows the result of \emph{self and descendants} filter.
 Each node is attributed with two boxes on its right.  The left box represents the node's inclusive cost, while the right box represents the exclusive cost.}
  \label{fig:filter}
\end{figure}
\begin{figure}
 \centering
 \includegraphics{fig/hpcviewer-filter-window.png}
 \caption{The window of filter property.}
 \label{fig:filter-window}
\end{figure}

\paragraph{Self only}: This filter is useful to hide intermediary runtime functions such as pthread or OpenMP runtime functions.
All nodes that match filter patterns will be removed, and their children will be augmented to the parent of the elided nodes.
The exclusive cost of the elided nodes will be also augmented into the exclusive cost of the parent of the elided nodes.
Figure~\ref{fig:filter-self} shows the result of filtering node \texttt{C} of the CCT from Figure~\ref{fig:filter-cct}.
After filtering, node \texttt{C} is elided and its exclusive cost is augmented into the exclusive cost of its parent (node \texttt{A}).
The children of node \texttt{C} (nodes \texttt{D} and \texttt{E}) are now the children of node \texttt{A}.

\paragraph{Descendants only}: This filter elides only the subtree of the matched node, while the matched node itself is not removed.
A common usage of this filter is to exclude any call chains after MPI functions.
As shown in Figure~\ref{fig:filter-children}, filtering node \texttt{C} incurs nodes \texttt{D} and \texttt{E} to be elided and their exclusive cost is augmented to node \texttt{C}.

\paragraph{Self and descendants}: This filter elides both the matched node and its subtree.
This type is useful to exclude any unnecessary details such as glibc or malloc functions.
Figure~\ref{fig:filter-selfchildren} shows that filtering node \texttt{C} will elide the node and its children (nodes \texttt{D} and \texttt{E}).
The total of the exclusive cost of the elided nodes is augmented to the exclusive cost of node \texttt{A}.

The filter feature can be accessed by clicking the menu ``Filter'' and then submenu ``Show filter property'', which will then show a  Filter property window (Figure~\ref{fig:filter-window}).
The window consists of a table of filters, and a group of action buttons: \emph{add} to create a new filter; \emph{edit} to modify a selected filter; and \emph{delete} to remove a set of selected filters..
The table comprises of two columns: the left column is to display a filter's switch whether the filter is enabled or disabled, and a glob-like filter pattern; and the second column is to show the type of pattern (self only, children only or self and children).
If a checkbox is checked, it signifies the filter is enabled; otherwise the filter is disabled.

Cautious is needed when using filter feature since it can change the shape of the tree, thus affects different interpretation of performance analysis.
Furthermore, if the filtered nodes are children of a ``fake'' procedures (such as \texttt{<program root>} and \texttt{<thread root>}), the exclusive metrics in Bottom-up view and flat view can be misleading.
This occurs since these views do not show ``fake'' procedures.
%Furthermore, \hpcviewer{} does not verify if the filter will remove all the entire tree or not.

Note that the filter set is global that it affects all open databases in all windows, and it is persistent that it will also affect across hpcviewer sessions.

% ===========================================================================
% ===========================================================================

\section{Convenience Features}

In this section we describe some features of \hpcviewer{} that help improve productivity.

% ==========================================================
% ==========================================================

\subsection{Editor Pane}

The editor pane is used to display \textit{a copy} of your program's source code or \HPCToolkit{}'s performance data in XML format; for this reason, it does not support editing of the pane's contents.
To edit your program, you should use your favorite editor to edit \textit{your} original copy of the source, not the one stored in \HPCToolkit{}'s performance database.
Thanks to built-in capabilities in Eclipse, \hpcviewer{} supports some useful shortcuts and customization:
\begin{itemize}

\item \textbf{Find}.
  To search for a string in the current source pane, \texttt{$<$ctrl$>$-f} (Linux and Windows) or \texttt{$<$command$>$-f} (Mac) will bring up a find dialog that enables you to enter the target string.

\end{itemize}


% ==========================================================
% ==========================================================

\subsection{Metric Pane}
\label{sec:hpcviewer:metric-pane}

For the metric pane, \hpcviewer{} has some convenient features:
\begin{itemize}

\item \textbf{Sorting the metric pane contents by a column's values}.
  First, select the column on which you wish to sort.
  If no triangle appears next to the metric, click again.
  A downward pointing triangle means that the rows in the metric pane are sorted in descending order according to the column's value.
  Additional clicks on the header of the selected column will toggle back and forth between ascending and descending.

\item \textbf{Changing column width}.
  To increase or decrease the width of a column, first put the cursor over the right or left border of the column's header field.
  The cursor will change into a vertical bar between a left and right arrow.
  Depress the mouse and drag the column border to the desired position.

\item \textbf{Changing column order}.
  If it would be more convenient to have columns displayed in a different order, they can be permuted as you wish.
  Depress and hold the mouse button over the header of column that you wish to move and drag the column right or left to its new position.

\item \textbf{Copying selected metrics into clipboard}.
  In order to copy selected lines of scopes/metrics, one can right click on the metric pane or navigation pane then select the menu \textbf{Copy}.
  The copied metrics can then be pasted into any text editor.

\item \textbf{Hiding or showing metric columns}.
  Sometimes, it may be more convenient to suppress the display of metrics that are not of current interest.
  When there are too many metrics to fit on the screen at once, it is often useful to suppress the display of some.
  The icon \includegraphics[scale=.7]{fig/hpcviewer-button-checkcolumns.png} above the metric pane will bring up the metric property pane on the source pane area.

The pane contains a list of metrics sorted according to their order in \HPCToolkit{}'s performance database for the application.
Each metric column is prefixed by a check box to indicate if the metric should be \textit{displayed} (if checked) or \textit{hidden} (unchecked).
To display all metric columns, one can click the \textbf{Check all} button.
A click to \textbf{Uncheck all} will hide all the metric columns.
The pane also allows to edit the name of the metric or change the formula of a derived metric.
If the metric has no cost, it will be marked with grey color and it isn't editable.

Finally, an option \textbf{Apply to all views} will set the configuration into all views (Top-down, Bottom-up and Flat views) when checked.
Otherwise, the configuration will be applied only on the current view.

\end{itemize}


% ===========================================================================
% ===========================================================================

\section{Trace view}
\label{sec:trace}

% ----------------------------------------------------------
% traceviewer
% ----------------------------------------------------------

\newcommand{\crosshair}{crosshair}
\newcommand{\traceview}{Main View}
\newcommand{\depthview}{Depth View}
\newcommand{\summaryview}{Summary View}
\newcommand{\miniview}{Mini Map View}
\newcommand{\callview}{Call Stack View}
\newcommand{\statview}{Statistics View}
\newcommand{\blameview}{GPU Idleness Blame View}

% ===========================================================================
% ===========================================================================


\begin{figure}[t]
\centering{\includegraphics[width=\textwidth]{fig/hpctraceviewer-callpath.png}}
\caption{Logical view of trace call path samples on three dimensions: time, execution context (rank/thread/GPU) and call path depth.}
\label{fig:hpctraceviewer-callpath}
\end{figure}

\hpctraceviewer{}~\cite{Tallent-MC-etal:2011:ICS-hpctoolkit-scalable-tracing} is a time-centric user interface for interactive examination of a sample-based time series (hereafter referred to as a trace) view of a program execution.
 \hpctraceviewer{} can interactively present a large-scale execution trace without concern for the scale of parallelism it represents.

To collect a trace for a program execution, one must instruct \HPCToolkit{}'s measurement system to collect a trace.
When launching a dynamically-linked executable with \hpcrun{}, add the {\tt -t } flag to enable tracing.
When launching a statically-linked executable, set the environment variable \verb|HPCRUN_TRACE=1| to enable tracing.
When collecting a trace, one must also specify a metric to measure.  The best way to collect a useful trace is to asynchronously sample the execution with a time-based metric such as {\tt REALTIME}, {\tt CYCLES}, or {\tt CPUTIME}.

As shown in Figure~\ref{fig:hpctraceviewer-callpath},  call path traces consist of data in three dimensions: \emph{profile} (process/thread rank), \emph{time}, and \emph{call path} depth.
A \emph{\crosshair} in \hpctraceviewer{} is defined by a triplet $(p,t,d)$ where $p$ is the selected process/thread rank, $t$ is the selected time, and $d$ is the selected call path depth.

\hpctraceviewer{} renders a view of processes and threads over time. The \emph{\depthview} (Section~\ref{sec:depthview}) shows the call path depth over time for the thread selected by the cursor.
\hpctraceviewer{}'s  \emph{\callview}  (Section~\ref{sec:callview}) shows the call path associated with the thread and time pair specified by the cursor.
Each of these views plays a role for understanding an application's performance.

In \hpctraceviewer, each procedure is assigned specific color. Figure~\ref{fig:hpctraceviewer-callpath} shows that at depth 1   each call path has the same color: blue. This node represents the main program that serves as the root of the call chain in all process at all times. At depth 2, all processes have a green node, which indicates another procedure.
At depth 3, in the first time step all processes  have a yellow node; in subsequent time steps they have purple nodes.
This might indicate that the processes first are observed in an initialization procedure (represented by yellow) and later observed in a solve procedure (represented by purple). The pattern of colors that appears in a particular depth slice of the \traceview{} enables a user to visually identify inefficiencies such as load imbalance and serialization.




\begin{figure}[t]
\centering{\includegraphics[width=.8\textwidth]{fig/traceview-legend.png}}
\caption{A screenshot of \hpcviewer{}'s Trace view.}
\label{fig:hpctraceviewer-legend}
\end{figure}

\begin{figure}[t]
\centering{\includegraphics[width=.8\textwidth]{fig/traceview-stat.png}}
\caption{A screenshot of \hpcviewer{}'s Trace view showing the \summaryview{} and \statview.}
\label{fig:hpctraceviewer-stat}
\end{figure}

Figures~\ref{fig:hpctraceviewer-legend} and~\ref{fig:hpctraceviewer-stat} show screenshots of \hpctraceviewer{}'s capabilities in presenting call path traces.
Figure~\ref{fig:hpctraceviewer-legend} highlights \hpctraceviewer{}'s four principal window panes: \traceview (the main view), \depthview, \callview{} and \miniview,
while Figure\ref{fig:hpctraceviewer-stat} shows additional two window panes: \summaryview{} and \statview.

\begin{itemize}
\item \textbf{\traceview} (top, left pane):
  This is \hpctraceviewer{}'s primary view.
  This view, which is similar to a conventional process/time (or space/time) view, shows time on the horizontal axis and process (or thread) rank on the vertical axis; time moves from left to right.
  Compared to typical process/time views, there is one key difference.
  To show call path hierarchy, the view is actually a user-controllable slice of the process/time/call-path space.
  Given a call path depth, the view shows the color of the currently active procedure at a given time and process rank.
  (If the requested depth is deeper than a particular call path, then \hpctraceviewer{} simply displays the deepest procedure frame and, space permitting, overlays an annotation indicating the fact that this frame represents a shallower depth.)

  \hpctraceviewer{} assigns colors to procedures based on (static) source code procedures.
  Although the color assignment is currently random, it is consistent across the different views.
  Thus, the same color within the Trace and Depth Views refers to the same procedure.

  The Trace View has a white \crosshair{} that represents a selected point in time and process space.
  For this selected point, the Call Path View shows the corresponding call path.
  The Depth View shows the selected process.

\item \textbf{\depthview} (tab in bottom, left pane):
  This is a call-path/time view for the process rank selected by the \traceview's \crosshair{}.
  Given a process rank, the view shows for each virtual time along the horizontal axis a stylized call path along the vertical axis, where `main' is at the top and leaves (samples) are at the bottom.
  In other words, this view shows for the whole time range, in qualitative fashion, what the Call Path View shows for a selected point.
  The horizontal time axis is exactly aligned with the Trace View's time axis; and the colors are consistent across both views.
  This view has its own \crosshair{} that corresponds to the currently selected time and call path depth.

\item \textbf{\summaryview} (tab in bottom, left pane):
  The view shows for the whole time range displayed, the proportion of each subroutine in a certain time.
  Similar to Depth view, the time range in Summary reflects to the time range in the Trace view.

\item \textbf{\callview} (tab in top, right pane):
  This view shows two things: (1) the current call path depth that defines the hierarchical slice shown in the Trace View; and (2) the actual call path for the point selected by the Trace View's \crosshair{}.
  (To easily coordinate the call path depth value with the call path, the Call Path View currently suppresses details such as loop structure and call sites; we may use indentation or other techniques to display this in the future.)

\item \textbf{\statview} (tab in top, right pane):
  This view shows the list of procedures active  in the  space-time region shown in the Trace View at the current Call Path Depth. Each procedure's percentage in the \statview{} indicates the percentage of pixels in the Trace View pane that are filled with this procedure's color at the current Call Path Depth. When the Trace View is navigated to show a new time-space interval  or the Call Path Depth is  changed, the statistics view will  update its list of procedures and the percentage of execution time to reflect the new space-time interval or depth selection.

\item \textbf{\blameview} (tab in top, right pane):
The view shows the list of procedures that cause GPU idleness displayed in the trace view.
If the trace view displays one CPU thread and multiple GPU streams, then the CPU thread will be blamed for the idleness for those GPU streams.
If the view contains more than one CPU threads and multiple GPU streams, then the cost of idleness is share among the CPU threads.

\item \textbf{\miniview} (right, bottom):
  The Mini Map shows, relative to the process/time dimensions, the portion of the execution shown by the Trace View.
  The Mini Map enables one to zoom and to move from one close-up to another quickly.

\end{itemize}

% ===========================================================================
% ===========================================================================

\subsection{\traceview}
\label{sec:traceview}

\traceview{} is divided into two parts: the top part which contains \emph{action pane} and the \emph{information pane}, and the main canvas which displays the traces.

The buttons in the action pane are the following:
\begin{itemize}

\item \textbf{Home} \includegraphics[scale=.5]{fig/hpctraceviewer-button-home-screen.png} : Resetting the view configuration into the original view, i.e., viewing traces for all times and processes.
\item \textbf{Horiontal zoom in \includegraphics{fig/hpctraceviewer-button-zoom-in-time.png} / out }\includegraphics{fig/hpctraceviewer-button-zoom-out-time.png} : Zooming in/out the time dimension of the traces.
\item \textbf{Vertical zoom in \includegraphics[scale=.5]{fig/hpctraceviewer-button-zoom-in-process.png} / out \includegraphics[scale=.5]{fig/hpctraceviewer-button-zoom-out-process.png} }: Zooming in/out the process dimension of the traces.
\item \textbf{Navigation buttons} \includegraphics[scale=.5]{fig/hpctraceviewer-button-go-east.png}, \includegraphics[scale=.5]{fig/hpctraceviewer-button-go-west.png}, \includegraphics[scale=.5]{fig/hpctraceviewer-button-go-north.png}, \includegraphics[scale=.5]{fig/hpctraceviewer-button-go-south.png} : Navigating the trace view to the left, right, up and bottom, respectively. It is also possible to navigate with the arrow keys in the keyboard. Since \traceview{} does not support scrool bars, the only way to navigate is through navigation buttons (or arrow keys).
\item \textbf{Undo} \includegraphics[scale=.5]{fig/hpctraceviewer-button-undo.png} : Canceling the action of zoom or navigation and returning back to the previous view configuration.
\item \textbf{Redo} \includegraphics[scale=.5]{fig/hpctraceviewer-button-redo.png} : Redoing of previously undo change of view configuration.
\item \textbf{Save} \includegraphics[scale=.5]{fig/hpctraceviewer-button-save.png}  / \textbf{Open \includegraphics[scale=.7]{fig/hpctraceviewer-button-open.png} a view configuration} : Saving/loading a saved view configuration.
A view configuration file contains the information about the process/thread and time ranges shown, the selected depth, and the position of the \crosshair{}.
It is recommended to store the view configuration file in the same directory as the database to ensure that the view configuration file matches the database since a configuration does not store its associated database.
Although it is possible to open a view configuration file associated with a different database, it is not recommended since each database has different time/process dimensions and depth.


\end{itemize}

At the top of an execution's \traceview{} pane is some information about the data shown in the pane.
\begin{itemize}
 \item \textbf{Time Range}. The time interval shown along the horizontal dimension.
 \item \textbf{Cross Hair}. The \crosshair{} indicates the current cursor position in the time and execution-context dimensions.
\end{itemize}

% ===========================================================================
% ===========================================================================
\subsection{\depthview}
\label{sec:depthview}

\depthview{} shows all the call path for a certain time range $[t_1,t_2]= \{t | t_1\leq t\leq t_2\}$ in a specified process rank $p$. The content of \depthview{} is always consistent with the position of the \crosshair{} in \traceview{}.
For instance once the user clicks in process $p$ and time $t$, while the current depth of call path is $d$, then the \depthview's content is updated to display all the call path of process $p$ and shows its \crosshair{} on the time $t$ and the call path depth $d$.

On the other hand, any user action such as \crosshair{} and time range selection in \depthview{} will update the content within \traceview. Similarly, the selection of new call path depth in \callview{} invokes a new position in \depthview.

In \depthview{} a user can specify a new \crosshair{} time and a new time range.
\paragraph{Specifying a new \crosshair{} time.} Selecting a new \crosshair{} time $t$ can be performed by clicking a pixel within \depthview{}. This will update the \crosshair{} in \traceview{} and the call path in \callview.

\paragraph{Selecting a new time range.} Selecting a new time range $[t_m,t_n]= \{t | t_m\leq t\leq t_n\}$ is performed by first clicking the position of $t_m$ and drag the cursor to the position of $t_n$. A new content in \depthview{} and \traceview{} is then updated. Note that this action will not update the call path in \callview{} since it does not change the position of the \crosshair.


% ===========================================================================
% ===========================================================================
\subsection{\summaryview}
\label{sec:summaryview}

\summaryview{} presents the proportion of number of calls of time $t$ across the current displayed rank of process $p$.
Similar to \depthview, the time range in \summaryview{} is always consistent with the time range in \traceview{}.


% ===========================================================================
% ===========================================================================
\subsection{\callview}
\label{sec:callview}


This view lists the call path of process $p$ and time $t$ specified in \traceview{} and \depthview.
Figure~\ref{fig:hpctraceviewer-legend} shows a call path of the current cross hair, and the current depth is $10$ as shown in the depth editor (located on the top part of the view).

In this view, the user can select the depth dimension of \traceview{} by either typing the depth in the depth editor or selecting a procedure in the table of call path.

% ===========================================================================
% ===========================================================================
\subsection{\miniview}
\label{sec:miniview}

The \miniview{} shows, relative to the process/time dimensions, the portion of the execution shown by the \traceview.
In \miniview{}, the user can select a new process/time $(p_a,t_a),(p_b,t_b)$ dimensions by clicking the first process/time position $(p_a,t_a)$ and then drag the cursor to the second position $(p_b,t_b)$.
The user can also moving the current selected region to another region by clicking the white rectangle and drag it to the new place.

\hpctraceviewer{} also provides a context menu to save the current image of the view.
This context menu is available is three views: trace view, depth view and summary view.

% ===========================================================================
% ===========================================================================

\section{Menus}

\hpcviewer{} provides four main menus:

% ==========================================================
% ==========================================================

\subsection{File}
This menu includes several menu items for controlling basic viewer operations.
\begin{itemize}
  \item \textbf{New window}
  Open a new \hpcviewer{} window that is independent from the existing one.
However, filtering CCT node operation (Section~\ref{sec:filter}) will affect all \hpcviewer{} windows.

  \item \textbf{Open database}
  Open a database without replacing the existing one. This menu can be used to compare two databases.
  Currently \hpcviewer{} restricts maximum of two database open at a time.

  \item \textbf{Switch database}
  Load a performance database into the current \hpcviewer{} window replacing the existing opened databases.

  \item \textbf{Close database}
  Unloading an open database.

  \item \textbf{Merge databases}
  Merging two database that are currently in the viewer.
  At the moment \hpcviewer{} doesn't support storing a merged database into a file.
  \begin{itemize}
    \item \textbf{Top-down tree} Merging the top-down tree of the databases.
    \item \textbf{Flat tree}     Merging the flat (static) tree of the databases.
  \end{itemize}

  \item \textbf{Preferences}
  Display the settings dialog box which consists of three sections:
  \begin{itemize}
     \item \textbf{General} Enable/disable debug mode.
     \item \textbf{Appearance} Change the fonts for tree and metric columns and source viewer.
     \item \textbf{Traces} Specify settings for Trace view such as the rendering option, the number of working threads to be used and the tooltip's delay.
  \end{itemize}


  \item \textbf{Exit}
  Quit the \hpcviewer{} application.

\end{itemize}

% ==========================================================
% ==========================================================

\subsection{Filter}
This menu only contains one submenu:
\begin{itemize}
 \item \textbf{Filter CCT nodes}
  Open a filter property window which lists a set of filters and its properties (Section~\ref{sec:filter}).

 \item \textbf{Filter execution contexts} \textit{(Trace view only)}
  Open a window for selecting which nodes will be hidden in the tree.
  Currently filtering CCT nodes only affect the Profile view, and doesn't affect the Trace view.

\end{itemize}

% ==========================================================
% ==========================================================

\subsection{View}

\begin{figure}[t]
\centering{\includegraphics[width=3.4in]{fig/hpctraceviewer-dialog-mapping}}
\caption{Procedure-color mapping dialog box. This window shows that any procedure names that match with "MPI*" pattern are assigned with red, while procedures that match with "PMPI*" pattern are assigned with color black.}
\label{fig:hpctraceviewer-mapping}
\end{figure}


This menu is only visible if at least one database is loaded.
All actions in this menu are intended primarily for tool developer use.
By default, the menu is hidden. Once you open a database, the menu is then shown.

\begin{itemize}

 \item \textbf{Show metrics}   \textit{(Profile view only)}
 Display a list of (metric name, metric name description) pairs in a window. For GPU metrics, the descriptions are useful for explaining what the short and somewhat cryptic metric names mean.
 From this window, you can use the edit button to modify the name of the selected metric. When editing a derived metric, the metric editor will allow you to modify the formula for the metric in addition to the name.
 Once you modify a metric and exit this window by selecting the OK button, the metric pane will refresh the display of any metrics whose name or formula was modified.

 \item \textbf{Show color mapping} \textit{(Trace view only)}
  Open a window which shows customized mapping between a procedure pattern and a color (Figure~\ref{fig:hpctraceviewer-mapping}). \hpctraceviewer{} allows users to customize assignment of a pattern of procedure names with a specific color.

 \item \textbf{Debug} \emph{(if the debug mode is enabled)}

   \begin{itemize}
     \item \textbf{Show database raw's XML}
 	Enable one to request display of \HPCToolkit{}'s raw XML representation for performance data.

  \end{itemize}

\end{itemize}
% ==========================================================
% ==========================================================

\subsection{Help}

This menu displays information about the viewer. The menu contains only one menu item:
\begin{itemize}

\item \textbf{About}.
  Displays brief information about the viewer, including JVM and Eclipse variables, and error log files.

\end{itemize}


% ===========================================================================
% ===========================================================================

\section{Limitations}

Some important \hpcviewer{} limitations are listed below:
\begin{itemize}

\item \textbf{Limited number of metric columns}.
With a large number of metric columns, \hpcviewer{}'s response time may become sluggish as this requires a large amount of memory.

\item \textbf{Experimental Windows 11 platform}.
The Windows version of \hpcviewer{} is mainly tested on Windows 10. Support for Windows 11 is still experimental.

\item \textbf{Dark theme on Linux platforms}.
We received reports that \hpcviewer{} is not very visible on Linux with dark theme. Support for dark theme on Linux is still an ongoing work.

\item \textbf{Linux TWM window manager is not supported}.
Reason: this window manager is too ancient.

%\item \textbf{Printing metrics}.
%  \hpcviewer{} does not currently support printing metric values.
%  However it is possible to export metrics into a file.

\end{itemize}