Extended Parmest Capability for weighted SSE objective

doc/OnlineDocs/explanation/analysis/parmest/covariance.rst

@@ -1,16 +1,156 @@
 Covariance Matrix Estimation
 =================================
 
-If the optional argument ``calc_cov=True`` is specified for :class:`~pyomo.contrib.parmest.parmest.Estimator.theta_est`, 
-parmest will calculate the covariance matrix :math:`V_{\theta}` as follows:
-
-.. math::
-   V_{\theta} = 2 \sigma^2 H^{-1} 
-
-This formula assumes all measurement errors are independent and identically distributed with 
-variance :math:`\sigma^2`. :math:`H^{-1}` is the inverse of the Hessian matrix for an unweighted 
-sum of least squares problem. Currently, the covariance approximation is only valid if the 
-objective given to parmest is the sum of squared error. Moreover, parmest approximates the 
-variance of the measurement errors as :math:`\sigma^2 = \frac{1}{n-l} \sum e_i^2` where :math:`n` is 
-the number of data points, :math:`l` is the number of fitted parameters, and :math:`e_i` is the 
-residual for experiment :math:`i`.
+The uncertainty in the estimated parameters is quantified using the covariance matrix.
+The diagonal of the covariance matrix contains the variance of the estimated parameters.
+Assuming Gaussian independent and identically distributed measurement errors, the
+covariance matrix of the estimated parameters can be computed using the following
+methods which have been implemented in parmest.
+
+1. Reduced Hessian Method
+
+    .. math::
+       V_{\boldsymbol{\theta}} = 2 \sigma^2 \left(\frac{\partial^2 \text{SSE}}
+        {\partial \boldsymbol{\theta} \partial \boldsymbol{\theta}}\right)^{-1}_{\boldsymbol{\theta}
+        = \boldsymbol{\theta}^*}
+
+    Where SSE is the sum of squared errors, WSSE is the weighted SSE,
+    :math:`\boldsymbol{\theta}` are the unknown parameters, :math:`\boldsymbol{\theta^*}`
+    are the estimate of the unknown parameters, and :math:`\sigma^2` is the variance of
+    the measurement error. When the standard deviation of the measurement error is not
+    supplied by the user, parmest approximates the variance of the measurement error as
+    :math:`\sigma^2 = \frac{1}{n-l} \sum e_i^2` where :math:`n` is the number of data
+    points, :math:`l` is the number of fitted parameters, and :math:`e_i` is the residual
+    for experiment :math:`i`.
+
+2. Finite Difference Method
+
+    .. math::
+       V_{\boldsymbol{\theta}} = \left( \sum_{r = 1}^n \mathbf{G}_{r}^{\mathrm{T}} \mathbf{W}
+        \mathbf{G}_{r} \right)^{-1}
+
+    This method uses central finite difference to compute the Jacobian matrix,
+    :math:`\mathbf{G}_{r}`, which is the sensitivity of the measured variables with
+    respect to the parameters, `\boldsymbol{\theta}`. :math:`\mathbf{W}` is a diagonal
+    matrix containing the inverse of the variance of the measurement errors,
+    :math:`\sigma^2`.
+
+3. Automatic Differentiation Method
+
+    .. math::
+       V_{\boldsymbol{\theta}} = \left( \sum_{r = 1}^n \mathbf{G}_{\text{kaug},\, r}^{\mathrm{T}}
+        \mathbf{W} \mathbf{G}_{\text{kaug},\, r} \right)^{-1}
+
+    This method uses the model optimality (KKT) condition to compute the Jacobian matrix,
+    :math:`\mathbf{G}_{\text{kaug},\, r}`.
+
+In parmest, the covariance matrix can be calculated after defining the
+:class:`~pyomo.contrib.parmest.parmest.Estimator` object and estimating the unknown
+parameters using :class:`~pyomo.contrib.parmest.parmest.Estimator.theta_est`. To
+estimate the covariance matrix, call
+:class:`~pyomo.contrib.parmest.parmest.Estimator.cov_est` and pass it the number
+of data points, e.g.,
+
+.. testsetup:: *
+    :skipif: not __import__('pyomo.contrib.parmest.parmest').contrib.parmest.parmest.parmest_available
+
+    # Data
+    import pandas as pd
+    data = pd.DataFrame(
+        data=[[1, 8.3], [2, 10.3], [3, 19.0],
+              [4, 16.0], [5, 15.6], [7, 19.8]],
+        columns=['hour', 'y'],
+    )
+    num_data = len(data)
+
+    # Create the Rooney-Biegler model
+    def rooney_biegler_model():
+        """
+        Formulates the Pyomo model of the Rooney-Biegler example
+
+        Returns:
+            m: Pyomo model
+        """
+        m = pyo.ConcreteModel()
+
+        m.asymptote = pyo.Var(within=pyo.NonNegativeReals, initialize=10)
+        m.rate_constant = pyo.Var(within=pyo.NonNegativeReals, initialize=0.2)
+
+        m.hour = pyo.Var(within=pyo.PositiveReals, initialize=0.1)
+        m.y = pyo.Var(within=pyo.NonNegativeReals)
+
+        @m.Constraint()
+        def response_rule(m):
+            return m.y == m.asymptote * (1 - pyo.exp(-m.rate_constant * m.hour))
+
+        return m
+
+    # Create the Experiment class
+    from pyomo.contrib.parmest.experiment import Experiment
+    class RooneyBieglerExperiment(Experiment):
+        def __init__(self, hour, y):
+            self.y = y
+            self.hour = hour
+            self.model = None
+
+        def get_labeled_model(self):
+            self.create_model()
+            self.finalize_model()
+            self.label_model()
+
+            return self.model
+
+        def create_model(self):
+            m = self.model = rooney_biegler_model()
+
+            return m
+
+        def finalize_model(self):
+            m = self.model
+
+            # fix the input variable
+            m.hour.fix(self.hour)
+
+            return m
+
+        def label_model(self):
+            m = self.model
+
+            # add experiment outputs
+            m.experiment_outputs = pyo.Suffix(direction=pyo.Suffix.LOCAL)
+            m.experiment_outputs.update([(m.y, self.y)])
+
+            # add unknown parameters
+            m.unknown_parameters = pyo.Suffix(direction=pyo.Suffix.LOCAL)
+            m.unknown_parameters.update(
+                (k, pyo.value(k)) for k in [m.asymptote, m.rate_constant]
+            )
+
+            # create the measurement error
+            m.measurement_error = pyo.Suffix(direction = pyo.Suffix.LOCAL)
+            m.measurement_error.update([(m.y, None)])
+
+    # Create an experiment list
+    exp_list = []
+    for i in range(data.shape[0]):
+        exp_list.append(RooneyBieglerExperiment(data["hour"][i], data["y"][i]))
+
+.. doctest::
+    :skipif: not __import__('pyomo.contrib.parmest.parmest').contrib.parmest.parmest.parmest_available
+
+    >>> import pyomo.contrib.parmest.parmest as parmest
+    >>> pest = parmest.Estimator(exp_list, obj_function="SSE")
+    >>> obj_val, theta_val = pest.theta_est()
+    >>> cov = pest.cov_est(cov_n=num_data)
+
+Optionally, one of the three methods; "reduced_hessian", "finite_difference",
+and "automatic_differentiation_kaug" can be supplied for the covariance calculation,
+e.g.,
+
+.. doctest::
+    :skipif: not __import__('pyomo.contrib.parmest.parmest').contrib.parmest.parmest.parmest_available
+
+    >>> pest = parmest.Estimator(exp_list, obj_function="SSE")
+    >>> obj_val, theta_val = pest.theta_est()
+    >>> cov_method = "reduced_hessian"
+    >>> cov = pest.cov_est(cov_n=num_data, method=cov_method)

doc/OnlineDocs/explanation/analysis/parmest/datarec.rst

@@ -42,13 +42,6 @@ The following example returns model values from a Pyomo Expression.
    >>> for i in range(data.shape[0]):
    ...     exp_list.append(RooneyBieglerExperiment(data.loc[i, :]))
 
-   >>> # Define objective
-   >>> def SSE(model):
-   ...     expr = (model.experiment_outputs[model.y]
-   ...             - model.response_function[model.experiment_outputs[model.hour]]
-   ...            ) ** 2
-   ...     return expr
-
-   >>> pest = parmest.Estimator(exp_list, obj_function=SSE, solver_options=None)
+   >>> pest = parmest.Estimator(exp_list, obj_function="SSE", solver_options=None)
    >>> obj, theta, var_values = pest.theta_est(return_values=['response_function'])
    >>> #print(var_values)

doc/OnlineDocs/explanation/analysis/parmest/driver.rst

@@ -16,19 +16,21 @@ the model and the observations (typically defined as the sum of squared
 deviation between model values and observed values).
 
 If the Pyomo model is not formatted as a two-stage stochastic
-programming problem in this format, the user can supply a custom
-function to use as the second stage cost and the Pyomo model will be
+programming problem in this format, the user can choose either the
+built-in "SSE" or "SSE_weighted" objective functions, or supply a custom
+objective function to use as the second stage cost. The Pyomo model will then be
 modified within parmest to match the required specifications.
-The stochastic programming callback function is also defined within parmest.  The callback
-function returns a populated and initialized model for each scenario.
+The stochastic programming callback function is also defined within parmest.
+The callback function returns a populated and initialized model for each scenario.
 
-To use parmest, the user creates a :class:`~pyomo.contrib.parmest.parmest.Estimator` object 
-which includes the following methods:
+To use parmest, the user creates a :class:`~pyomo.contrib.parmest.parmest.Estimator`
+object which includes the following methods:
 
 .. autosummary::
    :nosignatures:
 
    ~pyomo.contrib.parmest.parmest.Estimator.theta_est
+   ~pyomo.contrib.parmest.parmest.Estimator.cov_est
    ~pyomo.contrib.parmest.parmest.Estimator.theta_est_bootstrap
    ~pyomo.contrib.parmest.parmest.Estimator.theta_est_leaveNout
    ~pyomo.contrib.parmest.parmest.Estimator.objective_at_theta
@@ -65,40 +67,51 @@ Section.
         columns=['hour', 'y'],
     )
 
-    # Sum of squared error function
-    def SSE(model):
-        expr = (
-            model.experiment_outputs[model.y]
-            - model.response_function[model.experiment_outputs[model.hour]]
-        ) ** 2
-        return expr
-
     # Create an experiment list
     from pyomo.contrib.parmest.examples.rooney_biegler.rooney_biegler import RooneyBieglerExperiment
+
+    class NewRooneyBieglerExperiment(RooneyBieglerExperiment):
+        def label_model(self):
+            m = self.model
+
+            # create the experiment outputs
+            m.experiment_outputs = pyo.Suffix(direction=pyo.Suffix.LOCAL)
+            m.experiment_outputs.update(
+                [(m.response_function[self.data['hour']], self.data['y'])]
+            )
+
+            # create the unknown parameters
+            m.unknown_parameters = pyo.Suffix(direction=pyo.Suffix.LOCAL)
+            m.unknown_parameters.update((k, pyo.value(k)) for k in [m.asymptote, m.rate_constant])
+
+            # create the measurement error
+            m.measurement_error = pyo.Suffix(direction = pyo.Suffix.LOCAL)
+            m.measurement_error.update([(m.response_function[self.data['hour']], None)])
+
     exp_list = []
     for i in range(data.shape[0]):
-        exp_list.append(RooneyBieglerExperiment(data.loc[i, :]))
+        exp_list.append(NewRooneyBieglerExperiment(data.loc[i, :]))
 
 .. doctest::
     :skipif: not __import__('pyomo.contrib.parmest.parmest').contrib.parmest.parmest.parmest_available
 
     >>> import pyomo.contrib.parmest.parmest as parmest
-    >>> pest = parmest.Estimator(exp_list, obj_function=SSE)
+    >>> pest = parmest.Estimator(exp_list, obj_function="SSE")
 
 Optionally, solver options can be supplied, e.g.,
 
 .. doctest::
     :skipif: not __import__('pyomo.contrib.parmest.parmest').contrib.parmest.parmest.parmest_available
 
     >>> solver_options = {"max_iter": 6000}
-    >>> pest = parmest.Estimator(exp_list, obj_function=SSE, solver_options=solver_options)
+    >>> pest = parmest.Estimator(exp_list, obj_function="SSE", solver_options=solver_options)
 
 
 List of experiment objects
 --------------------------
 
 The first argument is a list of experiment objects which is used to
-create one labeled model for each expeirment. 
+create one labeled model for each experiment.
 The template :class:`~pyomo.contrib.parmest.experiment.Experiment` 
 can be used to generate a list of experiment objects.
 
@@ -137,17 +150,20 @@ expressions that are used to build an objective for the two-stage
 stochastic programming problem.  
 
 If the Pyomo model is not written as a two-stage stochastic programming problem in
-this format, and/or if the user wants to use an objective that is
-different than the original model, a custom objective function can be
-defined for parameter estimation.  The objective function has a single argument, 
-which is the model from a single experiment.
+this format, the user can select the "SSE" or "SSE_weighted" built-in objective
+functions. If the user wants to use an objective that is different from the built-in
+options, a custom objective function can be defined for parameter estimation. However,
+covariance matrix estimation will not support this custom objective function. The objective
+function (built-in or custom) has a single argument, which is the model from a single
+experiment.
 The objective function returns a Pyomo
 expression which is used to define "SecondStageCost".  The objective
 function can be used to customize data points and weights that are used
 in parameter estimation.
 
-Parmest includes one built in objective function to compute the sum of squared errors ("SSE") between the 
-``m.experiment_outputs`` model values and data values.
+Parmest includes two built-in objective functions ("SSE" and "SSE_weighted") to compute
+the sum of squared errors between the ``m.experiment_outputs`` model values and
+data values.
 
 Suggested initialization procedure for parameter estimation problems
 --------------------------------------------------------------------
@@ -162,4 +178,11 @@ estimation solve from the square problem solution, set optional argument ``solve
 argument ``(initialize_parmest_model=True)``. Different initial guess values for the fitted 
 parameters can be provided using optional argument `theta_values` (**Pandas Dataframe**)
 
-3. Solve parameter estimation problem by calling :class:`~pyomo.contrib.parmest.parmest.Estimator.theta_est`
+3. Solve parameter estimation problem by calling
+:class:`~pyomo.contrib.parmest.parmest.Estimator.theta_est`, e.g.,
+
+.. doctest::
+    :skipif: not __import__('pyomo.contrib.parmest.parmest').contrib.parmest.parmest.parmest_available
+
+    >>> pest = parmest.Estimator(exp_list, obj_function="SSE")
+    >>> obj_val, theta_val = pest.theta_est()