(127e) The Choice of Sensitivity Metrics in Model-Based Design of Optimal Experiments

Numerical modelling, simulation and optimisation are nowadays essential tools in understanding, explaining and exploiting the behaviour of large and complex dynamic systems. When using numerical models, it is very important to perform some sort of sensitivity analysis in order to quantify the nature of the relation between model responses and parameters. In the last twenty-twenty five years, the theory of sensitivity analysis has made significant progress and its use has become more widespread. Its practical value and usefulness have been demonstrated in many fields outside the chemical kinetics area for which it was originally developed (for example, in the study of air pollution and atmospheric phenomena, combustion, population biology, etc.). Sensitivity analysis has many different and useful applications:

• the investigation of how a system outputs are affected by changes in the input parameters (first-order sensitivities),

• the detection of correlations between the model parameters (second and higher-order sensitivities),

• the identification of optimal sensor and measurement location and optimal sampling times, in particular for parameter estimation and model-based design of experiments.

This last function of sensitivity analysis is the focus of this paper. Model-based design of experiments aims at planning optimal experiments which yield the most informative data, in a statistical sense, to improve the precision in the parameter estimation. The information content of the optimal experiments is assessed by means of different criteria, which are all based on some metric (determinant, trace, eigenvalues) of the information matrix or its inverse (the parameter variance-covariance matrix) [1]. The information matrix depends, by definition, on the sensitivity coefficients and therefore the entire design of experiments relies both on their correct and precise calculation, and their correct use within the optimal design algorithms.

Sensitivity coefficients are defined (in dynamic systems) as the time-varying derivatives of model responses with respect to the model parameters. They are therefore affected by scale: variations due to the magnitude of responses or parameters have a significant effect on their values. These scale effects are frequently encountered in practice: for example, with an Arrhenius' equation, the several orders of magnitude difference between pre-exponential factor and activation energy affect the sensitivity calculations.

Several formulation of scaled and normalised sensitivities have been proposed [2] aimed at eliminating these scale effects. This paper will investigate how the model-based design of experiments is influenced by these different types of sensitivities, and in particular, will address the following questions. Can scale effects be neglected during the planning of the experiments or are the results thus obtained unreliable (in other words, at which threshold do the scale effects become significant in the design?)? If the scale effects involve only the parameters or only the responses, which is the best formulation of scaled sensitivity to be used in the experiment design? What happens if scaled or normalised sensitivities are employed for a problem where scale effects are not present? How will the use of a modified information matrix affect the design results in such cases?

To answer these questions, the role played by sensitivity analysis in the design of a new experiment is investigated in detail. The consequences of the use of different types of sensitivity scaling in the calculation of the information matrix is thoroughly explored and identified with reference to several numerical examples (with or without scale effects). From these results, some practical suggestions and guidelines are developed and presented for the choice of the sensitivity coefficients most suitable for the problem under investigation. These results should be of particular help to the wide number of experimenters who are now for the first time approaching the model-based design of experiments through powerful, easy to use simulation software, without necessarily a heavy statistical or numerical background.

References

[1] S.P. Asprey and S. Macchietto (2000). Statistical Tools for Optimal Dynamic Model Building. Computers and Chemical Engineering, 24, 1261-1267.

[2] Turányi T. (1990). Sensitivity analysis of complex kinetic systems. Tools and applications. Journal of mathematical chemistry, 5 (3), 203-248.

Keywords

Dynamic Modelling, Model-based Experiment Design, Parameter estimation, Scale Effects, Sensitivity Analysis.