(531f) Fuzzy Model Clustering (FMC) Algorithm for Multiple Model Learning

In this work, the problem of identifying multiple linear models from operating data is considered. Depending on the operating region of the system, the multiple model learning can be either static or dynamic. The steady state operating region can be approximated using Piece-wise Linear (PWL) models and dynamics of the system can be represented using Piece-wise Autoregressive eXogenous (PWARX) models. We consider the case where the number of models, the orders of the models (in dynamic case) and the model parameters are unknown. This problem is of obvious interest in several applications including nonlinear model predictive control of chemical engineering processes. There has been prior work in this area in terms of identifying PWARX and PWL models. Most of these approaches assume either that both the number of models and model orders are known or that the model orders alone are known. These approaches can be classified as mainly optimization based or clustering based approaches. Other than the disadvantage of requiring prior knowledge, the optimization approaches are also computationally demanding to solve. The clustering approaches partition the input space based on Euclidean distance and identify the parameters in each cluster based on least squares. Since the Euclidean distance has no relation to the model fidelity, these approaches can lead to suboptimal results even when the model orders and the number of models are known. Several researchers have worked on this problem [1-9]. Recently, Jin and Huang [10] have formulated identification of PWARX using Expectation-Maximization (EM) framework. The EM method assumes that the residual errors follow single Gaussian distribution whereas the authors have used mixture distribution to offset the influence of outliers. They have assumed that the number of models and model order are known or can be estimated from methods in literature, which are iterative.

In this paper, an alternate model clustering approach to solve this problem is presented. In a traditional clustering approach, the cluster centers accumulate data points through a Euclidean distance based membership. The analogous clustering formulation in multiple model estimation is to let models accumulate data points through a prediction error based membership. This leads to the novel model clustering approach that is proposed in this paper. A similar approach based on residual error is proposed by Frigui and Krishnapuram [11], where a Multiple-Model General Linear Regression (MMGLR) algorithm is used for estimation of multiple models. The authors have modified the fuzzy clustering objective function to avoid over-estimation of data with too many models. The model update in their formulation is a modified least squares update. In the proposed formulation, the model update is based on steepest gradient method and steplength of the update is optimal in the given search direction. Since the proposed formulation is in terms of model parameters, the models migrate in an abstract model space to capture original models. This avoids over-estimation of models. The approach does not need any prior information on the number of models, the model orders and the model parameters. Moreover, the proposed formulation has minimal tuning parameters. The proposed approach identifies contiguous and non-overlapping regions in which different linear dynamic models operate. The efficacy of the proposed approach will be demonstrated on several example systems. The advantages of the proposed approach over the existing approaches will also be discussed.

References:

1. V. Cherkassky and Y. Ma, ?Multiple model regression estimation,? IEEE Trans. Neural Networks, vol. 16, pp. 785?798, 2005.

2. J. Roll, A. Bemporad, and L. Ljung, ?Identification of piecewise affine systems via mixed-integer programming,? Automatica, vol. 40, pp. 37?50, 2004.

3. A. Skeppstedt, L. Ljung, and M. Millnert, ?Construction of composite models from observed data,? International Journal of Control, vol. 55, pp. 141?152, 1992.

4. G. Ferrari-Trecate, M. Musellic, D. Liberatid, and M. Morari, ?A clustering technique for the identification of piecewise affine systems,? Automatica, vol. 39, pp. 205?217, 2003.

5. H. Nakada, K. Takaba, and T. Katayama, ?Identification of piecewise affine systems based on statistical clustering technique,? Automatica, vol. 41, pp. 905?913, 2005.

6. N. Venkat, P. Vijaysai, and D. Gudi, ?Identification of complex nonlinear processes based on fuzzy decomposition of the steady state space,? Journal of Process Control, vol. 13, pp. 473?488, 2003.

7. N. Venkat and D. Gudi, ?Fuzzy segregation-based identification and control of nonlinear dynamic systems,? Ind. Eng. Chem. Res., vol. 41,pp. 538?552, 2002.

8. R. Vidal, ?Recursive identification of switched arx systems,? Automatica, vol. 44, pp. 2274?2287, 2008.

9. F. Dufrenois and D. Hamad, ?Fuzzy weighted support vector regression for multiple linear model estimation : application to object tracking in image sequences,? in Proc. Int. Joint Conf. Neural Networks, Orlando, Florida, USA, August 2007, pp. 1289?1294.

10. X. Jin and B. Huang, Robust identification of piecewise/switching autoregressive exogenous process, AIChE Journal, Published online.

11. H. Frigui and R. Krishnapuram, A robust competitive clustering algorithm with applications in computer vision, IEEE transactions on pattern analysis and machine intelligence, vol. 21, pp. 450-465, 1999.

12. F. Klawonn and F. Hoppner, Advances in Intelligent Data Analysis V, SpringerLink, 2003.

13. L. Gröll and J. Jäkel, ?A new convergence proof of fuzzy c-means,? IEEE Trans. Fuzzy Systems, vol. 13, pp. 717?720, 2005.