A Neurocontrol Paradigm for Intelligent Process Control using Evolutionary Reinforcement Learning (2004)
Balancing multiple business and operational objectives within a comprehensive control strategy is a complex configuration task. Non-linearities and complex multiple process interactions combine as formidable cause-effect interrelationships. A clear understanding of these relationships is often instrumental to meeting the process control objectives. However, such control system configurations are generally conceived in a qualitative manner and with pronounced reliance on past effective configurations (Foss, 1973). Thirty years after Foss' critique, control system configuration remains a largely heuristic affair.

Biological methods of processing information are fundamentally different from the methods used in conventional control techniques. Biological neural mechanisms (i.e., intelligent systems) are based on partial models, largely devoid of the system's underlying natural laws. Neural control strategies are carried out without a pure mathematical formulation of the task or the environment. Rather, biological systems rely on knowledge of cause-effect interactions, creating robust control strategies from ill-defined dynamic systems.

Dynamic modelling may be either phenomenological or empirical. Phenomenological models are derived from first principles and typically consist of algebraic and differential equations. First principles modelling is both time consuming and expensive. Vast data warehouses of historical plant data make empirical modelling attractive. Singular spectrum analysis (SSA) is a rapid model development technique for identifying dominant state variables from historical plant time series data. Since time series data invariably covers a limited region of the state space, SSA models are almost necessarily partial models.

Interpreting and learning causal relationships from dynamic models requires sufficient feedback of the environment's state. Systemisation of the learning task is imperative. Reinforcement learning is a computational approach to understanding and automating goal-directed learning. This thesis aimed to establish a neurocontrol paradigm for non-linear, high dimensional processes within an evolutionary reinforcement learning (ERL) framework. Symbiotic memetic neuro-evolution (SMNE) is an ERL algorithm developed for global tuning of neurocontroller weights. SMNE is comprised of a symbiotic evolutionary algorithm and local particle swarm optimisation. Implicit fitness sharing ensures a global search and the synergy between global and local search speeds convergence.

Several simulation studies have been undertaken, viz. a highly non-linear bioreactor, a rigorous ball mill grinding circuit and the Tennessee Eastman control challenge. Pseudo-empirical modelling of an industrial fed-batch fermentation shows the application of SSA for developing partial models. Using SSA, state estimation is forthcoming without resorting to fundamental models. A dynamic model of a multieffect batch distillation (MEBAD) pilot plant was fashioned using SSA. Thereafter, SMNE developed a neurocontroller for on-line implementation using the SSA model of the MEBAD pilot plant.

Both simulated and experimental studies confirmed the robust performance of ERL neurocontrollers. Coordinated flow sheet design, steady state optimisation and nonlinear controller development encompass a comprehensive methodology. Effective selection of controlled variables and pairing of process and manipulated variables were implicit to the SMNE methodology. High economic performance was attained in highly non-linear regions of the state space. SMNE imparted significant generalisation in the face of process uncertainty. Nevertheless, changing process conditions may necessitate neurocontroller adaptation. Adaptive neural swarming (ANS) allows for adaptation to drifting process conditions and tracking of the economic optimum online. Additionally, SMNE allows for control strategy design beyond single unit operations. SMNE is equally applicable to processes with high dimensionality, developing plant-wide control strategies. Many of the difficulties in conventional plant-wide control may be circumvented in the biologically motivated approach of the SMNE algorithm. Future work will focus on refinements to both SMNE and SSA.

SMNE and SSA thus offer a non-heuristic, quantitative approach that requires minimal engineering judgement or knowledge, making the methodology free of subjective design input. Evolutionary reinforcement learning offers significant advantages for developing high performance control strategies for the chemical, mineral and metallurgical industries. Symbiotic memetic neuro-evolution (SMNE), adaptive neural swarming (ANS) and singular spectrum analysis (SSA) present a response to Foss' critique.

PhD Thesis, Department of Chemical Engineering, University of Stellenbosch.

Alex van Eck Conradie Formerly affiliated Visitor