*IEEE First International Conf. on Neural Networks, San Diego, CA Vol. II, pp 609-618,1987. 6. Pineda, F.J. "Generalization of backpropagation to recurrent and higher order neural networks", in Anderson, D.Z. (ed.) ...*

Skip to content
# Neural Networks

The present volume is a natural follow-up to Neural Networks: Advances and Applications which appeared one year previously. As the title indicates, it combines the presentation of recent methodological results concerning computational models and results inspired by neural networks, and of well-documented applications which illustrate the use of such models in the solution of difficult problems. The volume is balanced with respect to these two orientations: it contains six papers concerning methodological developments and five papers concerning applications and examples illustrating the theoretical developments. Each paper is largely self-contained and includes a complete bibliography. The methodological part of the book contains two papers on learning, one paper which presents a computational model of intracortical inhibitory effects, a paper presenting a new development of the random neural network, and two papers on associative memory models. The applications and examples portion contains papers on image compression, associative recall of simple typed images, learning applied to typed images, stereo disparity detection, and combinatorial optimisation.
# Neural Networks

Neural networks are a computing paradigm that is finding increasing attention among computer scientists. In this book, theoretical laws and models previously scattered in the literature are brought together into a general theory of artificial neural nets. Always with a view to biology and starting with the simplest nets, it is shown how the properties of models change when more general computing elements and net topologies are introduced. Each chapter contains examples, numerous illustrations, and a bibliography. The book is aimed at readers who seek an overview of the field or who wish to deepen their knowledge. It is suitable as a basis for university courses in neurocomputing.
# Mathematical Approaches to Neural Networks

The subject of Neural Networks is being seen to be coming of age, after its initial inception 50 years ago in the seminal work of McCulloch and Pitts. It is proving to be valuable in a wide range of academic disciplines and in important applications in industrial and business tasks. The progress being made in each approach is considerable. Nevertheless, both stand in need of a theoretical framework of explanation to underpin their usage and to allow the progress being made to be put on a firmer footing. This book aims to strengthen the foundations in its presentation of mathematical approaches to neural networks. It is through these that a suitable explanatory framework is expected to be found. The approaches span a broad range, from single neuron details to numerical analysis, functional analysis and dynamical systems theory. Each of these avenues provides its own insights into the way neural networks can be understood, both for artificial ones and simplified simulations. As a whole, the publication underlines the importance of the ever-deepening mathematical understanding of neural networks.
# Adaptive Control with Recurrent High order Neural Networks

The series Advances in Industrial Control aims to report and encourage technology transfer in control engineering. The rapid development of control technology has an impact on all areas of the control discipline. New theory, new controllers, actuators, sensors, new industrial processes, computer methods, new applications, new philosophies ... , new challenges. Much of this development work resides in industrial reports, feasibility study papers and the reports of advanced collaborative projects. The series offers an opportunity for researchers to present an extended exposition of such new work in all aspects of industrial control for wider and rapid dissemination. Neural networks is one of those areas where an initial burst of enthusiasm and optimism leads to an explosion of papers in the journals and many presentations at conferences but it is only in the last decade that significant theoretical work on stability, convergence and robustness for the use of neural networks in control systems has been tackled. George Rovithakis and Manolis Christodoulou have been interested in these theoretical problems and in the practical aspects of neural network applications to industrial problems. This very welcome addition to the Advances in Industrial Control series provides a succinct report of their research. The neural network model at the core of their work is the Recurrent High Order Neural Network (RHONN) and a complete theoretical and simulation development is presented. Different readers will find different aspects of the development of interest. The last chapter of the monograph discusses the problem of manufacturing or production process scheduling.
# New Trends in Neural Computation

Neural computation arises from the capacity of nervous tissue to process information and accumulate knowledge in an intelligent manner. Conventional computational machines have encountered enormous difficulties in duplicatingsuch functionalities. This has given rise to the development of Artificial Neural Networks where computation is distributed over a great number of local processing elements with a high degree of connectivityand in which external programming is replaced with supervised and unsupervised learning. The papers presented in this volume are carefully reviewed versions of the talks delivered at the International Workshop on Artificial Neural Networks (IWANN '93) organized by the Universities of Catalonia and the Spanish Open University at Madrid and held at Barcelona, Spain, in June 1993. The 111 papers are organized in seven sections: biological perspectives, mathematical models, learning, self-organizing networks, neural software, hardware implementation, and applications (in five subsections: signal processing and pattern recognition, communications, artificial vision, control and robotics, and other applications).
# Analog VLSI Neural Networks

This book brings together in one place important contributions and state-of-the-art research in the rapidly advancing area of analog VLSI neural networks. The book serves as an excellent reference, providing insights into some of the most important issues in analog VLSI neural networks research efforts.
# Industrial Applications of Neural Networks

Industrial Applications of Neural Networks explores the success of neural networks in different areas of engineering endeavors. Each chapter shows how the power of neural networks can be exploited in modern engineering applications. The first seven chapters focus on image processing as well as industrial or manufacturing perspectives. Topics discussed include: shape recognition shape from shading aircraft detection in SAR images visualization of high-dimensional data bases of industrial systems 3-D object learning and recognition from multiple 2-D views fingerprint classification performance optimization in flexible manufacturing systems The remaining chapters address issues and applications in the expansive area of multimedia communications as well as mobile and cellular communications.
# Recent Advances in Artificial Neural Networks

Neural networks represent a new generation of information processing paradigms designed to mimic-in a very limited sense-the human brain. They can learn, recall, and generalize from training data, and with their potential applications limited only by the imaginations of scientists and engineers, they are commanding tremendous popularity and research interest. Over the last four decades, researchers have reported a number of neural network paradigms, however, the newest of these have not appeared in book form-until now. Recent Advances in Artificial Neural Networks collects the latest neural network paradigms and reports on their promising new applications. World-renowned experts discuss the use of neural networks in pattern recognition, color induction, classification, cluster detection, and more. Application engineers, scientists, and research students from all disciplines with an interest in considering neural networks for solving real-world problems will find this collection useful.
# Fundamentals of Artificial Neural Networks

As book review editor of the IEEE Transactions on Neural Networks, Mohamad Hassoun has had the opportunity to assess the multitude of books on artificial neural networks that have appeared in recent years. Now, in Fundamentals of Artificial Neural Networks, he provides the first systematic account of artificial neural network paradigms by identifying clearly the fundamental concepts and major methodologies underlying most of the current theory and practice employed by neural network researchers. Such a systematic and unified treatment, although sadly lacking in most recent texts on neural networks, makes the subject more accessible to students and practitioners. Here, important results are integrated in order to more fully explain a wide range of existing empirical observations and commonly used heuristics. There are numerous illustrative examples, over 200 end-of-chapter analytical and computer-based problems that will aid in the development of neural network analysis and design skills, and a bibliography of nearly 700 references. Proceeding in a clear and logical fashion, the first two chapters present the basic building blocks and concepts of artificial neural networks and analyze the computational capabilities of the basic network architectures involved. Supervised, reinforcement, and unsupervised learning rules in simple nets are brought together in a common framework in chapter three. The convergence and solution properties of these learning rules are then treated mathematically in chapter four, using the "average learning equation" analysis approach. This organization of material makes it natural to switch into learning multilayer nets using backprop and its variants, described in chapter five. Chapter six covers most of the major neural network paradigms, while associative memories and energy minimizing nets are given detailed coverage in the next chapter. The final chapter takes up Boltzmann machines and Boltzmann learning along with other global search/optimization algorithms such as stochastic gradient search, simulated annealing, and genetic algorithms.
# Artificial Neural Networks for Intelligent Manufacturing

The quest for building systems that can function automatically has attracted a lot of attention over the centuries and created continuous research activities. As users of these systems we have never been satisfied, and demand more from the artifacts that are designed and manufactured. The current trend is to build autonomous systems that can adapt to changes in their environment. While there is a lot to be done before we reach this point, it is not possible to separate manufacturing systems from this trend. The desire to achieve fully automated manufacturing systems is here to stay. Manufacturing systems of the twenty-first century will demand more flexibility in product design, process planning, scheduling and process control. This may well be achieved through integrated software and hardware archi tectures that generate current decisions based on information collected from manufacturing systems environment, and execute these decisions by converting them into signals transferred through communication network. Manufacturing technology has not yet reached this state. However, the urge for achieving this goal is transferred into the term 'Intelligent Systems' that we started to use more in late 1980s. Knowledge-based systems, our first efforts in this endeavor, were not sufficient to generate the 'Intelligence' required - our quest still continues. Artificial neural network technology is becoming an integral part of intelligent manufacturing systems and will have a profound impact on the design of autonomous engineering systems over the next few years.
# Advances in Neural Networks ISNN 2004

This book constitutes the proceedings of the International Symposium on Neural N- works (ISNN 2004) held in Dalian, Liaoning, China duringAugust 19–21, 2004. ISNN 2004 received over 800 submissions from authors in ?ve continents (Asia, Europe, North America, South America, and Oceania), and 23 countries and regions (mainland China, Hong Kong, Taiwan, South Korea, Japan, Singapore, India, Iran, Israel, Turkey, Hungary, Poland, Germany, France, Belgium, Spain, UK, USA, Canada, Mexico, - nezuela, Chile, andAustralia). Based on reviews, the Program Committee selected 329 high-quality papers for presentation at ISNN 2004 and publication in the proceedings. The papers are organized into many topical sections under 11 major categories (theo- tical analysis; learning and optimization; support vector machines; blind source sepa- tion,independentcomponentanalysis,andprincipalcomponentanalysis;clusteringand classi?cation; robotics and control; telecommunications; signal, image and time series processing; detection, diagnostics, and computer security; biomedical applications; and other applications) covering the whole spectrum of the recent neural network research and development. In addition to the numerous contributed papers, ?ve distinguished scholars were invited to give plenary speeches at ISNN 2004. ISNN 2004 was an inaugural event. It brought together a few hundred researchers, educators,scientists,andpractitionerstothebeautifulcoastalcityDalianinnortheastern China. It provided an international forum for the participants to present new results, to discuss the state of the art, and to exchange information on emerging areas and future trends of neural network research. It also created a nice opportunity for the participants to meet colleagues and make friends who share similar research interests.
# High Level Feedback Control with Neural Networks

Complex industrial or robotic systems with uncertainty and disturbances are difficult to control. As system uncertainty or performance requirements increase, it becomes necessary to augment traditional feedback controllers with additional feedback loops that effectively "add intelligence" to the system. Some theories of artificial intelligence (AI) are now showing how complex machine systems should mimic human cognitive and biological processes to improve their capabilities for dealing with uncertainty. This book bridges the gap between feedback control and AI. It provides design techniques for "high-level" neural-network feedback-control topologies that contain servo-level feedback-control loops as well as AI decision and training at the higher levels. Several advanced feedback topologies containing neural networks are presented, including "dynamic output feedback", "reinforcement learning" and "optimal design", as well as a "fuzzy-logic reinforcement" controller. The control topologies areintuitive, yet are derived using sound mathematical principles where proofs of stability are given so that closed-loop performance can be relied upon in using these control systems. Computer-simulation examples are given to illustrate the performance.
# Neural Networks and Statistical Learning

This book provides a broad yet detailed introduction to neural networks and machine learning in a statistical framework. A single, comprehensive resource for study and further research, it explores the major popular neural network models and statistical learning approaches with examples and exercises and allows readers to gain a practical working understanding of the content. This updated new edition presents recently published results and includes six new chapters that correspond to the recent advances in computational learning theory, sparse coding, deep learning, big data and cloud computing. Each chapter features state-of-the-art descriptions and significant research findings. The topics covered include: • multilayer perceptron; • the Hopfield network; • associative memory models;• clustering models and algorithms; • t he radial basis function network; • recurrent neural networks; • nonnegative matrix factorization; • independent component analysis; •probabilistic and Bayesian networks; and • fuzzy sets and logic. Focusing on the prominent accomplishments and their practical aspects, this book provides academic and technical staff, as well as graduate students and researchers with a solid foundation and comprehensive reference on the fields of neural networks, pattern recognition, signal processing, and machine learning.
# Artificial Neural Networks for Modelling and Control of Non Linear Systems

Artificial neural networks possess several properties that make them particularly attractive for applications to modelling and control of complex non-linear systems. Among these properties are their universal approximation ability, their parallel network structure and the availability of on- and off-line learning methods for the interconnection weights. However, dynamic models that contain neural network architectures might be highly non-linear and difficult to analyse as a result. Artificial Neural Networks for Modelling and Control of Non-Linear Systems investigates the subject from a system theoretical point of view. However the mathematical theory that is required from the reader is limited to matrix calculus, basic analysis, differential equations and basic linear system theory. No preliminary knowledge of neural networks is explicitly required. The book presents both classical and novel network architectures and learning algorithms for modelling and control. Topics include non-linear system identification, neural optimal control, top-down model based neural control design and stability analysis of neural control systems. A major contribution of this book is to introduce NLq Theory as an extension towards modern control theory, in order to analyze and synthesize non-linear systems that contain linear together with static non-linear operators that satisfy a sector condition: neural state space control systems are an example. Moreover, it turns out that NLq Theory is unifying with respect to many problems arising in neural networks, systems and control. Examples show that complex non-linear systems can be modelled and controlled within NLq theory, including mastering chaos. The didactic flavor of this book makes it suitable for use as a text for a course on Neural Networks. In addition, researchers and designers will find many important new techniques, in particular NLq emTheory, that have applications in control theory, system theory, circuit theory and Time Series Analysis.
# Handbook of Neural Network Signal Processing

The use of neural networks is permeating every area of signal processing. They can provide powerful means for solving many problems, especially in nonlinear, real-time, adaptive, and blind signal processing. The Handbook of Neural Network Signal Processing brings together applications that were previously scattered among various publications to provide an up-to-date, detailed treatment of the subject from an engineering point of view. The authors cover basic principles, modeling, algorithms, architectures, implementation procedures, and well-designed simulation examples of audio, video, speech, communication, geophysical, sonar, radar, medical, and many other signals. The subject of neural networks and their application to signal processing is constantly improving. You need a handy reference that will inform you of current applications in this new area. The Handbook of Neural Network Signal Processing provides this much needed service for all engineers and scientists in the field.
# An Introduction to Neural Networks

Though mathematical ideas underpin the study of neural networks, the author presents the fundamentals without the full mathematical apparatus. All aspects of the field are tackled, including artificial neurons as models of their real counterparts; the geometry of network action in pattern space; gradient descent methods, including back-propagation; associative memory and Hopfield nets; and self-organization and feature maps. The traditionally difficult topic of adaptive resonance theory is clarified within a hierarchical description of its operation. The book also includes several real-world examples to provide a concrete focus. This should enhance its appeal to those involved in the design, construction and management of networks in commercial environments and who wish to improve their understanding of network simulator packages. As a comprehensive and highly accessible introduction to one of the most important topics in cognitive and computer science, this volume should interest a wide range of readers, both students and professionals, in cognitive science, psychology, computer science and electrical engineering.
# Neural Networks and Soft Computing

This volume presents new trends and developments in soft computing techniques. Topics include: neural networks, fuzzy systems, evolutionary computation, knowledge discovery, rough sets, and hybrid methods. It also covers various applications of soft computing techniques in economics, mechanics, medicine, automatics and image processing. The book contains contributions from internationally recognized scientists, such as Zadeh, Bubnicki, Pawlak, Amari, Batyrshin, Hirota, Koczy, Kosinski, Novák, S.-Y. Lee, Pedrycz, Raudys, Setiono, Sincak, Strumillo, Takagi, Usui, Wilamowski and Zurada. An excellent overview of soft computing methods and their applications.
# Advances in Intelligent Data Analysis Reasoning about Data

This book constitutes the refereed proceedings of the Second International Symposium on Intelligent Data Analysis, IDA-97, held in London, UK, in August 1997. The volume presents 50 revised full papers selected from a total of 107 submissions. Also included is a keynote, Intelligent Data Analysis: Issues and Opportunities, by David J. Hand. The papers are organized in sections on exploratory data analysis, preprocessing and tools; classification and feature selection; medical applications; soft computing; knowledge discovery and data mining; estimation and clustering; data quality; qualitative models.
# Progress in Neural Networks

This series reviews research in natural and synthetic neural networks, as well as reviews research in modelling, analysis, design and development of neural networks in software and hardware areas. Contributions from researchers and practitioners aim to shape academic and professional programs in this area, and serve as a platform for detailed and expanded discussion of topics of interest to the neural network and cognitive information processing communities. This series should be of interest to those professionally involved in neural networks research, such as lecturers and primary investigators in neural computing, modelling, learning, memory and neurocomputers.
# Constructive Neural Networks

This book presents a collection of invited works that consider constructive methods for neural networks, taken primarily from papers presented at a special th session held during the 18 International Conference on Artificial Neural Networks (ICANN 2008) in September 2008 in Prague, Czech Republic. The book is devoted to constructive neural networks and other incremental learning algorithms that constitute an alternative to the standard method of finding a correct neural architecture by trial-and-error. These algorithms provide an incremental way of building neural networks with reduced topologies for classification problems. Furthermore, these techniques produce not only the multilayer topologies but the value of the connecting synaptic weights that are determined automatically by the constructing algorithm, avoiding the risk of becoming trapped in local minima as might occur when using gradient descent algorithms such as the popular back-propagation. In most cases the convergence of the constructing algorithms is guaranteed by the method used. Constructive methods for building neural networks can potentially create more compact and robust models which are easily implemented in hardware and used for embedded systems. Thus a growing amount of current research in neural networks is oriented towards this important topic. The purpose of this book is to gather together some of the leading investigators and research groups in this growing area, and to provide an overview of the most recent advances in the techniques being developed for constructive neural networks and their applications.