Knowledge Extraction from the Neural ‘Black Box’ in Ecological Monitoring

Document Type: Research Paper

Authors

1 Industrial Systems Engineering, Ohio University, Athens, OH 45701-2979, USA

2 Florida Institute of Oceanography, University of South Florida, Saint Petersburg, Florida 33701, USA

3 Lake Michigan Field Station, Great Lakes Environmental Research Laboratory, National Oceanic & Atmospheric Administration, Muskegon, Michigan 49441,USA

Abstract

Phytoplankton biomass within the Saginaw Bay ecosystem (Lake Huron, Michigan, USA) was characterized as a function of select physical/chemical indicators. The complexity and variability of ecological systems typically make it difficult to model the influences of anthropogenic stressors and/or natural disturbances. Here, Artificial Neural Networks (ANNs) were developed to model chlorophyll a concentrations, a measure for water-column phytoplankton biomass and a proxy for system-level health. ANNs act like “black boxes” in the sense that relationships are encoded as weight vectors within the trained network and as such, cannot easily support the generation of scientific hypotheses unless these relationships can be explained in a comprehensible form. Accordingly, the ‘knowledge’ and/or rule-based information embedded within ANNs needs to be extracted and expressed as a set of comprehensible ‘rules’. Such extracted information would enhance the delineation and understanding of ecological complexity and aid in developing usable prediction tools. Comparisons of various computational approaches (including TREPAN, an algorithm for constructing decision trees from neural networks) used in extracting rule-based information from trained Saginaw Bay ANNs are discussed.

Keywords

Main Subjects


[1] Andrews R., Diederich, Tickle A. B. (1995), Survey and critique of techniques for extracting rules
from trained artificial neural networks; Knowledge Based Systems 8; 373-389.
[2] Aoki I., Komatsu T. (1999). Analysis and prediction of the fluctuation of sardine abundance using a
neural network; Oceanol. Acta 20; 81–88.
[3] Biggs D., de Ville B., Suen E. (1991), A method of choosing multiway partitions for classification and
decision tree; Journal of Applied Statistics 18(1); 49-62.
[4] Chen D.G., Ware D.M. (1999), A neural network model for forecasting fish stock recruitment, Can. J.
Fish. Aquat. Sci. 56; 2385–2396
[5] Craven M.W, Shavlik J.W. (1995), Using sampling and queries to extract rules from trained neural
networks; Machine Learning. Proceedings of the Eleventh Inter-national Conference, Cohen W.W &
Hirsh H. (Eds.), San Francisco, CA: Morgan Kaufmann.
[6] Craven M.W., Shavlik J.W. (1996a), Extracting tree-structured representations of trained networks;
Advances in Neural Information Processing 8; 24-30.
[7] Craven M.W. (1996b), Extracting Comprehensible models from trained Neural Networks; PhD Thesis;
Computer Science Department, University of Wisconsin, Madison, WI.
[8] Culverhouse PF., Simpson RG., Ellis R., Lindley JA., Williams R., Parisini T., Reguera B., Bravo I.,
Zoppoli R., Earnshaw G., McCall H., Smith G. (1996), Automatic Classification of Field Collected
Dinoflagellates by Artificial Neural Network; Mar. Ecol. Prog. Ser. 139(1-3); 281-287.
[9] Das S.K., Bhambri S. (1994), A decision tree approach for selecting between demand based, reorder
and JIT/Kanban methods for material procurement; Production Planning and Control 5(4); 342.
[10] Ercil A. (1993), Classification trees prove useful in non destructive testing of spot weld quality,
Welding Journal, Sept., Issue title: Special emphasis: Rebuilding America’s roads, railways and
bridges 72(9); 59.
[11] Evans B., Fisher D. (1994), Overcoming process delays with decision tree induction; IEEE Expert 9;
60-66.

[12] Famili A. (1994), Use of Decision Tree Induction for Process Optimization and Knowledge
Refinement of an Industrial Process, Artificial Intelligence for Engineering Design, Analysis and
Manufacturing (AI EDAM), Winter 8(1); 63-75.
[13] Fu L. (1991), Rule learning by searching on adapted nets; In Proceedings of the 9th National
Conference on Artificial Intelligence, Anaheim, CA; 590-595.
[14] Ganduri C.G. (2004), Rule driven job shop scheduling derived from neural networks through
extraction; M. S. Thesis, Department of Industrial Engineering, Ohio University, Athens, Ohio.
[15] Gallant S.I. (1988), Connectionist expert systems, Communications of the ACM 31; 152-169.
[16] Garson G.D. (1991), Interpreting neural network connection weights; Artificial Intelligence Expert 6;
47-51.
[17] Gross L., Thiria S., Frouin R. (1999), Applying artificial neural network methodology to ocean color
remote sensing; Ecological Modeling 120; 237-246.
[18] Guilfoyle C. (1986), Ten minutes to lay the foundations; Expert Systems User (Aug.), 16-19.
[19] Guo Y., Dooley K.J. (1994), Distinguishing between mean, variance and autocorrelation changes in
statistical quality control; International Journal of Production Research 33(2); 497-510.
[20] Irani K., Jie C., Fayyad U. M., Zhaogang Q. (1993). Applying machine learning to semiconductor
manufacturing; IEEE Expert, Feb., 8(1); 41-47.
[21] Kennedy D.M. (1993), Decision tree bears fruit; Products Finishing 57(10); 66.
[22] Lee J., Huang Y., Dickman M., Jayawardena A.W. (2003), Neural network modelling of coastal algal
blooms; Ecological modeling 159(2-3); 179-201.
[23] Leech W.J. (1986), A rule based process control method with feedback; Advances in Instrumentation
41; 169-175.
[24] Liepins G., Goeltz R., Rush R. (1990), Machine learning techniques for natural resource data analysis;
AI Applications 4(3); 9-18.
[25] Michie D. (1989), Problems of computer-aided concept formation, In Quinlan, J.R., (Ed). Applications
of Expert Systems Volume 2. Wokingham, UK: Addison-Wesley, 310-333.
[26] Mitchell T. (1997), Machine learning; 1st edition, Computer Science Series, Boston, MA; WCB
McGraw-Hill.
[27] Murthy S.K. (1998), Automatic Construction of Decision Trees from Data: A Multi-Disciplinary
Survey; Data Mining and Knowledge Discovery 2(4); 345-389.
[28] NeuroSolutions (1995), Software developed and distributed by Neurodimension Incorporated;
[http://www.neurosolutions.com/products/ns/].
[29] Olden J.D., Jackson D.A. (2001), Fish-Habitat Relationships in Lakes: Gaining Predictive and
Explanatory Insight by Artificial Neural Networks; Transactions of the American Fisheries Society
130; 878-897.
[30] Olden J.D. (2000), An artificial neural network approach for studying phytoplankton succession;
Hydrobiologia 436; 131-143.

[31] Özesmi S.L., Özesmi U. (1999), An artificial neural network approach to spatial habitat modelling
with interspecific interaction; Ecol. Model. 116; 15–31.
[32] Piramuthu S., Raman N., Shaw M.J. (1994), Learning-based scheduling in a flexible manufacturing
flow line, IEEE Trans. on Engineering Management 41(2); 172-182.
[33] Recknagel Friedrich (2003), Ecological Informatics: Understanding Ecology by Biologically-Inspired
Computation; New York, NY, Springer.
[34] Recknagel F., French M., Harkonen P., Yabunake K. (1997), Ecological Modelling 96; 11-28.
[35] Riddle P., Segal R., Etzioni O. (1994), Representation, Design and Brute-force Induction in a Boeing
manufacturing domain; Applied Artificial Intelligence 8(1); 125-147.
[36] Saito K., Nakano R. (1990), Rule extraction from facts and neural networks; Proceedings of the
International Neural Network Conference; San Diego, CA, 379-382.
[37] Schleiter I.M., Borrchardt D., Wagner R., Dapper T., Schmidt K., Schmidt H., Werner H. (1999),
Modeling water quality, bioindication and population dynamics in lotic ecosystems using neural
networks; Ecological Modelling 120; 271-286.
[38] Sestito S., Dillon T. (1992), Automated knowledge acquisition of rules with continuously valued
attributes; Proceedings of the Twelfth International Conference on Expert Systems and their
Application, Avignon, France, 645-656.
[39] Thrun S. (1995), Extracting rules from artificial neural networks with distributed representations, In
Tesauro G.,Touretzky D., and Leen T. eds. Advances in Neural Information Processing Systems 7;
Cambridge, MA: MIT Press; 505-512.
[40] Tickle A.B., Orlowski M., Diederich J. (1996), DEDEC: a methodology for extracting rule from
trained artificial neural networks; In: Proceedings of the AISB’96 Workshop on Rule Extraction from
Trained Neural Networks, Brighton, UK; 90-102.
[41] Towell G.G., Shavlik J.W. (1993), Extracting refined rules from knowledge-based neural networks;
Machine Learning 13; 71-101.