My degree project was related to neural networks and genetic algorithms, and I have used them in many commercial projects since then, including as part of a larger computer vision based system where they were used successfully for object classification. Neural networks are a useful tool for solving certain kinds of problems, but there are many pitfalls, and experience is often the best guide to getting the best out of a neural network in a given project.
Using a neural network successfully involves a number of issues such as:
- Choosing a data set which a NN will be capable of learning.
- Performing statistical analysis on the data to remove redundant and co-variant data.
- Scaling or otherwise preparing the data for the NN.
- Choosing a NN model and training methods suitable for the data.
- Training the NN, checking for problems such as early convergence and over-fitting.
- Testing the NN, and validating the final resulting network.
- Assessing the accuracy of the resulting NN, and analysing the strong/weak areas of performance (in some cases, improvements can be made; in other cases at least knowing which areas are strong vs. weak is the next best thing).
I have written several neural networks from scratch. However, these days it is more common to use a commercial (or open source) NN package when working on a project.
My favoured approach is to use the FANN package, which is an open source neural network library provided in C/C++. It is extremely fast, very easy to incorporate into a wider project, and I have used it successfully in a number of projects.
My recent neural network projects include
- Using OpenCV’s own built-in neural network, built a NN which could learn “contour signatures” in a complex computer vision problem. This project may be converted to use the more flexible FANN system shortly.
- Using FANN, built a very successful NN which learned a clinical data set provided by a client, and was able to make predictions on previously unseen data with 100% accuracy (warning: 100% accuracy is unusual, and some error rate should normally be expected with a NN-based solution).
- Wrote a NN (based on FANN) which learned with reasonable accuracy a financial data set, and was able to make useful predictions (to within 5-10%) on previously unseen data points.
- On the above project, also wrote from scratch a k-nearest-neighbours algorithm, tuned by a genetic algorithm (also written from scratch) to validate the output. Error rate was roughly the same.
- For another client, helped compile a Neural Network system that had been provided to them in source code form, delivered an executable along with a Visual Studio project, and also delivered several hours’ training as part of the package.
I am a general “all round” programmer, so as well as the above I am used to interfacing my work with other parts of a larger system, either by incorporating code directly, or communicating with other systems via HTTP, XML, JSON, etc. For example, one of the above projects was incorporated into a client’s website using the NN as the back-end.
I write code (C/C++ usually) and build finished applications for Windows, Linux, and Mac OS X, as well as Android devices including Google Glass, Raspberry Pi, etc.