Term Projects

An important goal of the class is to do a research project in Neural Networks. It does not have to be very extensive (we only have one semester), but it should be original and address a specific research question. However, it could be a start for a larger project that you can continue after class, even into a dissertation. Some of the most common kinds of projects are outlined below. Many others are possible; talk to Risto if you have something unusual in mind.

Types of Projects

Many projects focus on some application of neural networks, like time series prediction or fault detection or robot control. Here the architecture is pretty standard, but the application is novel, and a lot of thinking goes into how to pose the problem so that it can be solved with neural networks. Expect also to spend some time in getting the training data together. You should also compare the performance to whatever the standard techniques are in the domain.

Cognitive Science and Neuroscience applications often consists of modeling some human (or animal) data in an interesting behavioral task. Here you should be familiar with the biological, psychological, or linguistic phenomenon, and the creative part is to come up with the architecture that will match that. A good model will lead to predictions that go beyond what you trained it to do, about the architecture or impairments or behavior under different conditions. In other words, by using the neural network, you will learn something about the phenomenon you are modeling.

A third category of projects concerns the neural network techniques themselves. You'll come up with some idea of how to make the network learn faster, or generalize better, or solve harder or new kinds of problems. You will implement your technique and compare the performance with prevailing other algorithms (other neural nets and perhaps other learning algorithms or standard ways of solving the problem). Note that your idea may not work out in the end, but that's ok: not all ideas do. If we decided that it was worth checking out and you did a thorough study showing it doesn't work, that's a perfectly good project for this class.

Your project can also be a theoretical one, where you prove something about the existing algorithms, or an algorithm that you proposed. For example, under what conditions the learning will converge, or what kinds of problems it can solve. Often such a project requires that you have good command of the mathematical techniques already; most of the work will be in identifying a good problem. Techniques often used in NN include statistical mechanics, nonlinear optimization, regularization, probability theory, nonlinear dynamics, etc.

Timeline etc.

You should start thinking about possible projects immediately. The topic talk is designed to give you both inspiration and the necessary background: as you are doing the reading and preparing the talk, try to identify potential research problems. Usually you can come up with a number of them that way, and it will be an easy task to continue the topic talk into a project. Your project does not necessarily have to be related to your topic talk though, and although you can do the project together with your topic talk partner, you don't have to.

As you are trying to define a project (even before your topic talk), talk to Risto; he'll give you feedback on the feasibility and scope of the project, and you'll set some concrete goals for it. He may also be able to point you to the relevant literature and tools (e.g. simulator code).

A couple of weeks before the project talks begin, you should turn in (by email) your final project proposal to Risto; see the class schedule for the due date of project proposals.

You should have completed your project by the time you are giving your project presentation to the class. At the very least, you should have your model implemented and have preliminary results. A talk where you just talk about what you are going to do is not very useful. For the remainder of the class, you can then obtain more substantial results (i.e. from multiple runs), and write up the project in the project paper.

Sun Aug 14 21:29:46 CDT 2022