Cognitive Science and Neuroscience applications often consists of modeling some human (or animal) data in an interesting behavioral task. Here you should be familiar with the biological, psychological, or linguistic phenomenon, and the creative part is to come up with the architecture that will match that. A good model will lead to predictions that go beyond what you trained it to do, about the architecture or impairments or behavior under different conditions. In other words, by using the neural network, you will learn something about the phenomenon you are modeling.
A third category of projects concerns the neural network techniques themselves. You'll come up with some idea of how to make the network learn faster, or generalize better, or solve harder or new kinds of problems. You will implement your technique and compare the performance with prevailing other algorithms (other neural nets and perhaps other learning algorithms or standard ways of solving the problem). Note that your idea may not work out in the end, but that's ok: not all ideas do. If we decided that it was worth checking out and you did a thorough study showing it doesn't work, that's a perfectly good project for this class.
Your project can also be a theoretical one, where you prove something about the existing algorithms, or an algorithm that you proposed. For example, under what conditions the learning will converge, or what kinds of problems it can solve. Often such a project requires that you have good command of the mathematical techniques already; most of the work will be in identifying a good problem. Techniques often used in NN include statistical mechanics, nonlinear optimization, regularization, probability theory, nonlinear dynamics, etc.
As you are trying to define a project (even before your topic talk), talk to Risto; he'll give you feedback on the feasibility and scope of the project, and you'll set some concrete goals for it. He may also be able to point you to the relevant literature and tools (e.g. simulator code).
A couple of weeks before the project talks begin, you should turn in (by email) your final project proposal to Risto; see the class schedule for the due date of project proposals.
You should have completed your project by the time you are giving your project presentation to the class. At the very least, you should have your model implemented and have preliminary results. A talk where you just talk about what you are going to do is not very useful. For the remainder of the class, you can then obtain more substantial results (i.e. from multiple runs), and write up the project in the project paper.