Example Topics

Supervised learning

• Transformers for natural language
• Transformers for visual tasks
• Transformers for tabular data
• Diffusion models (for generating images etc.)
• Foundational language models (e.g. GPT-3)
• Foundational visual models (e.g. DALL-E)
• Generative Adversarial Networks
• Neural Turing Machines / Differentiable Neural Computers
• Hyperparameter optimization with gradient descent
• Simplification of deep networks through distillation
• Regularization techniques
• Second-order methods
• ResNet-type architectures
• Reservoir/Liquid State machines
• Variational autoencoders
• LSTMs for sequence processing
• Image recognition with convolutional networks
• Physics-based neural networks
• Bias, fairness, trustworthiness of trained neural networks

Reinforcement learning

• Deep dive to Actor-Critic, DQN methods
• Policy-search methods (e.g. REINFORCE, natural policy gradients)
• Neural Architecture Search with Reinforcement Learning
• Lifelong reinforcement learning

Evolutionary computation

• Neural architecture search with evolution
• Hyperparameter optimization with evolution
• Programming quantum computers with evolution
• Evolution with uncertain evaluations
• Evolution of language
• Evolutionary optimization of quantum computers
• Neutrality, genetic drift, and weak selection*
• Optimizing aphasia treatments*
• Lifelong evolution
• Optimizing network design to maximize fairness*

Computational Neuroscience

• Networks with spiking neural models
• Modeling fMRI, DTI, EEG, MEG experiments
• Lifelong learning
• Avoiding catastrophic forgetting

Cognitive Science

• Forming embeddings of structure with RAAMs
• Modeling cognitive disorders
• Modeling aphasia, dyslexia
• Modeling consciousness

Neural network hardware

• Hardware accelerators for deep learning
• Optical, molecular, atomic neural networks
• Quantum neural networks

Applications

• Natural language understanding
• Document processing
• Speech understanding
• Robotic control
• Stock market prediction*
• Automated driving
• Medical image understanding
• Optimizing medical treatments*

Theory

• Regularization
• Evolutionary optimization
• Super-Turing computing with neural networks
• Advantages of overparameterization
• Effective vector embeddings