# Research

## Notes

### Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning http://arxiv.org/abs/1511.06342

Instead of using a fresh network for each game, this team combined deep multitask reinforcement learning with deep-transfer learning to be able to use the same deep neural network across different types of games. This leads not only to a single instance that can succeed in multiple different games, but to one that also learns new games better and faster because of what it remembers about those other games. For example, it can learn a new tennis video game faster because it already gets the concept — the meaningful abstraction of hitting a ball with a paddle — from when it was playing Pong [Source](http://futureoflife.org/2015/12/29/the-top-a-i-breakthroughs-of-2015/)


### Bridge Correlational Neural Networks for Multilingual Multimodal Representation Learning http://arxiv.org/abs/1510.03519

Joint embeddings to support the confluence of multiple meaningfully related mappings at once, across different modalities and different languages. As these embeddings get more sophisticated and detailed, they can become workhorses for more elaborate AI techniques


### Learning semantic relationships for better action retrieval in images http://web.eecs.umich.edu/~jiadeng/paper/RamanathanEtAl_CVPR2015.pdf

Create a system that learns a meaningful schema of relationships between different types of actions from a set of photographs and a dictionary.


### DEEP MANIFOLD TRAVERSAL: CHANGING LABELS WITH CONVOLUTIONAL FEATURES http://arxiv.org/pdf/1511.06421v1.pdf

 Uses a dimensionality reduction of a deep net’s weights to form a surface of convolutional features that can simply be slid along to meaningfully, automatically, photorealistically alter particular aspects of photographs, e.g., changing people’s facial expressions or their ages, or colorizing photos.


### Expressing an Image Stream with a Sequence of Natural Sentences http://www.cs.cmu.edu/~gunhee/publish/nips15_stream2text.pdf

**Video summarizing** Novel architecture called a coherent recurrent convolutional network, applying it to creating novel and fluid textual stories from sequences of images.


### Open Areas in Deep Learning (by Yoshua Bengio)

Every researcher has their opinion on this, which is a good thing. Here are some I see:

• Unsupervised learning that would really kick ass
• generative models that generate crisp images and sounds over a wide set of variations covering natural images and sounds
• semi-supervised learning that makes a difference even when the labeled dataset is not tiny
• learning a two-way transformation of the data into a space where variables are disentangled (or mostly independent)
• bringing (iterative) inference back in deep learning to handle non-factorial posteriors over the latent variables
• Introducing more reasoning abilities in our models
• Natural language understanding and knowledge representation on a large scale
• Models of really long-term dependencies in sequential data and having learners that discover a hierarchical representation at multiple time scales
• Understand better (and fix) the optimization issues that sometimes arise (for example in unsupervised learning or recurrent nets with long-term dependencies)
• Train models that incorporate planning (playing out what-if scenarios, maybe stochastically via a generative component) as part of the learning procedure (and also to actually take decisions)
• Scaling up reinforcement learning to large action spaces
• Maximum likelihood has known deficiencies (such as a mismatch between training and test conditions) and we need to go around them (maybe ditching maximum likelihood altogether)
• Bridging the gap between deep learning and biology
• Ramping up our theoretical understanding of deep learning (optimization issues being one aspect, but expressive / statistical aspects being also in need of more theory)
• Building specialized hardware that will allow us not just to build consumer products from models trained offline, but maybe more importantly from a scientific point of view, to train much larger models which capture much more knowledge, so as to open the way towards human-level AI
• Many applications which are under-explored, and in particular I would like to see much more work in the area of health (with some specific issues like missing values and being to exploit data from small studies via transfer learning)