Introduction to Neural Networks and Bilinear RESCAL

This exercise is an introduction to deep learning framework pytorch and Bilinear RESCAL. Pytorch is a state of the art deep learning framework very common among researchers. We will use it for this exercise and upcoming ones. There are two notebooks one is an introduction to pytorch and second is an implementation of RESCAL. It is highly recommended to look at pytorch introduction before proceeding with RESCAL exercise.

Exercise 5-a: Introduction to Pytorch

Exercise 5-b: Bilinear RESCAL and Stochastic Gradient Descent (SGD)

Exercise Overview

• Exercise 5-a: Introduction to Deep Learning Framework: Pytorch.
• Exercise 5-b: Bilinear RESCAL and Stochastic Gradient Descent (SGD).

Task

• Task 5-a-1: Linear Regression for Non Linear Data.
• Task 5-a-2: Implement XOR Function.
• Task 5-a-3: Implement MLP for MNIST Dataset.
• Taske 5-b-1: Effect of learning rate and momentum on convergence.
• Task 5-b-2: Parameter Tuning for RESCAL.
• Task 5-b-3: Implement CP and TUCKER.

Requirements

• Python 3.5+
• Pytorch 0.2

References

• Pytorch
• Tsne Visualization
• A Review of Relational Machine Learning for Knowledge Graphs