initial commit

Author: Atomu2014

README.md

@@ -1 +1,79 @@
-# KNI
+Tensorflow implementation of the paper ``An End-to-End Neighborhood-based Interaction Model for Knowledge-enhanced Recommendation``.
+This paper is accepted by, and and wins the ``best paper award`` of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data, KDD'19 (DLP-KDD'19), Anchorage, AK, USA.
+
+See our [paper](todo), [poster](./material/kni_poster.pdf), [ppt](./material/kni_presentation.pdf).
+If you have any questions, please contact ``kevinqu16@gmail.com`` directly or open a github issue.
+I will reply ASAP.
+
+### What's KNI model?
+
+In recommender systems, graph-based models build interaction graphs from historical feedbacks (e.g., user ratings) and side information (e.g., film tags, artists), 
+and utilize the rich structural information to boost recommendation performance.
+
+![graph-methods](./material/graph-methods.png)
+
+Due to the complex structures and large scales (of the graphs), it is hard to make predictions directly, 
+thus the existing approaches turn to encoding the meticulous structures into user/item embeddings.
+Since the rich structural information is compressed in only 2 nodes and 1 edge, 
+we concern the valuable local structures are not fully utilized in previous literature,
+which we call the ``early summarization issue``.
+
+After reviewing the existing methods, we derive a general architecture of these methods, 
+and propose ``Neighborhood Interaction`` (NI) model to make predictions from the graph structures directly.
+NI is further integrated with graph neural networks (GNNs) and knowledge graphs (KGs), namely Knowledge-enhanced NI (KNI).
+
+KNI model is not only theoretically more expressive, but also achieves great improvements (1.1% ~ 8.4%) over SOTA models.
+We also provide statistical analysis and case study to explain the early summarization issue and compare different models' behaviors.
+
+For more details, please refer to our [paper](todo), [poster](./material/kni_poster.pdf), [ppt](./material/kni_presentation.pdf).
+
+### Running step-by-step
+
+Requirements:
+- python3
+- numpy
+- ccipy
+- sklearn
+- tqdm
+- tensorflow-gpu
+
+Step 1. Download the data from [https://pan.baidu.com/s/1usnQtW-YodlPUQ1TNrrafw#list/path=%2Fdataset%2Fkg4rs](https://pan.baidu.com/s/1usnQtW-YodlPUQ1TNrrafw#list/path=%2Fdataset%2Fkg4rs) and uncompress ``pickled_data.tar.gz`` under ``./data/``, like:
+
+
+    ./data
+        ab.pkl
+        bc.pkl
+        ml-1m.pkl
+        ml-20m.pkl
+    ./process
+        *.py
+    *.py
+
+The data is processed and pickled by python3, up to 4-hop. 
+According to your experiment settings, you can remove unreachable nodes and edges of the datasets.
+
+Step 2. Run ``train.py`` with default parameters for ``bc`` dataset.
+
+    cd /path/to/code/
+    python3 train.py --dataset=bc --model=ni
+
+After a while, you will see logs like the following (the train/dev scores are disabled for speed concern):
+
+    ...
+    Epoch: 0057 test: auc=0.771917 ll=0.575557 acc=0.706051
+    Epoch: 0058 test: auc=0.772221 ll=0.575257 acc=0.705843
+    Epoch: 0059 test: auc=0.772380 ll=0.575088 acc=0.703685
+    Epoch: 0060 test: auc=0.771758 ll=0.575617 acc=0.704059
+    Epoch: 0061 test: auc=0.771504 ll=0.575559 acc=0.704017
+    ...
+
+The default script will run the same experiment for 5 times with different random seeds. 
+You may find the experiments early stop at 0.772 AUC, 0.706 ACC (+/- 0.002).
+
+Now you achieve the new ``state-of-the-art`` :-) (the most recently reported SOTA is RippleNet, 0.729 AUC, 0.663 ACC).
+
+### Stay Connected!
+
+If you see this paper/data/code helpful or related, please cite our paper with the following BibTeX entry
+
+todo

layers.py

@@ -0,0 +1,161 @@
+from utils import zeros, glorot
+import tensorflow as tf
+
+flags = tf.app.flags
+FLAGS = flags.FLAGS
+
+_LAYER_UIDS = {}
+
+
+def get_layer_uid(layer_name=''):
+    if layer_name not in _LAYER_UIDS:
+        _LAYER_UIDS[layer_name] = 1
+        return 1
+    else:
+        _LAYER_UIDS[layer_name] += 1
+        return _LAYER_UIDS[layer_name]
+
+
+class Layer(object):
+    def __init__(self, name='layer', verbose=True, **kwargs):
+        if not name:
+            layer_name = self.__class__.__name__.lower()
+            name = layer_name + '_' + str(get_layer_uid(layer_name))
+        else:
+            layer_name = name
+            name = layer_name + '_' + str(get_layer_uid(layer_name))
+        self.name = name
+        self.vars = {}
+        self.verbose = verbose
+
+    def _call(self, inputs):
+        return inputs
+
+    def __call__(self, inputs=None):
+        if self.verbose and inputs is not None:
+            if not isinstance(inputs, list):
+                tf.summary.histogram(self.name + '/inputs', inputs)
+            else:
+                for i, x in enumerate(inputs):
+                    tf.summary.histogram(self.name + '/inputs_%d' % i, x)
+        outputs = self._call(inputs)
+        if self.verbose:
+            tf.summary.histogram(self.name + '/outputs', outputs)
+        return outputs
+
+    def _log_vars(self):
+        if self.verbose:
+            for var in self.vars:
+                tf.summary.histogram(self.name + '/vars/' + var, self.vars[var])
+
+
+class UniformSampler(Layer):
+    def __init__(self, name='uniform', verbose=False, adj_list=None):
+        super(UniformSampler, self).__init__(name=name, verbose=verbose)
+        self.adj_list = adj_list
+
+    def _call(self, inputs):
+        ids, n_sample = inputs
+        # len(id) * max_degree
+        neighbors = tf.nn.embedding_lookup(self.adj_list, ids)
+        neighbors = tf.transpose(
+            tf.random_shuffle(
+                tf.transpose(neighbors)))
+        neighbors = neighbors[:, :n_sample]
+        return neighbors
+
+
+class GCNAgg(Layer):
+    def __init__(self, name='gcn_agg', verbose=False, input_dim=None, output_dim=None,
+                 act=tf.nn.relu, weight=True, dropout=0.):
+        super(GCNAgg, self).__init__(name=name, verbose=verbose)
+
+        self.input_dim = input_dim
+        self.output_dim = output_dim
+        self.act = act
+        self.weight = weight
+        self.dropout = dropout
+
+        with tf.variable_scope(self.name):
+            if self.weight:
+                self.vars['weights'] = glorot([input_dim, output_dim], name='weights')
+            self.vars['bias'] = zeros([output_dim], name='bias')
+
+        self._log_vars()
+
+    def _call(self, inputs):
+        # n_sup * k, n_sup * n_sample * k, (n_sup * n_sample)
+        self_vecs, neigh_vecs, n_sample = inputs
+        neigh_vecs = tf.nn.dropout(neigh_vecs, 1 - self.dropout)
+        self_vecs = tf.nn.dropout(self_vecs, 1 - self.dropout)
+
+        hidden = tf.reduce_mean(tf.concat([tf.expand_dims(self_vecs, axis=1), neigh_vecs], axis=1), axis=1)
+        if self.weight:
+            hidden = tf.matmul(hidden, self.vars['weights'])
+        hidden += self.vars['bias']
+        return self.act(hidden)
+
+
+class GATAgg(Layer):
+    def __init__(self, name='gat_agg', verbose=False, input_dim=None, output_dim=None,
+                 act=tf.nn.relu, bias=True, weight=True, dropout=0., atn_type=1, atn_drop=False):
+        super(GATAgg, self).__init__(name=name, verbose=verbose)
+
+        self.input_dim = input_dim
+        self.output_dim = output_dim
+        self.act = act
+        self.bias = bias
+        self.weight = weight
+        self.dropout = dropout
+        self.atn_type = atn_type
+        self.atn_drop = dropout if atn_drop else 0.
+
+        with tf.variable_scope(self.name):
+            if self.weight:
+                self.vars['weights'] = glorot(shape=[input_dim, output_dim], name='weights')
+            else:
+                assert input_dim == output_dim
+
+            self.vars['atn_weights_1'] = glorot([output_dim, 1], name='atn_weights_1')
+            self.vars['atn_weights_2'] = glorot([output_dim, 1], name='atn_weights_2')
+            self.vars['atn_bias_1'] = zeros([1], name='atn_bias_1')
+            self.vars['atn_bias_2'] = zeros([1], name='atn_bias_2')
+
+            if self.bias:
+                self.vars['bias'] = zeros([output_dim], name='bias')
+
+        self._log_vars()
+
+    def _call(self, inputs):
+        # n_sup * k, n_sup * n_sample * k
+        self_vecs, neigh_vecs, n_sample, _ = inputs
+        neigh_vecs = tf.nn.dropout(neigh_vecs, 1 - self.dropout)
+        self_vecs = tf.nn.dropout(self_vecs, 1 - self.dropout)
+
+        if self.weight:
+            self_vecs = tf.matmul(self_vecs, self.vars['weights'])
+            neigh_vecs = tf.reshape(
+                tf.matmul(tf.reshape(neigh_vecs, [-1, self.input_dim]),
+                          self.vars['weights']),
+                [-1, n_sample, self.output_dim])
+
+        # append self_vecs to neigh_vecs
+        neigh_vecs = tf.concat([tf.expand_dims(self_vecs, axis=1), neigh_vecs], axis=1)
+        n_neigh = n_sample + 1
+
+        # n_sup * 1
+        f_1 = tf.matmul(self_vecs, self.vars['atn_weights_1']) + self.vars['atn_bias_1']
+        # n_sup * (n_sample + 1)
+        f_2 = tf.reshape(
+            tf.matmul(tf.reshape(neigh_vecs, [-1, self.output_dim]),
+                      self.vars['atn_weights_2']),
+            [-1, n_neigh]) + self.vars['atn_bias_2']
+        # n_sup * (n_sample + 1)
+        logits = f_1 + f_2
+        scores = tf.nn.dropout(tf.nn.tanh(logits), 1 - self.atn_drop) / FLAGS.temp
+        coefs = tf.nn.softmax(scores)
+        output = tf.reduce_sum(tf.expand_dims(coefs, 2) * neigh_vecs, axis=1)
+
+        if self.bias:
+            output += self.vars['bias']
+        return self.act(output)