Taking a look at Graph attention Networks

Author: gnthibault

SignalProcessingGraph.ipynb

@@ -73,6 +73,17 @@
     "#IFrame(\"doc/SpectralGraphTheory/cbms.pdf\", width=1200, height=800)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Here are two very good references from Petar Veličković :\n",
+    "* [Theoretical Foundations of Graph Neural Networks](https://www.youtube.com/watch?v=uF53xsT7mjc&ab_channel=PetarVeli%C4%8Dkovi%C4%87)\n",
+    "* [Intro to graph neural networks (ML Tech Talks](https://www.youtube.com/watch?v=8owQBFAHw7E&ab_channel=TensorFlow)\n",
+    "* [Recent set of references](https://twitter.com/PetarV_93/status/1306689702020382720)\n",
+    "* [Some introduction to Graph attention network](https://petar-v.com/GAT/)"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -125,12 +136,11 @@
     "| Gradient operator | $(\\nabla f)_{ij} = \\sqrt{a_{ij}}(f_i-f_j)$ | $\\nabla : L^2(\\mathcal{V}) \\rightarrow L^2(\\mathcal{E})$ |\n",
     "| Divergence operator | $(div F)_{i} = \\frac{1}{b_i} \\sum_{j:(i,j) \\in \\mathcal{E}} \\sqrt{a_{ij}}(F_{ji}-F_{ij})$ |$div : L^2(\\mathcal{E}) \\rightarrow L^2(\\mathcal{V})$ |\n",
     "| Gradient Adjoint |$\\nabla^{\\star}F = -div F$ | $\\langle F,\\nabla f \\rangle_{L^2(\\mathcal{E})} = \\langle \\nabla^{\\star}F, f \\rangle_{L^2(\\mathcal{V})} = \\langle -div F, f \\rangle_{L^2(\\mathcal{V})}$ |\n",
-    "| Laplacian operator | $(\\Delta F)_{i} = \\frac{1}{b_i} \\sum_{j:(i,j) \\in \\mathcal{E}} a_{ij}(f_i-f_j)$ | $\\Delta : L^2(\\mathcal{V}) \\rightarrow L^2(\\mathcal{V})$, this one is the $D-A$ |\n",
     "| Weight matrix | $A = (a_{ij})$ | | |\n",
     "| Degree matrix | $D = diag\\left(d_i\\right)$ where $d_i=\\sum_{j\\neq i}a_{ij})$ | | |\n",
-    "| Unnormalized Laplacian | $\\Delta = D - A$ | | |\n",
-    "| Normalized Laplacian | $\\Delta = I - D^{-\\frac{1}{2}} A D^{-\\frac{1}{2}}$ | | |\n",
-    "| Random walk Laplacian | $\\Delta = I - D^{-1} A $ | | |\n",
+    "| (Unnormalized) Laplacian operator | $(\\Delta F)_{i} = \\frac{1}{b_i} \\sum_{j:(i,j) \\in \\mathcal{E}} a_{ij}(f_i-f_j)$ | $\\Delta : L^2(\\mathcal{V}) \\rightarrow L^2(\\mathcal{V})$, this one is $\\Delta = D - A$ |\n",
+    "| Normalized Laplacian | $(\\Delta F)_{i} = \\sum_{j:(i,j) \\in \\mathcal{E}} \\frac{1}{b_i\\sqrt{d_i d_j}} a_{ij}(f_i-f_j)$ | $\\Delta : L^2(\\mathcal{V}) \\rightarrow L^2(\\mathcal{V})$, this one is $\\Delta = I - D^{-\\frac{1}{2}} A D^{-\\frac{1}{2}}$ |\n",
+    "| Random walk Laplacian | $\\Delta = I - D^{-1} A $ |  $\\Delta : L^2(\\mathcal{V}) \\rightarrow L^2(\\mathcal{V})$, this one is $\\Delta = I - D^{-1} A $ |\n",
     "\n",
     "Those definitions are coming from slides from Xavier Bresson."
    ]

SpectralClusteringIntroduction.ipynb

@@ -3,9 +3,7 @@
   {
    "cell_type": "code",
    "execution_count": 1,
-   "metadata": {
-    "collapsed": false
-   },
+   "metadata": {},
    "outputs": [
     {
      "name": "stderr",
@@ -73,9 +71,7 @@
   {
    "cell_type": "code",
    "execution_count": 2,
-   "metadata": {
-    "collapsed": false
-   },
+   "metadata": {},
    "outputs": [
     {
      "data": {
@@ -126,9 +122,7 @@
   {
    "cell_type": "code",
    "execution_count": 3,
-   "metadata": {
-    "collapsed": false
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "#Now we will concatenate those points\n",
@@ -138,9 +132,7 @@
   {
    "cell_type": "code",
    "execution_count": 4,
-   "metadata": {
-    "collapsed": false
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "#Threshold: Below this value we consider that vertices are not connected\n",
@@ -186,9 +178,7 @@
   {
    "cell_type": "code",
    "execution_count": 5,
-   "metadata": {
-    "collapsed": false
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "#Spectral Dimensionality reduction that will be used, must be equal to nb of class\n",
@@ -217,9 +207,7 @@
   {
    "cell_type": "code",
    "execution_count": 6,
-   "metadata": {
-    "collapsed": false
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "NbIterKMean = 250\n",
@@ -263,9 +251,7 @@
   {
    "cell_type": "code",
    "execution_count": 7,
-   "metadata": {
-    "collapsed": false
-   },
+   "metadata": {},
    "outputs": [
     {
      "data": {
@@ -340,9 +326,9 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.0"
+   "version": "3.8.10"
   }
  },
  "nbformat": 4,
- "nbformat_minor": 0
+ "nbformat_minor": 1
 }