examples/keyword_extraction-paper_example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Keyword Extraction with Graph-of-Words\n",
    "\n",
    "This notebook reproduces the example from the following paper:\n",
    "\n",
    "[A Graph Degeneracy-based Approach to Keyword Extraction](https://www.aclweb.org/anthology/D16-1191/).\n",
    "     *Tixier, Antoine, Malliaros, Fragkiskos, and Vazirgiannis, Michalis*.\n",
    "     *Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing*.\n",
    "     (EMNLP 2016)\n",
    "     \n",
    "It exemplifies the results of these alternative methods:\n",
    "- Batch keyword extraction based on k-core\n",
    "   + main core\n",
    "   + k-core + dense selection method\n",
    "   + k-core + inflexion selection method\n",
    "- Word-level keyword extraction\n",
    "   + CoreRank + elbow method\n",
    "   + CoreRank + top 33%"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from gowpy.summarization.unsupervised import KcoreKeywordExtractor\n",
    "from gowpy.summarization.unsupervised import CoreRankKeywordExtractor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "Mathematical aspects of computer-aided share trading. We consider\n",
    "problems of statistical analysis of share prices and propose\n",
    "probabilistic characteristics to describe the price series.\n",
    "We discuss three methods of mathematical modelling of price \n",
    "series with given probabilistic characteristics.\n",
    "\"\"\"\n",
    "\n",
    "preprocessed_text = \"\"\"\n",
    "Mathemat aspect computer-aid  share trade  problem \n",
    "statist analysi share price probabilist characterist price  \n",
    "seri method mathemat model  price   seri probabilist\n",
    "characterist\n",
    "\"\"\".strip().lower()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Batch keyword extraction based on k-core"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Main Core"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "extractor_kw = KcoreKeywordExtractor(directed=False, weighted = True, window_size=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('mathemat', 11),\n",
       " ('method', 11),\n",
       " ('model', 11),\n",
       " ('probabilist', 11),\n",
       " ('price', 11),\n",
       " ('characterist', 11),\n",
       " ('seri', 11)]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "extractor_kw.extract(preprocessed_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Density"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "extractor_kw = KcoreKeywordExtractor(directed=False, weighted = True, window_size=8, \n",
    "                                     selection_method='density')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('mathemat', 11),\n",
       " ('price', 11),\n",
       " ('probabilist', 11),\n",
       " ('characterist', 11),\n",
       " ('seri', 11),\n",
       " ('method', 11),\n",
       " ('model', 11),\n",
       " ('share', 10)]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "extractor_kw.extract(preprocessed_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inflexion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "extractor_kw = KcoreKeywordExtractor(directed=False, weighted = True, window_size=8, \n",
    "                                     selection_method='inflexion')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('mathemat', 11),\n",
       " ('price', 11),\n",
       " ('probabilist', 11),\n",
       " ('characterist', 11),\n",
       " ('seri', 11),\n",
       " ('method', 11),\n",
       " ('model', 11),\n",
       " ('share', 10),\n",
       " ('trade', 9),\n",
       " ('problem', 9),\n",
       " ('statist', 9),\n",
       " ('analysi', 9)]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "extractor_kw.extract(preprocessed_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Word-level keyword extraction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CoreRank + elbow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "extractor_kw_cr = CoreRankKeywordExtractor(directed=False, weighted = True, window_size=8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('mathemat', 128),\n",
       " ('price', 120),\n",
       " ('analysi', 119),\n",
       " ('share', 118),\n",
       " ('probabilist', 112),\n",
       " ('characterist', 112),\n",
       " ('statist', 108),\n",
       " ('trade', 97),\n",
       " ('problem', 97),\n",
       " ('seri', 94)]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "extractor_kw_cr.extract(preprocessed_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CoreRank + top 33%"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "extractor_kw_cr = CoreRankKeywordExtractor(directed=False, weighted = True, window_size=8, n=0.33)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('mathemat', 128),\n",
       " ('price', 120),\n",
       " ('analysi', 119),\n",
       " ('share', 118),\n",
       " ('probabilist', 112)]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "extractor_kw_cr.extract(preprocessed_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:gowpy36]",
   "language": "python",
   "name": "conda-env-gowpy36-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}