Scraper API
===========

.. contents:: :depth: 2

The scraper API extracts relevant information from any web page.

Scraping web pages
------------------

.. code::

   GET /api/1.3/scraper

Parameters
**********

+-----------+-------------------+--------------------------+
| Name      | Type              | Description              |
+===========+===================+==========================+
| ``url``   | String            | Web page URL - Required  |
+-----------+-------------------+--------------------------+

Response
********

The response can contain several fields, but the following are some of the most important ones to take note of:

============= ================ =================================
Name          Type             Description
============= ================ =================================
``title``     String           Page title
``images``    Array of objects Images in web page
``embed``     Object           Embed information from web page
``tags``      Array of strings Tags
============= ================ =================================

Embed
*****

============== ================ ==========================
Name           Type             Description
============== ================ ==========================
``shortcode``  String           Shortcode string
``media_html`` String           HTML markup of shortcode
============== ================ ==========================

Example
-------

The following is an example for a Facebook post link:
``https://www.facebook.com/RebelMouse/posts/2511871082203772``

Request
*******

.. code::

    GET /api/1.3/scraper?url=https%3A%2F%2Fwww.facebook.com%2FRebelMouse%2Fposts%2F2511871082203772

Response
********

.. code:: json

   {
      "body":"It's estimated that 45% of consumers will unfollow a brand on social media if their platform is dominated by self-promotion.\n\nThat's why we loved United's #HerArtHere contest. The campaign blended...",
      "cacheable":false,
      "description":"It's estimated that 45% of consumers will unfollow a brand on social media if their platform is dominated by self-promotion.\n\nThat's why we loved United's #HerArtHere contest. The campaign blended...",
      "extra":{
         "source_video":"False",
         "profile_type":"page"
      },
      "url":"https://www.facebook.com/RebelMouse/posts/2511871082203772",
      "parser":"Facebook Fallback Parser",
      "favicon":"https://static.xx.fbcdn.net/rsrc.php/yz/r/KFyVIAWzntM.ico?_nc_x=Ij3Wp8lg5Kz",
      "headline":"RebelMouse",
      "images":[
         {
            "url":"https://scontent-iad3-1.xx.fbcdn.net/v/t1.0-0/p235x350/67694909_2511871085537105_8463491295971639296_n.jpg?_nc_cat=101&_nc_oc=AQkjQS5YF1B79mJORIH1arGenN8g76H7nlV6ivkc-ampKmWlMjikGd6o6_hvxLukxzI&_nc_ht=scontent-iad3-1.xx&oh=31e2cb3d1f0014b53e95f0ba7c68cf5f&oe=5D9FD0D8",
            "width":525,
            "type":"image",
            "weight":10.96,
            "height":350
         },
         {
            "url":"https://scontent-iad3-1.xx.fbcdn.net/v/t1.0-1/p56x56/10947220_10152607711836479_1379722055746200799_n.png?_nc_cat=1&_nc_oc=AQkSLE6aW7lBszGGj5GRXoa4aQ2K859ZyAah7IHQQbtNCSfr5X1KSuPg2wyIF7GGkME&_nc_ht=scontent-iad3-1.xx&oh=3e18bfd37f08344aaa5a5f37ec2ba6a2&oe=5DEC9A25",
            "width":56,
            "type":"image",
            "weight":10.92,
            "height":56
         },
         {
            "url":"https://scontent-iad3-1.xx.fbcdn.net/v/t1.0-1/p56x56/13076838_1156702941007658_6208331935499835699_n.jpg?_nc_cat=106&_nc_oc=AQkze-aDp8av3-9c1hmurH1jMWa624OUm8IDx3SR3CltD6fyufEjn51zr0n9KGoFT-8&_nc_ht=scontent-iad3-1.xx&oh=ddddadb1ea34e98a975327abd4524c5d&oe=5DDA14D5",
            "width":56,
            "type":"image",
            "weight":10.88,
            "height":56
         },
         {
            "url":"https://scontent-iad3-1.xx.fbcdn.net/v/t1.0-1/p56x56/19904944_1551279964892956_2604710908252532721_n.png?_nc_cat=106&_nc_oc=AQlI9j_PS4Pq8Wr-C49tV-pmkw7TIme4Bc9qUdgEIIO7Fu8NY57pDk03_n6vVW_9PYQ&_nc_ht=scontent-iad3-1.xx&oh=8c7dfd513e424a50792b43b57f79686a&oe=5DE542EB",
            "width":56,
            "type":"image",
            "weight":10.84,
            "height":56
         },
         {
            "url":"https://scontent-iad3-1.xx.fbcdn.net/v/t1.0-1/p50x50/41755189_1993893697334849_3000527795810992128_n.png?_nc_cat=106&_nc_oc=AQlNdUViQqh4oVKvsd1FFpY_JHIE05sv_QDtz2A0vKMyJxZQBLpwm_Wbx9ulkf1aNdA&_nc_ht=scontent-iad3-1.xx&oh=595ef0c9cf3f4d01bf30c137bea0323a&oe=5DCD5D7E",
            "width":50,
            "type":"image",
            "weight":10.8,
            "height":50
         }
      ],
      "title":"RebelMouse",
      "embed":{
         "shortcode":"[facebook https://www.facebook.com/RebelMouse/posts/2511871082203772 expand=1]",
         "shortcode_id":"6OMI041565303846",
         "shortcode_adapter":"facebook",
         "media_html":"<div class=\"rm-shortcode\" data-rm-shortcode-id=\"6OMI041565303846\"><div class=\"fb-post\" data-href=\"https://www.facebook.com/RebelMouse/posts/2511871082203772\"></div></div>"
      },
      "type":"html",
      "tags":[
         "facebook.com"
      ]
   }