Web Scraping API

Getting Started

 

This Scraper API is designed to simplify data crawling from the specified website. So, a few things to consider before we get started:

 

  • Each request will be performed within 2s. So, it will be better to take sleep for a minimum of 1s before sending another request.
  • You will not be charged for the unsuccessful request. It will occur on roughly 4-5% of requests for hard to scrape websites.
  • If you exceed your plan request limit, the API will respond with an error message.
  • There is no overage allowed on the free plan, if you exceed 100 requests per month on the free plan, you will receive an error message.
  • Each request will return the data in JSON format.
  • We will provide a detailed description of the data types and values used in JSON data after signing up for a plan.

 

 

Usage

 

Scraper API exposes a single API endpoint, simply send a GET request to 'https://scraper.kbsofts.com/api/{apiName}' with two query string parameters, 'api_key' which contains your API key, and 'url' which contains the URL you would like to scrape.

 

  • An example of coupang Scraper API: https://scraper.kbsofts.com/api/coupang?api_key=Coupang&url=https://www.coupang.com/vp/products/58544519?itemId=203176354&vendorItemId=3481680524&isAddedCart=
  • An example of coupang Scraper API response: 
    • 						
{
   "Request":"Success",
   "Message":"Have fun!",
   "productInfo":{
	  "productId":205030200,
	  "productURL":"https://www.coupang.com/vp/products/205030200?itemId=603388136&vendorItemId=4581425966",
	  "title":"Teepie드레스섹시원피스클럽복EW0304",
	  "description":null,
	  "brand":null,
	  "model":null,
	  "origin":null,
	  "tag":[],
	  "originalPrice":26700,
	  "salePrice":26700,
	  "currency":"Won",
	  "deliveryType":"VENDOR_DELIVERY",
	  "deliveryCharge":0,
	  "mainImageList":[
		 "https://image9.coupangcdn.com/image/vendor_inventory/b8d0/07130df3bb0864593172f88e1383bdd35f83cfb3f247152f093489b.png",
		 "https://image8.coupangcdn.com/image/vendor_inventory/2ef3/2940c8a9f618398913294ada4d14aeeb2ac9a37da1b4b8bf5cda2d6.png"
	  ],
	  "detailImageList":[
		 "https://thumbnail9.coupangcdn.com/thumbnails/remote/q89/image/vendor_inventory/9d6a/9d26eb65f708e59bc13dbc85f4199.png",
		 "https://thumbnail9.coupangcdn.com/thumbnails/remote/q89/image/vendor_inventory/cae4/182b38f4a39fbda5814ce476cd25d.png"
	  ],
	  "optionDetails":{
		 "props":[
			{
			   "propName":"색상",
			   "propValues":[
				  {
					 "name":"블랙",
					 "image":"https://image14.coupangcdn.com/image/vendor_inventory/b8d0/07130df3bb0864593172f8.png"
				  }
			   ]
			},
			{
			   "propName":"사이즈",
			   "propValues":[
				  {
					 "name":"S"
				  },
				  {
					 "name":"M"
				  }
			   ]
			}
		 ],
		 "skus":[
			{
			   "optionStock":200,
			   "option_price":26700,
			   "propValue":"블랙;S"
			},
			{
			   "optionStock":199,
			   "option_price":26700,
			   "propValue":"블랙;M"
			}
		 ],
		 "propMap":"색상;사이즈"
	  }
   },
   "apiInfo":{
	  "apiName":"coupang",
	  "apiVersion":"1.0"
   }
}

 

 

 

Account Information

 

If you would like to monitor your account usage and limits programmatically (i.e. how many requests you've made, how many requests have been failed) you may use the 'https://scraper.kbsofts.com/api/account' endpoint, with one query string parameter, api_key which contains your API key.

 

  • An example of Scraper API Account: https://scraper.kbsofts.com/api/account?api_key=Coupang
  • An example of Scraper API Account details: 
    • 						

{
	Request: "Success",
	Message: "Have a Fun!",
	APIInfo: {
		APICallLimit: 500,
		UnusedAPICall: 437,
		FailedAPICall: 2,
		Activation: "2020-09-01 00:00:01",
		Validity: "2020-09-30 23:59:59"
	}
}