Raj Nath Sah

Raj Nath Sah

Another developer with interest in Database, Python, Analytics, Automation and Big Data.

Scrape web using Scrapy FrameWork

less than 1 minute read

Scrape Naukri.com job using Scrapy

Scrapy

An open source and collaborative framework for extracting the data you need from websites.

Using scrapy you can scrape data and save it in the any format you wish to. For demo, i have scraped naukri.com and searched python jobs in Hyderabad. Xpath understanding will be handy while defining the selectors. You can setup a path to save scraped files. Scrapy allows you to deploy the job easily using scrapyd, it is very intutive and userfriendly.

Please refer demo github repo for details. In this demo, i have scraped naukri.com for python job and saved the scraped file in json and html format. I have also included steps for deploying using scrapyd. Feel free to add comments or suggestions.

Happy learning.

Leave a comment

You may also enjoy

Python Learning Path - 2


1 minute read

Curated learning path to be a Python developer. Starting from basic to advance concepts and followed by DIY approach. Follow this post and keep learning.

Python Learning Path - 1


2 minute read

Curated learning path to be a Python developer. Starting from basic to advance concepts and followed by DIY approach. Follow this post and keep learning.