Search icon Close icon
Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Checkout

Change country

Modal Close icon
Country selected Country selected
country flag Germany
Country selected
country flag Slovakia
Country selected
country flag Canada
Country selected
country flag Brazil
Country selected
country flag Singapore
Country selected
country flag Hungary
Country selected
country flag Philippines
Country selected
country flag Mexico
Country selected
country flag Thailand
Country selected
country flag Ukraine
Country selected
country flag Luxembourg
Country selected
country flag Estonia
Country selected
country flag Lithuania
Country selected
country flag Norway
Country selected
country flag Chile
Country selected
country flag United States
Country selected
country flag Great Britain
Country selected
country flag India
Country selected
country flag Spain
Country selected
country flag South Korea
Country selected
country flag Ecuador
Country selected
country flag Colombia
Country selected
country flag Taiwan
Country selected
country flag Switzerland
Country selected
country flag Indonesia
Country selected
country flag Cyprus
Country selected
country flag Denmark
Country selected
country flag Finland
Country selected
country flag Poland
Country selected
country flag Malta
Country selected
country flag Czechia
Country selected
country flag New Zealand
Country selected
country flag Austria
Country selected
country flag Turkey
Country selected
country flag France
Country selected
country flag Sweden
Country selected
country flag Italy
Country selected
country flag Egypt
Country selected
country flag Belgium
Country selected
country flag Portugal
Country selected
country flag Slovenia
Country selected
country flag Ireland
Country selected
country flag Romania
Country selected
country flag Greece
Country selected
country flag Argentina
Country selected
country flag Malaysia
Country selected
country flag South Africa
Country selected
country flag Netherlands
Country selected
country flag Bulgaria
Country selected
country flag Latvia
Country selected
country flag Australia
Country selected
country flag Japan
Country selected
country flag Russia
Country selected
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Hands-On Web Scraping with Python

You're reading from   Hands-On Web Scraping with Python Extract quality data from the web using effective Python techniques

Arrow left icon
Product type Paperback
Published in Oct 2023
Publisher Packt
ISBN-13 9781837636211
Length 324 pages
Edition 2nd Edition
Languages
Tools
Concepts
Arrow right icon
Author (1):
Arrow left icon
Anish Chapagain Anish Chapagain
Author Profile Icon Anish Chapagain
Anish Chapagain
LinkedIn
Anish Chapagain is a software engineer with a passion for data science, its processes, and Python programming, which began around 2007. He has been working with web scraping and analysis-related tasks for more than 5 years, and is currently pursuing freelance projects in the web scraping domain. Anish previously worked as a trainer, web/software developer, and as a banker, where he was exposed to data and gained further insights into topics including data analysis, visualization, data mining, information processing, and knowledge discovery. He has an MSc in computer systems from Bangor University (University of Wales), United Kingdom, and an Executive MBA from Himalayan Whitehouse International College, Kathmandu, Nepal.
Read more
See other products by Anish Chapagain
Arrow right icon
View More author details
Toc

Table of Contents (20) Chapters Close

Preface 1. Part 1:Python and Web Scraping
2. Chapter 1: Web Scraping Fundamentals FREE CHAPTER 3. Chapter 2: Python Programming for Data and Web 4. Part 2:Beginning Web Scraping
5. Chapter 3: Searching and Processing Web Documents 6. Chapter 4: Scraping Using PyQuery, a jQuery-Like Library for Python 7. Chapter 5: Scraping the Web with Scrapy and Beautiful Soup 8. Part 3:Advanced Scraping Concepts
9. Chapter 6: Working with the Secure Web 10. Chapter 7: Data Extraction Using Web APIs 11. Chapter 8: Using Selenium to Scrape the Web 12. Chapter 9: Using Regular Expressions and PDFs 13. Part 4:Advanced Data-Related Concepts
14. Chapter 10: Data Mining, Analysis, and Visualization 15. Chapter 11: Machine Learning and Web Scraping 16. Part 5:Conclusion
17. Chapter 12: After Scraping – Next Steps and Data Analysis 18. Index 19. Other Books You May Enjoy

Scraping the Web with Scrapy and Beautiful Soup

In previous chapters, we learned about web scraping-related technologies, data-finding techniques, and using various Python libraries to scrape data from the web.

In this chapter, we will explore and learn practically about two popular Python libraries, Scrapy and Beautiful Soup. Scrapy is a web crawling framework for Python and provides a project-oriented scope for web scraping. Beautiful Soup, on the other hand, deals with document or content parsing. Parsing a document is normally done to effectively traverse and extract content. Apart from this, both libraries are heavily loaded with DOM-related features.

In particular, we will learn about the following topics in this chapter:

  • Web parsing using Python
  • Web scraping using Beautiful Soup
  • Web scraping using Scrapy
  • Deploying a web crawler
lock icon The rest of the chapter is locked
arrow left Previous Section
Section 1 of 8
Next Section arrow right
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Sign up now
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Start free trial
Renews at $19.99/month. Cancel anytime

Authors (1)

Profile icon Anish Chapagain
Anish Chapagain
LinkedIn icon Github icon
Anish Chapagain is a software engineer with a passion for data science, and artificial intelligence, its processes and Python programming, which began around 2007. He has been working with web scraping, data analysis, visualization and reporting-related tasks, projects for more than 10 years, and is also working as freelancer. Anish previously worked as a trainer, web/software developer, team leader, and as a banker, where he was exposed to data and gained further insights into topics like data mining, data analysis, reporting, information processing and knowledge discovery. He has an MSc in computer systems from Bangor University (United Kingdom), and an Executive MBA from Himalayan Whitehouse International College, Kathmandu, Nepal.
Read more
See other products by Anish Chapagain
Banner background image

Personalised recommendations for you

Based on your interests and search pattern
Left arrow icon
Modern Full-Stack React Projects
Modern Full-Stack React Projects
Read more
Full-Stack React Projects is a complete guide to learning full-stack web development, understanding the creation and integration of backend systems, and advancing your career as a frontend developer.
Read more
Jun 2024 16h 52m
Modern Full-Stack React Projects
Modern Full-Stack React Projects
Read more
Full-Stack React Projects is a complete guide to learning full-stack web development, understanding the creation and integration of backend systems, and advancing your career as a frontend developer.
Read more
Jun 2024 16h 52m
Modern Full-Stack React Projects
Modern Full-Stack React Projects
Read more
Full-Stack React Projects is a complete guide to learning full-stack web development, understanding the creation and integration of backend systems, and advancing your career as a frontend developer.
Read more
Jun 2024 16h 52m
Modern Full-Stack React Projects
Modern Full-Stack React Projects
Read more
Full-Stack React Projects is a complete guide to learning full-stack web development, understanding the creation and integration of backend systems, and advancing your career as a frontend developer.
Read more
Jun 2024 16h 52m
Modern Full-Stack React Projects
Modern Full-Stack React Projects
Read more
Full-Stack React Projects is a complete guide to learning full-stack web development, understanding the creation and integration of backend systems, and advancing your career as a frontend developer.
Read more
Jun 2024 16h 52m
Modern Full-Stack React Projects
Modern Full-Stack React Projects
Read more
Full-Stack React Projects is a complete guide to learning full-stack web development, understanding the creation and integration of backend systems, and advancing your career as a frontend developer.
Read more
Jun 2024 16h 52m
Modern Full-Stack React Projects
Modern Full-Stack React Projects
Read more
Full-Stack React Projects is a complete guide to learning full-stack web development, understanding the creation and integration of backend systems, and advancing your career as a frontend developer.
Read more
Jun 2024 16h 52m
Modern Full-Stack React Projects
Modern Full-Stack React Projects
Read more
Full-Stack React Projects is a complete guide to learning full-stack web development, understanding the creation and integration of backend systems, and advancing your career as a frontend developer.
Read more
Jun 2024 16h 52m
Mastering Node.js Web Development
Mastering Node.js Web Development
Read more
Explore Node.js with practical examples that will teach you how to utilize open-source packages for real-world solutions. Gain the skills to develop and deploy server-side applications that enhance your client-side projects.
Read more
Jun 2024 25h 56m
Mastering Node.js Web Development
Mastering Node.js Web Development
Read more
Explore Node.js with practical examples that will teach you how to utilize open-source packages for real-world solutions. Gain the skills to develop and deploy server-side applications that enhance your client-side projects.
Read more
Jun 2024 25h 56m
Mastering Node.js Web Development
Mastering Node.js Web Development
Read more
Explore Node.js with practical examples that will teach you how to utilize open-source packages for real-world solutions. Gain the skills to develop and deploy server-side applications that enhance your client-side projects.
Read more
Jun 2024 25h 56m
Mastering Node.js Web Development
Mastering Node.js Web Development
Read more
Explore Node.js with practical examples that will teach you how to utilize open-source packages for real-world solutions. Gain the skills to develop and deploy server-side applications that enhance your client-side projects.
Read more
Jun 2024 25h 56m
Right arrow icon