Complete Python Web Scraping : Real Projects & Modern Tools
- Description
- Curriculum
- FAQ
- Reviews
In today’s data-driven world, web scraping is a powerful tool that enables you to gather data from websites efficiently.
I designed this course to be the most complete web scraping course on Udemy. It is practical and exercise-based, ensuring you learn by doing through exercises and real-life projects.
We’ll start with the basics on bookstoscrape and quotestoscrape (which are designed to be scraped) to help you grasp the fundamentals of web scraping. After learning the basics, we’ll dive deep into web scraping on real websites.
If you’re new to Python, don’t worry, we’ve got an extra section covering Python fundamentals to get you ready for this course.
What You’ll Learn:
-
Requests and BeautifulSoup:
Parse and extract data from HTML using eBay as an example.
-
Selenium:
Automate browser interactions with real projects from IMDb.
-
Scrapy:
Build scalable web scrapers with real-life examples from Flying Tiger and Yelp.
-
Scrapy-Playwright:
Learn how to scrape dynamic websites with Scrapy by integrating Playwright.
Why This Course?
-
Hands-on Learning: The course is packed with exercises and real-life projects to help you apply what you learn immediately.
-
Practical Approach: I will focus on teaching you practical skills that you can use in your own projects.
-
Support for Beginners: An extra section on Python fundamentals ensures that even those new to programming can follow along and succeed.
Join me in this journey to unlock the full potential of web scraping. With practical exercises and real-world examples, you’ll be well-equipped to gather data from the web effectively. Let’s get started!
-
1Section IntroductionVideo lesson
-
2Course MaterialText lesson
-
3Checking If the Website is Static or DynamicVideo lesson
-
4Deciding on the Method to be UsedVideo lesson
-
5Reading and Analyzing HTML CodeVideo lesson
-
6Installing Python and Pycharm (WINDOWS)Video lesson
-
7Installing Python and Pycharm (LINUX)Video lesson
-
8Installing Python and Pycharm (MACOS)Video lesson
-
9Installing Jupyter Notebook on Windows, Linux and MacOS (OPTIONAL)Video lesson
-
10About Real Life Examples...Video lesson
-
11Course SyllabusVideo lesson
-
12Section IntroductionVideo lesson
-
13Installing Libraries and Making the First RequestVideo lesson
-
14Selectors of Beuatiful SoupVideo lesson
-
15Scraping 1 Book's DataVideo lesson
-
16Constructing the Outer Loop and Dealing with PaginationVideo lesson
-
17Constructing the Inner Loop and Scraping Every BookVideo lesson
-
18Saving the Data and Summary of the ProjectVideo lesson
-
19Downloading Images with RequestsVideo lesson
-
20Using Proxies with RequestsVideo lesson
-
21About Proxy AuthenticationText lesson
-
29Section IntroductionVideo lesson
-
30Installing Selenium and ChromedriverVideo lesson
-
31Creating Driver and Opening BrowserVideo lesson
-
32CSS SelectorsVideo lesson
-
33XPATHVideo lesson
-
34Login with SeleniumVideo lesson
-
35Scraping the First PageVideo lesson
-
36Browsing Through PagesVideo lesson
-
37Infinite ScrollVideo lesson
-
38WaitsVideo lesson
-
39ActionsVideo lesson
-
43Section IntroductionVideo lesson
-
44Creating Driver and Opening BrowserVideo lesson
-
45Important NoteText lesson
-
46Adjustment to the Next Video (Automation)Video lesson
-
47Reaching the Page with AutomationVideo lesson
-
48Scrolling to Load All ItemsVideo lesson
-
49Scraping the DataVideo lesson
-
55Section IntroductionVideo lesson
-
56Installing Scrapy, Creating Scrapy Project and SpiderVideo lesson
-
57Scrapy Shell and Scrapy's SelectorsVideo lesson
-
58Constructing Parse MethodVideo lesson
-
59ItemsVideo lesson
-
60Constructing Parse Books MethodVideo lesson
-
61PipelinesVideo lesson
-
62Dropping Items with PipelinesVideo lesson
-
63Saving Data to Excel with PipelinesVideo lesson
-
64Note about SQLiteText lesson
-
65Saving Data to SQLite Database with PipelinesVideo lesson
-
66MiddlewaresVideo lesson
-
67CrawlerVideo lesson
External Links May Contain Affiliate Links read more