How do I use soup.find_all() and soup.find() in BeautifulSoup?
soup.find_all() returns a list of every matching element. soup.find() returns only the first match, or None if nothing matches. Pass a tag name and a class_ argument (with the underscore, because class is a reserved word in Python) to filter what you want.
products = soup.find_all('div', class_='product')
for p in products:
print(p.get_text(strip=True))find_all() always returns a ResultSet (behaves like a list). If nothing matches, you get [], not None, so iterating is always safe. Add limit=N to cap the number of results.
Use find() when you expect exactly one element, like a page title or a unique container.
title = soup.find('h1')
if title:
print(title.get_text(strip=True))Always check for None after find(). Calling .get_text() or .find() on a missing tag raises AttributeError, which is the most common BeautifulSoup bug in production scrapers.
Related articles
All articles →How to Use Beautiful Soup for Web Scraping in Python
Parse HTML with Python's Beautiful Soup. Find elements, extract text and attributes, handle elements that are missing, and build a full scraper.
XPath for Web Scraping, from Syntax to Selenium
XPath cheat sheet for web scraping. Syntax, absolute and relative paths, operators, every axis with a working example, and the same queries running in Selenium.
Best Ways to Find All URLs on Any Website
Five ways to list every URL on a domain, measured across nine sites. Four of them publish no sitemap at all.
Hand BeautifulSoup the right HTML
HasData fetches blocked, rendered, and rate-limited pages, so BS4 always sees clean input.