Step-by-Step Guide to Scraping Tables with Python

Valuable data is often hidden in plain sight across the web. Many websites present information in structured tables such as product listings, sports statistics, and financial summaries. While the data is already organized in rows and columns, manually copying it can be slow and inefficient.
Python provides a much faster way to extract this information. With a few useful libraries, table data can be pulled directly from webpages and converted into clean datasets ready for analysis.
This guide explains how to scrape tables from websites using Python and store the results in CSV files for further use.

Things You Should Prepare

Before getting started, make sure your development environment is properly set up with any modern version of Python installed and running on your system. You will also need three libraries to handle most of the heavy lifting:

• requests: Fetches the HTML content from a webpage.
• Beautiful Soup: Parses HTML so you can extract elements like tables, rows, and cells.
• pandas: Structures the extracted data and saves it in reusable formats like CSV.

Install everything in one go:

pip install requests beautifulsoup4 pandas

Analyze the Website Structure

Every scraper begins with inspecting the webpage and understanding its structure. Skipping this step often leads to unnecessary debugging and wasted time.

Tables in HTML are straightforward:

• <table>: The container.
• <tr>: Each row.
• <th>: Header cells.
• <td>: Data cells.

Many tables include identifying attributes like class or id. Those attributes are golden—they let your script target exactly the table you want.

Retrieve the Webpage

Next, grab the page using Python. requests makes this painless.

import requests

url = "https://www.scrapethissite.com/pages/forms/"
response = requests.get(url)

if response.status_code == 200:
    print("Page fetched successfully!")
    html_content = response.text
else:
    print(f"Failed to fetch the page. Status code: {response.status_code}")
    exit()

One short block of code. Clear outcome. The HTML is now in your Python environment.

Parse the Table Data

Here’s where the scraper does its magic.

Beautiful Soup parses the HTML and lets us locate the table quickly:

from bs4 import BeautifulSoup

soup = BeautifulSoup(html_content, "html.parser")
table = soup.find("table", {"class": "table"})

if not table:
    print("No table found on the page!")
    exit()

Once located, extract headers and rows:

headers = [header.text.strip() for header in table.find_all("th")]

rows = []
for row in table.find_all("tr", class_="team"):
    cells = [cell.text.strip() for cell in row.find_all("td")]
    rows.append(cells)

Breakdown: headers come from <th> tags, rows from <td> tags. Each row becomes a Python list. All rows together form a dataset ready for analysis.

Write the Data to a CSV File

Scraping without saving is wasted effort. Pandas makes exporting data trivial.

import pandas as pd

df = pd.DataFrame(rows, columns=headers)
csv_filename = "scraped_table_data.csv"
df.to_csv(csv_filename, index=False, encoding="utf-8")

print(f"Data saved to {csv_filename}")

Run this, and a CSV appears in your working directory. Open it in Excel, Google Sheets, or feed it straight into a data pipeline. Clean. Organized. Ready to work.

The Full Script

Here’s everything combined into one executable workflow:

import requests
from bs4 import BeautifulSoup
import pandas as pd

url = "https://www.scrapethissite.com/pages/forms/"
response = requests.get(url)

if response.status_code == 200:
    print("Page fetched successfully!")
    html_content = response.text
else:
    print(f"Failed to fetch the page. Status code: {response.status_code}")
    exit()

soup = BeautifulSoup(html_content, "html.parser")
table = soup.find("table", {"class": "table"})

if not table:
    print("No table found on the page!")
    exit()

headers = [header.text.strip() for header in table.find_all("th")]
rows = []
for row in table.find_all("tr", class_="team"):
    cells = [cell.text.strip() for cell in row.find_all("td")]
    rows.append(cells)

df = pd.DataFrame(rows, columns=headers)
csv_filename = "scraped_table_data.csv"
df.to_csv(csv_filename, index=False, encoding="utf-8")

print(df.head())
print(f"Data saved to {csv_filename}")

One script. Endless possibilities.

Final Thoughts

Scraping tables with Python turns structured web content into datasets you can actually use. With the right libraries and a simple workflow, collecting large amounts of data becomes fast and repeatable. Once mastered, this technique makes it far easier to gather, organize, and analyze information directly from the web.