Introducing DrissionPage: A Python Library for Fast Browser Automation and Requests Integration

Background

When using requests for data collection on sites that require login, developers often face complex challenges such as analyzing network packets, handling JavaScript, captchas, and anti‑scraping measures, which reduces efficiency. Using a browser can bypass many of these issues but at the cost of performance.

Core Capabilities

DrissionPage combines the convenience of browser automation with the speed of requests, offering features like no WebDriver requirement, no need for browser‑specific drivers, faster execution, seamless iframe handling, multi‑tab operations, direct access to browser cache, full‑page screenshots, and shadow‑root processing.

Getting Started

The library provides SessionPage and WebPage objects; SessionPage uses a requests‑like Session to fetch and parse HTML, enabling element searching with a unified API. Below is a simple example demonstrating how to create a SessionPage, open a page, locate h3 elements, extract links, and print their text and URLs.

# Import
from DrissionPage import SessionPage
# Create page object
page = SessionPage()
# Visit webpage
page.get('https://gitee.com/explore/all')
# Find elements
items = page.eles('t:h3')
# Iterate elements
for item in items[:-1]:
    # Get <a> under current <h3>
    lnk = item('tag:a')
    # Print text and href
    print(lnk.text, lnk.link)