web scraping | BestHub

DevOps

Jun 16, 2025 · Artificial Intelligence

Unlock AI’s Real‑World Power: 6 Must‑Have MCP Tools with Hands‑On Code

This article reviews six open‑source MCP servers—Bright Data, Graphiti, GitIngest, Terminal, Code Executor, and MindsDB—showing how each extends large language models with web scraping, long‑term memory, code navigation, command‑line control, sandboxed Python execution, and multi‑source data integration, complete with practical code examples.

AI toolsMCPcode execution

0 likes · 9 min read

Unlock AI’s Real‑World Power: 6 Must‑Have MCP Tools with Hands‑On Code

DataFunTalk

Jun 11, 2025 · Backend Development

Master Modern Web Scraping: From Classic Tools to DeepSeek AI Integration

This article provides a comprehensive overview of web‑scraping technologies, compares popular tools such as requests, BeautifulSoup and Selenium, introduces AI‑assisted crawling with DeepSeek, and walks through practical steps for using BrightData’s platform to collect industry data, complete with ready‑to‑run Python code.

BeautifulSoupBrightDataDeepSeek

0 likes · 13 min read

Master Modern Web Scraping: From Classic Tools to DeepSeek AI Integration

Python Programming Learning Circle

Jun 10, 2025 · Backend Development

How to Scrape Beike Real‑Estate Listings with Python: A Complete Guide

This tutorial walks you through building a Python web‑scraper for Beike (Lianjia) second‑hand property listings, covering session spoofing, dynamic parameters, pagination, multithreaded detail fetching, data cleaning, and exporting results to Excel in a step‑by‑step manner.

BeautifulSoupData ExtractionPython

0 likes · 18 min read

How to Scrape Beike Real‑Estate Listings with Python: A Complete Guide

Python Programming Learning Circle

Jun 7, 2025 · Backend Development

Master Python Web Scraping: From Requests to Selenium and Scrapy

Learn how to efficiently scrape web pages using Python by exploring multiple approaches—including simple requests with BeautifulSoup, fast parsing with lxml, dynamic content extraction with Selenium, and large‑scale crawling with Scrapy—complete with installation steps, code snippets, and detailed explanations.

BeautifulSoupPythonRequests

0 likes · 10 min read

Master Python Web Scraping: From Requests to Selenium and Scrapy

Python Programming Learning Circle

Jun 4, 2025 · Backend Development

How to Scrape JD.com Product Reviews with Python and Save to CSV

This tutorial explains how to use Python to scrape product reviews from JD.com via its AJAX comment API, extract fields such as nickname, score, content, and image count, and save the collected data into a CSV file using the requests and csv modules.

CSVData ExtractionJD.com

0 likes · 7 min read

How to Scrape JD.com Product Reviews with Python and Save to CSV

Python Programming Learning Circle

May 29, 2025 · Big Data

Common Python Web Scraping Techniques for E‑commerce Data Collection

This article introduces ten practical Python-based web scraping methods—including requests, Selenium, Scrapy, Crawley, PySpider, aiohttp, asks, vibora, Pyppeteer, and Fiddler‑plus‑Node reverse engineering—explaining their use cases, advantages, and code examples for efficiently gathering e‑commerce and app data.

PythonRequestsScrapy

0 likes · 8 min read

Common Python Web Scraping Techniques for E‑commerce Data Collection

Python Programming Learning Circle

May 28, 2025 · Backend Development

Using Proxy IPs for Web Scraping with Python: A Practical Guide

This article explains why proxy IPs are essential for reliable web crawling, compares dynamic and static residential proxies, and provides step‑by‑step Python code to scrape product titles, prices and links from Snapdeal while demonstrating how to integrate proxies for improved efficiency and security.

BeautifulSoupData ExtractionPython

0 likes · 12 min read

Using Proxy IPs for Web Scraping with Python: A Practical Guide

php中文网 Courses

May 14, 2025 · Backend Development

Python Advantages for Web Scraping and Core Library Guide

This article outlines Python's advantages for web crawling, introduces core libraries such as Requests, BeautifulSoup, and Scrapy, details a step-by-step development workflow, provides practical code examples for extracting news titles, and highlights important considerations and advanced techniques for robust scraper implementation.

BeautifulSoupData ExtractionPython

0 likes · 5 min read

Python Advantages for Web Scraping and Core Library Guide

Python Programming Learning Circle

May 12, 2025 · Fundamentals

Basic Python Web Scraping Techniques and Tips

This article introduces beginner-friendly Python web‑scraping methods, covering the simplest urllib/requests approach, adding request headers, inspecting network traffic for hidden data, handling dynamically loaded content with Selenium, and provides links to deeper tutorials for each technique.

PythonRequestsSelenium

0 likes · 4 min read

Basic Python Web Scraping Techniques and Tips

Python Programming Learning Circle

Apr 30, 2025 · Backend Development

Python Weather Data Scraping, CSV Export, and Visualization Using Requests, BeautifulSoup, Pandas, and Matplotlib

This article demonstrates how to use Python's requests and BeautifulSoup libraries to scrape current and 14‑day weather data from China Weather, store the results in CSV files, and perform comprehensive visual analysis—including temperature, humidity, AQI, wind direction, and forecast charts—using pandas, numpy, and matplotlib.

MatplotlibPythonWeather Analysis

0 likes · 26 min read

Python Weather Data Scraping, CSV Export, and Visualization Using Requests, BeautifulSoup, Pandas, and Matplotlib

Spring Full-Stack Practical Cases

Apr 25, 2025 · Backend Development

Master jsoup: Real‑World Spring Boot 3 Examples for HTML Parsing

This tutorial walks through practical jsoup usage within Spring Boot 3, covering dependency setup, parsing HTML from strings, fragments, URLs or files, extracting titles, links, images, applying CSS selectors, modifying elements, and sanitizing content to prevent XSS attacks.

HTML parsingJavaJsoup

0 likes · 10 min read

Master jsoup: Real‑World Spring Boot 3 Examples for HTML Parsing

Code Mala Tang

Apr 19, 2025 · Fundamentals

Master HTML Parsing in Python: BeautifulSoup, lxml, and html.parser Compared

Learn why HTML parsing is essential for web scraping, explore three popular Python libraries—BeautifulSoup, lxml, and the built‑in html.parser—covering installation, core usage, advanced techniques, and a comparative analysis to help you choose the right tool for your project.

BeautifulSoupHTML parsingPython

0 likes · 11 min read

Master HTML Parsing in Python: BeautifulSoup, lxml, and html.parser Compared

Python Programming Learning Circle

Apr 1, 2025 · Backend Development

Implementing Local QR Code Login for QQ Music Using Python

This tutorial walks through building a Python script that fetches a QQ Music QR code, displays it locally, monitors its status, extracts required encrypted parameters from cookies, and completes the login process by repeatedly polling the authentication endpoint to obtain a session.

PythonQR LoginRequests

0 likes · 13 min read

Implementing Local QR Code Login for QQ Music Using Python

Python Programming Learning Circle

Mar 25, 2025 · Backend Development

Python Script for Scraping and Converting Anime Images from Huashi6

This article explains how to use Python's requests, BeautifulSoup, and Pillow libraries to crawl high‑resolution anime pictures from the Huashi6 website, extract their webp URLs via regex, download them, and convert the images to the more common png format for personal use.

BeautifulSoupRequestsimage-processing

0 likes · 8 min read

Python Script for Scraping and Converting Anime Images from Huashi6

Python Programming Learning Circle

Mar 25, 2025 · Backend Development

Comprehensive Python Guide to Download Files from the Web, S3, and Other Sources

This tutorial walks through multiple Python techniques for downloading regular files, web pages, Amazon S3 objects, and other resources, covering basic requests, wget, handling redirects, chunked large‑file downloads, parallel downloads, progress bars, urllib, urllib3, proxy usage, boto3 for S3, and asynchronous downloads with asyncio.

PythonRequestsasyncio

0 likes · 8 min read

Comprehensive Python Guide to Download Files from the Web, S3, and Other Sources

Rare Earth Juejin Tech Community

Mar 1, 2025 · Artificial Intelligence

Predicting Movie Box Office with Playwright Data Scraping and DeepSeek AI

This article demonstrates how to combine Playwright web‑scraping of multiple Chinese movie platforms with the DeepSeek AI model to automatically collect data and generate a scientific prediction of the box‑office revenue for the film "Ne Zha 2".

AI predictionDeepSeekJavaScript

0 likes · 12 min read

Predicting Movie Box Office with Playwright Data Scraping and DeepSeek AI

DataFunSummit

Feb 25, 2025 · Artificial Intelligence

Collecting High-Quality LLM Training Data and Custom Model Training Guide

This article explains what constitutes high‑quality LLM training data, why large datasets are essential, outlines the step‑by‑step process for collecting, preprocessing, and fine‑tuning models, and highlights the best data sources—including web content, books, code repositories, and news—while noting available free datasets.

AILLMdata collection

0 likes · 9 min read

Collecting High-Quality LLM Training Data and Custom Model Training Guide

DataFunSummit

Feb 13, 2025 · Big Data

E‑commerce Data Scraping: Fundamentals, Tools, Python Scripts, and Challenges

This tutorial explains e‑commerce web scraping fundamentals, covering definitions, tool types, data categories, step‑by‑step Python script creation with Requests, BeautifulSoup, and Selenium, provides sample code for Amazon, Walmart, and eBay, discusses challenges like dynamic pages and anti‑scraping measures, and recommends using specialized scraping APIs.

BeautifulSoupBright DataData Extraction

0 likes · 15 min read

E‑commerce Data Scraping: Fundamentals, Tools, Python Scripts, and Challenges

Python Programming Learning Circle

Feb 7, 2025 · Backend Development

Python Web Crawling Tutorial: From Basics to a Full‑Scale Novel Scraper

This article introduces web crawling fundamentals, demonstrates how to inspect HTML elements, walks through simple examples using urllib, requests, and BeautifulSoup, and culminates in a complete Python script that extracts chapter links and contents from a novel website, saving them to a text file.

BeautifulSoupRequestsTutorial

0 likes · 19 min read

Python Web Crawling Tutorial: From Basics to a Full‑Scale Novel Scraper

Python Programming Learning Circle

Jan 23, 2025 · Artificial Intelligence

Python Script for Scraping Zhihu “Beauty” Topic Images with Baidu AI Face Detection

This tutorial explains how to use Python 3 with Requests, lxml, and Baidu's AipFace SDK to crawl images from Zhihu's "美女" topic, filter them by face detection, gender, authenticity, and beauty score, and store the qualified pictures locally.

Image ProcessingZhihubaidu-ai

0 likes · 4 min read

Python Script for Scraping Zhihu “Beauty” Topic Images with Baidu AI Face Detection