Extracting Project Dependencies and File Header Comments in Python
This guide explains how to automatically collect all third‑party modules used in a Python project—by manually inspecting imports or using tools such as pipreqs, Poetry, or pipdeptree to generate a requirements.txt file—and provides a Python script for extracting the first‑line comments from each .py file and saving them to a text file.
To reproduce a Python project's environment, you need a complete list of third‑party packages. For small projects you can manually examine import statements and record direct dependencies. For larger or more complex codebases, automated tools are recommended.
Using pipreqs : Install the tool with pip install pipreqs and run it in the project root: pipreqs /path/to/your/project --force . It scans the source files, detects imported packages, and creates a requirements.txt containing package names and versions.
Using Poetry : If the project already uses Poetry, the dependency tree is stored in pyproject.toml . Export it to a requirements file with poetry export -f requirements.txt > requirements.txt .
Using pipdeptree : Install with pip install pipdeptree . To list all installed packages and their dependencies, run pipdeptree | awk '/^\w/ {print $1}' | sort -u > requirements.txt . This produces a flat list of top‑level packages.
Note that some methods may only list direct dependencies; to include transitive (indirect) packages, add the --include-indirect option to pipreqs or configure the chosen tool accordingly.
The second part of the article provides a script for extracting the first line comment from every .py file in a directory and writing the filename‑comment pairs to an output text file.
import os
def get_first_line_comments(directory, output_file):
# List all .py files except __init__.py and sort them
python_files = sorted([f for f in os.listdir(directory) if f.endswith('.py') and f != '__init__.py'])
comments_and_files = []
for file in python_files:
filepath = os.path.join(directory, file)
with open(filepath, 'r', encoding='utf-8') as f:
# Read the first line
first_line = f.readline().strip()
# If the first line is a comment, store it
if first_line.startswith('#'):
comment = first_line[1:].strip()
comments_and_files.append((file, comment))
# Write the collected filename‑comment pairs to the output file
with open(output_file, 'w', encoding='utf-8') as out:
for filename, comment in comments_and_files:
out.write(f"{filename}: {comment}\n")
# Example usage
get_first_line_comments('path/to/your/directory', 'output.txt')Test Development Learning Exchange
Test Development Learning Exchange
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.