Managing Python Project Environments and Dependencies with pip and conda

Initially the author was reluctant to use Python because the language often lacks systematic environment and dependency management, unlike Node.js (npm) or Java (Maven/Gradle). Many Python projects start without declaring dependencies or configuration files, which leads to hidden problems as the codebase grows.

Based on pip

Effective dependency management should allow quick setup of a project’s dependencies, a clear view of third‑party packages and their dependency trees, and easy addition or removal of packages.

Quick environment configuration (pip)

To preview the packages in the current environment, run pip list :

<code>$ pip list
Package    Version
---------- -------------------
certifi    2020.6.20
pip        19.3.1
setuptools 44.0.0.post20200106
wheel      0.36.2</code>

For a fresh Python environment, only these four packages are present.

Install additional packages, e.g., Flask:

<code>$ pip install flask</code>

After installation, pip list shows the new dependencies introduced by Flask:

<code>$ pip list
Package      Version
------------ -------------------
certifi      2020.6.20
click        7.1.2
Flask        1.1.2
itsdangerous 1.1.0
Jinja2       2.11.3
MarkupSafe   1.1.1
pip          19.3.1
setuptools   44.0.0.post20200106
Werkzeug    1.0.1
wheel        0.36.2</code>

Record the exact versions with pip freeze and save them to requirements.txt :

<code>$ pip freeze > requirements.txt
$ cat requirements.txt
certifi==2020.6.20
click==7.1.2
Flask==1.1.2
itsdangerous==1.1.0
Jinja2==2.11.3
MarkupSafe==1.1.1
Werkzeug==1.0.1</code>

Later, recreate the environment elsewhere with:

<code>$ pip install -r requirements.txt</code>

Explicit dependency tree (pipdeptree)

While pip list or pip freeze show installed packages, they do not reveal which package depends on which. Use pipdeptree to visualize the dependency tree:

<code>$ pip install pipdeptree
$ pipdeptree
certifi==2020.6.20
Flask==1.1.2
  - click [required: >=5.1, installed: 7.1.2]
  - itsdangerous [required: >=0.24, installed: 1.1.0]
  - Jinja2 [required: >=2.10.1, installed: 2.11.3]
    - MarkupSafe [required: >=0.23, installed: 1.1.1]
  - Werkzeug [required: >=0.15, installed: 1.0.1]
pipdeptree==2.0.0
  - pip [required: >=6.0.0, installed: 19.3.1]
setuptools==44.0.0.post20200106
wheel==0.36.2</code>

This makes it clear that Jinja2 is a dependency of Flask, preventing accidental removal.

Dependency cleanup (pip‑autoremove)

To remove a package and all of its unused dependencies, install and use pip‑autoremove :

<code>$ pip install flask
$ pip install pip-autoremove
$ pip-autoremove flask -y
$ pipdeptree
certifi==2020.6.20
pip-autoremove==0.9.1
pipdeptree==2.0.0
  - pip [required: >=6.0.0, installed: 19.3.1]
setuptools==44.0.0.post20200106
wheel==0.36.2</code>

The environment is now clean.

Based on conda

While pip works for single‑project environments, Python’s global interpreter makes it hard to isolate multiple projects. Conda provides a cross‑language virtual‑environment solution.

Package, dependency and environment management for any language—Python, R, Ruby, Lua, Scala, Java, JavaScript, C/C++, FORTRAN Conda is an open‑source package and environment manager that runs on Windows, macOS, and Linux. It quickly installs, runs, and updates packages and their dependencies, and can create, save, load, and switch between environments.

Installation

Choose Miniconda (lightweight) over Anaconda (full scientific stack). After installing, the installer adds initialization code to ~/.bashrc :

<code># >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$(/home/zhenping/miniconda3/bin/conda 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/home/zhenping/miniconda3/etc/profile.d/conda.sh" ]; then
        . "/home/zhenping/miniconda3/etc/profile.d/conda.sh"
    else
        export PATH="/home/zhenping/miniconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<</code>

Reload the shell with source ~/.bashrc to enable the conda command. The prompt will show the default (base) environment.

Disable auto‑activation of the base environment if desired:

<code>conda config --set auto_activate_base false</code>

Environment operations

Create and activate a clean Python 2.7 environment named frida :

<code>$ conda create -n frida python=2.7 -y
$ conda activate frida</code>

List all environments:

<code>(frida) $ conda env list
# conda environments:
#
base                     /home/myths/miniconda3
frida               *   /home/myths/miniconda3/envs/frida</code>

Show packages in the current environment:

<code>(frida) $ conda list
# packages in environment at /home/myths/miniconda3/envs/frida:
#
# Name                    Version                   Build  Channel
_libgcc_mutex               0.1                        main
ca-certificates            2021.4.13               h06a4308_1
certifi                    2020.6.20           pyhd3eb1b0_3
... (additional packages) ...</code>

Unlike pip list , conda list also displays non‑Python dependencies.

Deactivate the environment (may need to run twice if nested):

<code>(frida) $ conda deactivate</code>

Dependency export and recreation

Export the current environment to a YAML file:

<code>(frida) $ conda env export > environment.yaml
(frida) $ cat environment.yaml
name: frida
channels:
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - ca-certificates=2021.4.13=h06a4308_1
  - certifi=2020.6.20=pyhd3eb1b0_3
  ... (other packages) ...
prefix: /home/myths/miniconda3/envs/frida</code>

Recreate the environment elsewhere with:

<code>$ conda env create -f environment.yaml</code>

IDE integration

Conda integrates smoothly with many IDEs, allowing the interpreter and packages to be selected directly from the IDE settings.

Some reflections

Can conda manage virtual environments for other languages? Yes – it can create isolated environments for Java, Node, Ruby, and even system tools like curl or wget, making cross‑language projects easier to handle.

How to find packages supported by conda? Use conda search <package> or browse anaconda.org .

Should Python packages be installed with conda or pip? For pure Python packages, pip is recommended; for packages that bundle compiled binaries or span multiple languages (e.g., TensorFlow), conda may be preferable.

References

anaconda‑vs‑miniconda, conda official documentation, pip‑deptree, pip‑autoremove.

For further reading, scan the QR code in the original article to get free Python learning resources.