installing packages

pip install <package-name>==<version>  # version is optional
pip install -e git+REPO  # To install a package from git repository:

pip supports installing over git, git+https and git+ssh, Here are the supported forms:

[-e] git+git://git.myproject.org/MyProject#egg=MyProject
[-e] git+https://git.myproject.org/MyProject#egg=MyProject
[-e] git+ssh://git.myproject.org/MyProject#egg=MyProject  # for private repo, only this works
[-e] git+git@git.myproject.org:MyProject#egg=MyProject

# Passing branch names, a commit hash or a tag name is possible like so:

[-e] git://git.myproject.org/MyProject.git@master#egg=MyProject
[-e] git://git.myproject.org/MyProject.git@v1.0#egg=MyProject
[-e] git://git.myproject.org/MyProject.git@da39a3ee5e6b4b0d3255bfef95601890afd80709#egg=MyProject

If you add the -e(editable) option, then the version info is saved when freezing, which is exactly what you need.

upgrade a package:

pip install package-name -U/--upgrade

requirements file

Save current dependencies to a requirement file:

pip freeze > requirements.txt

Install from a requirement file

pip install -r requirements.txt

使用国内源

• 阿里云 https://mirrors.aliyun.com/pypi/simple/
• 中国科技大学 https://pypi.mirrors.ustc.edu.cn/simple/
• 豆瓣 http://pypi.douban.com/simple/
• 清华大学 https://pypi.tuna.tsinghua.edu.cn/simple/
• 中国科学技术大学 http://pypi.mirrors.ustc.edu.cn/simple/

可以使用 -i 指定源：

pip install flask -i https://pypi.tuna.tsinghua.edu.cn/simple/

也可以修改 ~/.pip/pip.conf（没有就创建一个），内容如下：

[global]
https://pypi.tuna.tsinghua.edu.cn/simple/

setuptools

在 setup.py 中定义包的元数据

from setuptools import setup

setup(
  name='demo',
  author='deephub',
  version='0.1',
  install_requires=[
      'pandas',
      'numpy',
      'matplotlib',
  ],
  # ... more options/metadata
)

或者使用 setup.cfg + setup.py

# setup.cfg
[metadata]
name = demo
author = deephub
version = 0.1
[options]
install_requires =
  pandas
  numpy
  matplotlib

# setup.py
from setuptools import setup

if __name__ == "__main__":
  setup()

分离开发和运行依赖

pip 在依赖管理方面还是非常拉跨的，甚至都不支持分离开发依赖和运行时依赖。这就导致调试时 用到的 ipdb 等等还会出现在 requirements.txt 中。我做了一个简单的 work-around, 先安装 开发依赖并保存到 dev-requirements.txt 中，然后再安装运行时依赖，每次 freeze 的时候把 开发依赖剔除掉就好了。开发依赖变动较少，这种方法基本可以解决问题。

还有一个神器：pip install pipdeptree, 可以把依赖显示成树形，这样只要保存根部的依赖即可。

先装开发依赖：

pip install ipython ipdb black isort pipdeptree

保存到 dev-requirements.txt 中：

pipdeptree --json-tree | jq -r '.[] | .package_name + "==" + .installed_version'

再安装依赖，比如 pyyaml redis, 然后保存的时候过滤掉开发依赖：

pipdeptree --json-tree | \
jq -r '.[] | .package_name + "==" + .installed_version' | \
grep -vxFf dev-requirements.txt > requirements.txt

还可以写一个 Makefile:

freeze:
    pipdeptree -f -w silence | grep -E '^\w' | grep -vxFf dev-requirements.txt > requirements.txt

这样只要 make freeze 就好啦

卸载的时候删除所有依赖

可以使用 pip-autoremove 包

pip install pip-autoremove
# requests 有严重的内存泄露，而且代码非常不优雅，强烈建议使用 httpx 代替。
pip-autoremove requrests -y

还可以使用 pipdeptree -r 来显示出一个库被哪些库使用了。

$ pipdeptree -r -p six

six==1.16.0
  - bleach==4.1.0 [requires: six>=1.9.0]
    - nbconvert==6.4.5 [requires: bleach]
      - jupyter-server==1.16.0 [requires: nbconvert>=6.4.4]
        - jupyterlab==3.3.2 [requires: jupyter-server~=1.4]
        - jupyterlab-server==2.12.0 [requires: jupyter-server~=1.8]
          - jupyterlab==3.3.2 [requires: jupyterlab-server~=2.10]
        - nbclassic==0.3.7 [requires: jupyter-server>=1.8]
          - jupyterlab==3.3.2 [requires: nbclassic~=0.2]
        - notebook-shim==0.1.0 [requires: jupyter-server~=1.8]
          - nbclassic==0.3.7 [requires: notebook-shim>=0.1.0]
            - jupyterlab==3.3.2 [requires: nbclassic~=0.2]
      - notebook==6.4.10 [requires: nbconvert>=5]
        - nbclassic==0.3.7 [requires: notebook<7]
          - jupyterlab==3.3.2 [requires: nbclassic~=0.2]

参考

1. http://crazygit.wiseturtles.com/2018/01/08/pipenv-tour/
2. https://stackoverflow.com/questions/5900201/pip-freeze-without-dependencies-of-installed-packages
3. https://github.com/naiquevin/pipdeptree
4. https://mp.weixin.qq.com/s/ICLmK8HX9hkgh5nrDy6G9g