$ ls ~yifei/notes/

Python 中的包管理工具

Posted on:

Last modified:

installing packages

pip install <package-name>==<version>  # version is optional
pip install -e git+REPO  # To install a package from git repository:

pip supports installing over git, git+https and git+ssh, Here are the supported forms:

[-e] git+git://git.myproject.org/MyProject#egg=MyProject
[-e] git+https://git.myproject.org/MyProject#egg=MyProject
[-e] git+ssh://git.myproject.org/MyProject#egg=MyProject  # for private repo, only this works
[-e] git+git@git.myproject.org:MyProject#egg=MyProject

# Passing branch names, a commit hash or a tag name is possible like so:

[-e] git://git.myproject.org/MyProject.git@master#egg=MyProject
[-e] git://git.myproject.org/MyProject.git@v1.0#egg=MyProject
[-e] git://git.myproject.org/MyProject.git@da39a3ee5e6b4b0d3255bfef95601890afd80709#egg=MyProject

If you add the -e(editable) option, then the version info is saved when freezing, which is exactly what you need.

upgrade a package:

pip install package-name -U/--upgrade

requirements file

Save current dependencies to a requirement file:

pip freeze > requirements.txt

Install from a requirement file

pip install -r requirements.txt

使用国内源

可以使用 -i 指定源:

pip install flask -i https://pypi.tuna.tsinghua.edu.cn/simple/

也可以修改 ~/.pip/pip.conf(没有就创建一个),内容如下:

[global]
https://pypi.tuna.tsinghua.edu.cn/simple/

setuptools

在 setup.py 中定义包的元数据

from setuptools import setup

setup(
  name='demo',
  author='deephub',
  version='0.1',
  install_requires=[
      'pandas',
      'numpy',
      'matplotlib',
  ],
  # ... more options/metadata
)

或者使用 setup.cfg + setup.py

# setup.cfg
[metadata]
name = demo
author = deephub
version = 0.1
[options]
install_requires =
  pandas
  numpy
  matplotlib

# setup.py
from setuptools import setup

if __name__ == "__main__":
  setup()

分离开发和运行依赖

pip 在依赖管理方面还是非常拉跨的,甚至都不支持分离开发依赖和运行时依赖。这就导致调试时 用到的 ipdb 等等还会出现在 requirements.txt 中。我做了一个简单的 work-around, 先安装 开发依赖并保存到 dev-requirements.txt 中,然后再安装运行时依赖,每次 freeze 的时候把 开发依赖剔除掉就好了。开发依赖变动较少,这种方法基本可以解决问题。

还有一个神器:pip install pipdeptree, 可以把依赖显示成树形,这样只要保存根部的依赖即可。

先装开发依赖:

pip install ipython ipdb black isort pipdeptree

保存到 dev-requirements.txt 中:

pipdeptree --json-tree | jq -r '.[] | .package_name + "==" + .installed_version'

再安装依赖,比如 pyyaml redis, 然后保存的时候过滤掉开发依赖:

pipdeptree --json-tree | \
jq -r '.[] | .package_name + "==" + .installed_version' | \
grep -vxFf dev-requirements.txt > requirements.txt

还可以写一个 Makefile:

freeze:
    pipdeptree -f -w silence | grep -E '^\w' | grep -vxFf dev-requirements.txt > requirements.txt

这样只要 make freeze 就好啦

卸载的时候删除所有依赖

可以使用 pip-autoremove

pip install pip-autoremove
# requests 有严重的内存泄露,而且代码非常不优雅,强烈建议使用 httpx 代替。
pip-autoremove requrests -y

还可以使用 pipdeptree -r 来显示出一个库被哪些库使用了。

$ pipdeptree -r -p six

six==1.16.0
  - bleach==4.1.0 [requires: six>=1.9.0]
    - nbconvert==6.4.5 [requires: bleach]
      - jupyter-server==1.16.0 [requires: nbconvert>=6.4.4]
        - jupyterlab==3.3.2 [requires: jupyter-server~=1.4]
        - jupyterlab-server==2.12.0 [requires: jupyter-server~=1.8]
          - jupyterlab==3.3.2 [requires: jupyterlab-server~=2.10]
        - nbclassic==0.3.7 [requires: jupyter-server>=1.8]
          - jupyterlab==3.3.2 [requires: nbclassic~=0.2]
        - notebook-shim==0.1.0 [requires: jupyter-server~=1.8]
          - nbclassic==0.3.7 [requires: notebook-shim>=0.1.0]
            - jupyterlab==3.3.2 [requires: nbclassic~=0.2]
      - notebook==6.4.10 [requires: nbconvert>=5]
        - nbclassic==0.3.7 [requires: notebook<7]
          - jupyterlab==3.3.2 [requires: nbclassic~=0.2]

参考

  1. http://crazygit.wiseturtles.com/2018/01/08/pipenv-tour/
  2. https://stackoverflow.com/questions/5900201/pip-freeze-without-dependencies-of-installed-packages
  3. https://github.com/naiquevin/pipdeptree
  4. https://mp.weixin.qq.com/s/ICLmK8HX9hkgh5nrDy6G9g
WeChat Qr Code

© 2016-2022 Yifei Kong. Powered by ynotes

All contents are under the CC-BY-NC-SA license, if not otherwise specified.

Opinions expressed here are solely my own and do not express the views or opinions of my employer.

友情链接: MySQL 教程站