Add base vars and sudo check

This commit is contained in:
Benjamin Zimmerman 2022-12-13 14:20:23 +00:00
parent c151fd6910
commit 054f5ad80c
8733 changed files with 137813 additions and 15 deletions

View file

@ -0,0 +1,153 @@
================
Vendoring Policy
================
* Vendored libraries **MUST** not be modified except as required to
successfully vendor them.
* Vendored libraries **MUST** be released copies of libraries available on
PyPI.
* Vendored libraries **MUST** be available under a license that allows
them to be integrated into ``pip``, which is released under the MIT license.
* Vendored libraries **MUST** be accompanied with LICENSE files.
* The versions of libraries vendored in pip **MUST** be reflected in
``pip/_vendor/vendor.txt``.
* Vendored libraries **MUST** function without any build steps such as ``2to3``
or compilation of C code, practically this limits to single source 2.x/3.x and
pure Python.
* Any modifications made to libraries **MUST** be noted in
``pip/_vendor/README.rst`` and their corresponding patches **MUST** be
included ``tools/vendoring/patches``.
* Vendored libraries should have corresponding ``vendored()`` entries in
``pip/_vendor/__init__.py``.
Rationale
=========
Historically pip has not had any dependencies except for ``setuptools`` itself,
choosing instead to implement any functionality it needed to prevent needing
a dependency. However, starting with pip 1.5, we began to replace code that was
implemented inside of pip with reusable libraries from PyPI. This brought the
typical benefits of reusing libraries instead of reinventing the wheel like
higher quality and more battle tested code, centralization of bug fixes
(particularly security sensitive ones), and better/more features for less work.
However, there are several issues with having dependencies in the traditional
way (via ``install_requires``) for pip. These issues are:
**Fragility**
When pip depends on another library to function then if for whatever reason
that library either isn't installed or an incompatible version is installed
then pip ceases to function. This is of course true for all Python
applications, however for every application *except* for pip the way you fix
it is by re-running pip. Obviously, when pip can't run, you can't use pip to
fix pip, so you're left having to manually resolve dependencies and
installing them by hand.
**Making other libraries uninstallable**
One of pip's current dependencies is the ``requests`` library, for which pip
requires a fairly recent version to run. If pip depended on ``requests`` in
the traditional manner, then we'd either have to maintain compatibility with
every ``requests`` version that has ever existed (and ever will), OR allow
pip to render certain versions of ``requests`` uninstallable. (The second
issue, although technically true for any Python application, is magnified by
pip's ubiquity; pip is installed by default in Python, in ``pyvenv``, and in
``virtualenv``.)
**Security**
This might seem puzzling at first glance, since vendoring has a tendency to
complicate updating dependencies for security updates, and that holds true
for pip. However, given the *other* reasons for avoiding dependencies, the
alternative is for pip to reinvent the wheel itself. This is what pip did
historically. It forced pip to re-implement its own HTTPS verification
routines as a workaround for the Python standard library's lack of SSL
validation, which resulted in similar bugs in the validation routine in
``requests`` and ``urllib3``, except that they had to be discovered and
fixed independently. Even though we're vendoring, reusing libraries keeps
pip more secure by relying on the great work of our dependencies, *and*
allowing for faster, easier security fixes by simply pulling in newer
versions of dependencies.
**Bootstrapping**
Currently most popular methods of installing pip rely on pip's
self-contained nature to install pip itself. These tools work by bundling a
copy of pip, adding it to ``sys.path``, and then executing that copy of pip.
This is done instead of implementing a "mini installer" (to reduce
duplication); pip already knows how to install a Python package, and is far
more battle-tested than any "mini installer" could ever possibly be.
Many downstream redistributors have policies against this kind of bundling, and
instead opt to patch the software they distribute to debundle it and make it
rely on the global versions of the software that they already have packaged
(which may have its own patches applied to it). We (the pip team) would prefer
it if pip was *not* debundled in this manner due to the above reasons and
instead we would prefer it if pip would be left intact as it is now. The one
exception to this, is it is acceptable to remove the
``pip/_vendor/requests/cacert.pem`` file provided you ensure that the
``ssl.get_default_verify_paths().cafile`` API returns the correct CA bundle for
your system. This will ensure that pip will use your system provided CA bundle
instead of the copy bundled with pip.
In the longer term, if someone has a *portable* solution to the above problems,
other than the bundling method we currently use, that doesn't add additional
problems that are unreasonable then we would be happy to consider, and possibly
switch to said method. This solution must function correctly across all of the
situation that we expect pip to be used and not mandate some external mechanism
such as OS packages.
Modifications
=============
* ``setuptools`` is completely stripped to only keep ``pkg_resources``.
* ``pkg_resources`` has been modified to import its dependencies from
``pip._vendor``.
* ``packaging`` has been modified to import its dependencies from
``pip._vendor``.
* ``html5lib`` has been modified to import six from ``pip._vendor``, to prefer
importing from ``collections.abc`` instead of ``collections`` and does not
import ``xml.etree.cElementTree`` on Python 3.
* ``CacheControl`` has been modified to import its dependencies from
``pip._vendor``.
* ``requests`` has been modified to import its other dependencies from
``pip._vendor`` and to *not* load ``simplejson`` (all platforms) and
``pyopenssl`` (Windows).
Automatic Vendoring
===================
Vendoring is automated via the `vendoring <https://pypi.org/project/vendoring/>`_ tool from the content of
``pip/_vendor/vendor.txt`` and the different patches in
``tools/vendoring/patches``.
Launch it via ``vendoring sync . -v`` (requires ``vendoring>=0.2.2``).
Debundling
==========
As mentioned in the rationale, we, the pip team, would prefer it if pip was not
debundled (other than optionally ``pip/_vendor/requests/cacert.pem``) and that
pip was left intact. However, if you insist on doing so, we have a
semi-supported method (that we don't test in our CI) and requires a bit of
extra work on your end in order to solve the problems described above.
1. Delete everything in ``pip/_vendor/`` **except** for
``pip/_vendor/__init__.py`` and ``pip/_vendor/vendor.txt``.
2. Generate wheels for each of pip's dependencies (and any of their
dependencies) using your patched copies of these libraries. These must be
placed somewhere on the filesystem that pip can access (``pip/_vendor`` is
the default assumption).
3. Modify ``pip/_vendor/__init__.py`` so that the ``DEBUNDLED`` variable is
``True``.
4. Upon installation, the ``INSTALLER`` file in pip's own ``dist-info``
directory should be set to something other than ``pip``, so that pip
can detect that it wasn't installed using itself.
5. *(optional)* If you've placed the wheels in a location other than
``pip/_vendor/``, then modify ``pip/_vendor/__init__.py`` so that the
``WHEEL_DIR`` variable points to the location you've placed them.
6. *(optional)* Update the ``pip_self_version_check`` logic to use the
appropriate logic for determining the latest available version of pip and
prompt the user with the correct upgrade message.
Note that partial debundling is **NOT** supported. You need to prepare wheels
for all dependencies for successful debundling.

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/78/4f/f2/a0710baba2b0fbe5aa84011c2a4bc72ca6e64d1d7e257fd3875c1c3597

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/36/dd/b4/d0a745a93ab203203d7190814b1b89727d254caff48c7a7afbbd47000b

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/33/a2/18/449b5d6609923c25c248c051074553dcff0c7456d60836d22eb07611b8

View file

@ -0,0 +1 @@
from cachecontrol import *

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/62/37/16/1756bc47b65cf3f820d93995d76f98ccbd4658dfc10ee09bd455adcb47

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/a4/9b/40/694c4eb0c3e7cad2350378ee009917603afc92b7529ec838620dce0448

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/51/11/84/d0aac0f3b41e9021b74863dab6548f4f9ef57594c38cd6be6575f7a437

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/b1/2c/1a/49877ddc821f085538b4e3204a8e9bd8b0adfe0052690523f24b4914e0

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/d5/f7/38/c093fc1d8b75c9c9c95de130e690a97812f60aac71ea0f456f40180d64

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/fa/01/cd/298bda783d243a4e4cef878eaec4a020a52d0ba8ba19f6e6ba01b0784a

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/9d/85/4a/b09b5787a8095ef767d625b2ae1c6f930a50acaf9e2a8311cee8b090a9

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/1f/17/a5/329342a3e758af67e2243c0cde1861466c5462d079b579b51a90004f86

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/90/73/6f/31176deacfd7c2aabff6a266afda2edf060c38c50cc4f3dcc0dc53f0c7

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/09/61/17/de979d20cf6ceb4b2e7f8cd93ed9bf26f5609efa203062bf3a2046e45f

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/bc/00/8a/3bc2e5ceefd95b28d5d45c67d4c0384c653ad0de4ddc64ab0057406364

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/04/51/87/277c90731bd98b37e8f742cb674e13fd9e574825ef168b6ba7b52cd2c7

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/bc/86/b8/8efab8c7f29238b74421e7689275f669760742e8cb0c5578f85db50e7a

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/e4/b5/f4/b89c2435052d612130dda1a61aef5663cc068a977cd6627c946d1dd0ce

View file

@ -0,0 +1 @@
from certifi import *

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/6a/70/a4/bf6b010016d59a64b8ae4ad8dc7f5ef16f1fb453cc2ecd771c5a341131

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/4a/c9/9d/9851e31c263854bb6ac29a7d3ff8ec39c02e1e3fb97395aaa04cfea058

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/d6/4d/c2/afde6f0b1c464460e58eb5b7c0c76965d2f73617f4bb59fe936a9db026

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/bb/77/f1/3d3fbec9e98bbf28ac95046b44196c7d8f55ab7720061e99991a829197

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/80/e1/5d/d331d8971e24aeb2c49fdf367ac3ad9b3ddd8e21b40454838608e5bdc2

View file

@ -0,0 +1 @@
from chardet import *

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/60/95/e9/ffa777dd22839f7801aa845b31c9ed07f3d6bf8a26dc5d2dec8ccc0ef3

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/99/66/5a/5a6bd9921c1f044013f4ed58ea74537cace14fb1478504d302e8dba940

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/0f/fc/ca/e46cb3a15b117acd0790b2738a5b45417d1b2822ceac57bdff10ef3bff

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/90/1c/47/6dd7ad0693deef1ae56fe7bdf748a8b7ae20fde1922dddf6941eff8773

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/df/0a/16/4bad8aac6a282b2ab3e334129e315b2696ba57b834d9d68089b4f0725f

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/19/92/d1/7873fa151467e3786f48ea060b161a984acacf2a7a460390c55782de48

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/29/29/b0/244ae3ca9ca3d1b459982e45e5e33b73c61080b6088d95e29ed64db2d8

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/01/ba/47/19c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/5c/ae/73/aa3506d9ae3ecba78b1d9f13858729e96594add96610bc4dca971cd921

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/55/8a/7f/e9ccb2922e6c1e05c34999d75b8ab5a1e94773772ef40c904d7eeeba0f

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/e3/4c/eb/eb0202670927c72b8b18670838fcaf7bc0d379b0426dbbedb6f9e6a794

View file

@ -0,0 +1 @@
/home/runner/.cache/pip/pool/4d/9e/37/e105fccf306c9d4bcbffcc26e004154d9d9992a10440bfe5370f5ff68c

Some files were not shown because too many files have changed in this diff Show more