Merge pull request #205 from nateprewitt/pre-commit

Add pre-commit config

Author: nateprewitt

.github/workflows/lint.yml

@@ -0,0 +1,19 @@
+name: Lint code
+
+on:
+  push:
+  pull_request:
+    branches-ignore: [ master ]
+
+jobs:
+  lint:
+    runs-on: ubuntu-20.04
+
+    steps:
+    - uses: actions/checkout@v2
+    - name: Set up Python 3.9
+      uses: actions/setup-python@v2
+      with:
+        python-version: 3.9
+    - name: Run pre-commit
+      uses: pre-commit/[email protected]

.pre-commit-config.yaml

@@ -0,0 +1,12 @@
+exclude: ^(.changes/|CHANGELOG.rst)
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.0.1
+    hooks:
+    -   id: check-yaml
+    -   id: end-of-file-fixer
+    -   id: trailing-whitespace
+-   repo: https://github.com/pycqa/flake8
+    rev: 3.9.2
+    hooks:
+    -   id: flake8

ACCEPTANCE_TESTS.rst

@@ -95,7 +95,7 @@ Retry Logic
 ~~~~~~~~~~~
 * [x] Retry on connection related errors when downloading data
 * [ ] Compare MD5 to ``ETag`` and retry for mismatches if all following scenarios are met:
-      
+
   * If MD5 is available
   * Response does not have a ``ServerSideEncryption`` header equal to ``aws:kms``
   * Response does not have ``SSECustomerAlgorithm``
@@ -148,7 +148,7 @@ Validation
 Extra Parameters
 ~~~~~~~~~~~~~~~~
 * [ ] Copy multipart and nonmultipart file with any of the following properties:
-  
+
   * [x] ACL's
   * [x] CacheControl
   * [x] ContentDisposition

CHANGELOG.rst

@@ -182,4 +182,3 @@ CHANGELOG
 =====
 
 * feature:manager: Add boto3 s3 transfer logic to package. (`issue 2 <https://github.com/boto/s3transfer/pull/2>`__)
-

CONTRIBUTING.rst

@@ -0,0 +1,56 @@
+Contributing Code
+-----------------
+A good pull request:
+
+-  Is clear.
+-  Works across all supported versions of Python.
+-  Follows the existing style of the code base (see Codestyle section).
+-  Has comments included as needed.
+
+-  A test case that demonstrates the previous flaw that now passes with
+   the included patch, or demonstrates the newly added feature.
+-  If it adds/changes a public API, it must also include documentation
+   for those changes.
+-  Must be appropriately licensed (Apache 2.0).
+
+Reporting An Issue/Feature
+--------------------------
+First, check to see if there's an existing
+`issue <https://github.com/boto/botocore/issues>`__/`pull requests <https://github.com/boto/botocore/pulls>`__ for the bug/feature.
+
+If there isn't an existing issue there, please file an issue *first*. The
+ideal report includes:
+
+-  A description of the problem/suggestion.
+-  How to recreate the bug.
+-  If relevant, including the versions of your:
+
+   -  Python interpreter
+   -  s3transfer
+   -  Optionally of the other dependencies involved (e.g. Botocore)
+
+-  If possible, create a pull request with a (failing) test case
+   demonstrating what's wrong. This makes the process for fixing bugs
+   quicker & gets issues resolved sooner.
+
+Codestyle
+---------
+This project uses flake8 to enforce codstyle requirements. We've codified this
+process using a tool called `pre-commit <https://pre-commit.com/>`__. pre-commit
+allows us to specify a config file with all tools required for code linting,
+and surfaces either a git commit hook, or single command, for enforcing these.
+
+To validate your PR prior to publishing, you can use the following
+`installation guide <https://pre-commit.com/#install>`__ to setup pre-commit.
+
+If you don't want to use the git commit hook, you can run the below command
+to automatically perform the codestyle validation:
+
+.. code-block:: bash
+
+    $ pre-commit run
+
+This will automatically perform simple updates (such as white space clean up)
+and provide a list of any failing flake8 checks. After these are addressed,
+you can commit the changes prior to publishing the PR.
+These checks are also included in our CI setup under the "Lint" workflow which will provide output on Github for anything missed locally.

LICENSE.txt

@@ -200,4 +200,3 @@
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.
-

requirements-dev-lock.txt

@@ -12,6 +12,14 @@ attrs==21.2.0 \
     --hash=sha256:149e90d6d8ac20db7a955ad60cf0e6881a3f20d37096140088356da6c716b0b1 \
     --hash=sha256:ef6aaac3ca6cd92904cdd0d83f629a15f18053ec84e6432106f7a4d04ae4f5fb
     # via pytest
+backports.entry-points-selectable==1.1.0 \
+    --hash=sha256:988468260ec1c196dab6ae1149260e2f5472c9110334e5d51adcb77867361f6a \
+    --hash=sha256:a6d9a871cde5e15b4c4a53e3d43ba890cc6861ec1332c9c2428c92f977192acc
+    # via virtualenv
+cfgv==3.3.1 \
+    --hash=sha256:c6a0883f3917a037485059700b9e75da2464e6c27051014ad85ba6aaa5884426 \
+    --hash=sha256:f5a830efb9ce7a445376bb66ec94c638a9787422f96264c98edc6bdeed8ab736
+    # via pre-commit
 colorama==0.4.4 \
     --hash=sha256:5941b2b48a20143d2267e95b1c2a7603ce057ee39fd88e7329b0c292aa16869b \
     --hash=sha256:9f47eda37229f68eee03b24b9748937c7dc3868f906e8ba69fbcbdd3bc5dc3e2
@@ -72,16 +80,40 @@ coverage==5.5 \
     # via
     #   -r requirements-dev.txt
     #   pytest-cov
+distlib==0.3.3 \
+    --hash=sha256:c8b54e8454e5bf6237cc84c20e8264c3e991e824ef27e8f1e81049867d861e31 \
+    --hash=sha256:d982d0751ff6eaaab5e2ec8e691d949ee80eddf01a62eaa96ddb11531fe16b05
+    # via virtualenv
+filelock==3.3.0 \
+    --hash=sha256:8c7eab13dc442dc249e95158bcc12dec724465919bdc9831fdbf0660f03d1785 \
+    --hash=sha256:bbc6a0382fe8ec4744ecdf6683a2e07f65eb10ff1aff53fc02a202565446cde0
+    # via virtualenv
+identify==2.3.0 \
+    --hash=sha256:d1e82c83d063571bb88087676f81261a4eae913c492dafde184067c584bc7c05 \
+    --hash=sha256:fd08c97f23ceee72784081f1ce5125c8f53a02d3f2716dde79a6ab8f1039fea5
+    # via pre-commit
 importlib-metadata==4.8.1 \
     --hash=sha256:b618b6d2d5ffa2f16add5697cf57a46c76a56229b0ed1c438322e4e95645bd15 \
     --hash=sha256:f284b3e11256ad1e5d03ab86bb2ccd6f5339688ff17a4d797a0fe7df326f23b1
     # via
     #   pluggy
+    #   pre-commit
     #   pytest
+    #   virtualenv
+importlib-resources==5.2.2 \
+    --hash=sha256:2480d8e07d1890056cb53c96e3de44fead9c62f2ba949b0f2e4c4345f4afa977 \
+    --hash=sha256:a65882a4d0fe5fbf702273456ba2ce74fe44892c25e42e057aca526b702a6d4b
+    # via
+    #   pre-commit
+    #   virtualenv
 iniconfig==1.1.1 \
     --hash=sha256:011e24c64b7f47f6ebd835bb12a743f2fbe9a26d4cecaa7f53bc4f35ee9da8b3 \
     --hash=sha256:bc3af051d7d14b2ee5ef9969666def0cd1a000e121eaea580d4a313df4b37f32
     # via pytest
+nodeenv==1.6.0 \
+    --hash=sha256:3ef13ff90291ba2a4a7a4ff9a979b63ffdd00a464dbe04acf0ea6471517a4c2b \
+    --hash=sha256:621e6b7076565ddcacd2db0294c0381e01fd28945ab36bcf00f41c5daf63bef7
+    # via pre-commit
 nose==1.3.7 \
     --hash=sha256:9ff7c6cc443f8c51994b34a667bbcf45afd6d945be7477b52e97516fd17c53ac \
     --hash=sha256:dadcddc0aefbf99eea214e0f1232b94f2fa9bd98fa8353711dacb112bfcbbb2a \
@@ -91,10 +123,18 @@ packaging==21.0 \
     --hash=sha256:7dc96269f53a4ccec5c0670940a4281106dd0bb343f47b7471f779df49c2fbe7 \
     --hash=sha256:c86254f9220d55e31cc94d69bade760f0847da8000def4dfe1c6b872fd14ff14
     # via pytest
+platformdirs==2.4.0 \
+    --hash=sha256:367a5e80b3d04d2428ffa76d33f124cf11e8fff2acdaa9b43d545f5c7d661ef2 \
+    --hash=sha256:8868bbe3c3c80d42f20156f22e7131d2fb321f5bc86a2a345375c6481a67021d
+    # via virtualenv
 pluggy==1.0.0 \
     --hash=sha256:4224373bacce55f955a878bf9cfa763c1e360858e330072059e10bad68531159 \
     --hash=sha256:74134bbf457f031a36d68416e1509f34bd5ccc019f0bcc952c7b909d06b37bd3
     # via pytest
+pre-commit==2.15.0 \
+    --hash=sha256:3c25add78dbdfb6a28a651780d5c311ac40dd17f160eb3954a0c59da40a505a7 \
+    --hash=sha256:a4ed01000afcb484d9eb8d504272e642c4c4099bbad3a6b27e519bd6a3e928a6
+    # via -r requirements-dev.txt
 psutil==4.4.2 \
     --hash=sha256:10fbb631142a3200623f4ab49f8bf82c32b79b8fe179f6056d01da3dfc589da1 \
     --hash=sha256:15aba78f0262d7839702913f5d2ce1e97c89e31456bb26da1a5f9f7d7fe6d336 \
@@ -124,25 +164,67 @@ pytest==6.2.5 \
     # via
     #   -r requirements-dev.txt
     #   pytest-cov
+pyyaml==5.4.1 \
+    --hash=sha256:08682f6b72c722394747bddaf0aa62277e02557c0fd1c42cb853016a38f8dedf \
+    --hash=sha256:0f5f5786c0e09baddcd8b4b45f20a7b5d61a7e7e99846e3c799b05c7c53fa696 \
+    --hash=sha256:129def1b7c1bf22faffd67b8f3724645203b79d8f4cc81f674654d9902cb4393 \
+    --hash=sha256:294db365efa064d00b8d1ef65d8ea2c3426ac366c0c4368d930bf1c5fb497f77 \
+    --hash=sha256:3b2b1824fe7112845700f815ff6a489360226a5609b96ec2190a45e62a9fc922 \
+    --hash=sha256:3bd0e463264cf257d1ffd2e40223b197271046d09dadf73a0fe82b9c1fc385a5 \
+    --hash=sha256:4465124ef1b18d9ace298060f4eccc64b0850899ac4ac53294547536533800c8 \
+    --hash=sha256:49d4cdd9065b9b6e206d0595fee27a96b5dd22618e7520c33204a4a3239d5b10 \
+    --hash=sha256:4e0583d24c881e14342eaf4ec5fbc97f934b999a6828693a99157fde912540cc \
+    --hash=sha256:5accb17103e43963b80e6f837831f38d314a0495500067cb25afab2e8d7a4018 \
+    --hash=sha256:607774cbba28732bfa802b54baa7484215f530991055bb562efbed5b2f20a45e \
+    --hash=sha256:6c78645d400265a062508ae399b60b8c167bf003db364ecb26dcab2bda048253 \
+    --hash=sha256:72a01f726a9c7851ca9bfad6fd09ca4e090a023c00945ea05ba1638c09dc3347 \
+    --hash=sha256:74c1485f7707cf707a7aef42ef6322b8f97921bd89be2ab6317fd782c2d53183 \
+    --hash=sha256:895f61ef02e8fed38159bb70f7e100e00f471eae2bc838cd0f4ebb21e28f8541 \
+    --hash=sha256:8c1be557ee92a20f184922c7b6424e8ab6691788e6d86137c5d93c1a6ec1b8fb \
+    --hash=sha256:bb4191dfc9306777bc594117aee052446b3fa88737cd13b7188d0e7aa8162185 \
+    --hash=sha256:bfb51918d4ff3d77c1c856a9699f8492c612cde32fd3bcd344af9be34999bfdc \
+    --hash=sha256:c20cfa2d49991c8b4147af39859b167664f2ad4561704ee74c1de03318e898db \
+    --hash=sha256:cb333c16912324fd5f769fff6bc5de372e9e7a202247b48870bc251ed40239aa \
+    --hash=sha256:d2d9808ea7b4af864f35ea216be506ecec180628aced0704e34aca0b040ffe46 \
+    --hash=sha256:d483ad4e639292c90170eb6f7783ad19490e7a8defb3e46f97dfe4bacae89122 \
+    --hash=sha256:dd5de0646207f053eb0d6c74ae45ba98c3395a571a2891858e87df7c9b9bd51b \
+    --hash=sha256:e1d4970ea66be07ae37a3c2e48b5ec63f7ba6804bdddfdbd3cfd954d25a82e63 \
+    --hash=sha256:e4fac90784481d221a8e4b1162afa7c47ed953be40d31ab4629ae917510051df \
+    --hash=sha256:fa5ae20527d8e831e8230cbffd9f8fe952815b2b7dae6ffec25318803a7528fc \
+    --hash=sha256:fd7f6999a8070df521b6384004ef42833b9bd62cfee11a09bda1079b4b704247 \
+    --hash=sha256:fdc842473cd33f45ff6bce46aea678a54e3d21f1b61a7750ce3c498eedfe25d6 \
+    --hash=sha256:fe69978f3f768926cfa37b867e3843918e012cf83f680806599ddce33c2c68b0
+    # via pre-commit
+six==1.16.0 \
+    --hash=sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926 \
+    --hash=sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254
+    # via virtualenv
 tabulate==0.7.5 \
     --hash=sha256:9071aacbd97a9a915096c1aaf0dc684ac2672904cd876db5904085d6dac9810e
     # via -r requirements-dev.txt
 toml==0.10.2 \
     --hash=sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b \
     --hash=sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f
     # via
+    #   pre-commit
     #   pytest
     #   pytest-cov
 typing-extensions==3.10.0.2 \
     --hash=sha256:49f75d16ff11f1cd258e1b988ccff82a3ca5570217d7ad8c5f48205dd99a677e \
     --hash=sha256:d8226d10bc02a29bcc81df19a26e56a9647f8b0a6d4a83924139f4a8b01f17b7 \
     --hash=sha256:f1d25edafde516b146ecd0613dabcc61409817af4766fbbcfb8d1ad4ec441a34
     # via importlib-metadata
+virtualenv==20.8.1 \
+    --hash=sha256:10062e34c204b5e4ec5f62e6ef2473f8ba76513a9a617e873f1f8fb4a519d300 \
+    --hash=sha256:bcc17f0b3a29670dd777d6f0755a4c04f28815395bca279cdcb213b97199a6b8
+    # via pre-commit
 wheel==0.37.0 \
     --hash=sha256:21014b2bd93c6d0034b6ba5d35e4eb284340e09d63c59aef6fc14b0f346146fd \
     --hash=sha256:e2ef7239991699e3355d54f8e968a21bb940a1dbf34a4d226741e64462516fad
     # via -r requirements-dev.txt
 zipp==3.5.0 \
     --hash=sha256:957cfda87797e389580cb8b9e3870841ca991e2125350677b2ca83a0e99390a3 \
     --hash=sha256:f5812b1e007e48cff63449a5e9f4e7ebea716b4111f9c4f9a645f91d579bf0c4
-    # via importlib-metadata
+    # via
+    #   importlib-metadata
+    #   importlib-resources

requirements-dev.txt

@@ -3,6 +3,7 @@ tabulate==0.7.5
 nose==1.3.7
 coverage==5.5
 wheel==0.37.0
+pre-commit>=2.3.0
 
 # Pytest specific deps
 pytest==6.2.5

s3transfer/bandwidth.py

@@ -274,7 +274,7 @@ def _release_requested_amt_for_scheduled_request(self, amt, request_token,
         return self._release_requested_amt(amt, time_now)
 
     def _raise_request_exceeded_exception(self, amt, request_token, time_now):
-        allocated_time = amt/float(self._max_rate)
+        allocated_time = amt / float(self._max_rate)
         retry_time = self._consumption_scheduler.schedule_consumption(
             amt, request_token, allocated_time)
         raise RequestExceededException(

s3transfer/compat.py

@@ -98,4 +98,5 @@ def fallocate(fileobj, size):
         fileobj.truncate(size)
 
 
-from multiprocessing.managers import BaseManager
+# Import at end of file to avoid circular dependencies
+from multiprocessing.managers import BaseManager  # noqa: F401,E402