MNT: Add pre-commit hooks and checks (#138)

Author: greglucas

.github/workflows/tests.yml

@@ -18,7 +18,7 @@ jobs:
       - name: Install ruff
         run: pip install . ruff
       - name: Run ruff
-        run: ruff check space_packet_parser
+        run: ruff check
 
   ci-tests:
     runs-on: ${{ matrix.os }}
@@ -43,12 +43,12 @@ jobs:
         uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python-version }}
-      
+
       - name: Install dependencies
         run: |
           python -m pip install --upgrade pip
           pip install ".[test,xarray]"
-      
+
       - name: Testing
         run: |
           pytest --color=yes --cov

.pre-commit-config.yaml

@@ -0,0 +1,28 @@
+ci:
+  autofix_prs: false
+  autoupdate_schedule: 'quarterly'
+  skip: [poetry-lock]
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v4.5.0
+    hooks:
+    - id: check-added-large-files
+      args: ['--maxkb=1000']
+    - id: detect-aws-credentials
+      args: [--allow-missing-credentials]
+    - id: detect-private-key
+    - id: mixed-line-ending
+    - id: trailing-whitespace
+    - id: no-commit-to-branch
+      args: [--branch, main, --branch, dev]
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: 'v0.9.6'
+    hooks:
+    - id: ruff
+      args: [--fix]
+    # - id: ruff-format
+  - repo: https://github.com/codespell-project/codespell
+    rev: v2.2.6
+    hooks:
+      - id: codespell
+        files: ^.*\.(py|md|rst|yml)$

README.md

@@ -6,8 +6,8 @@
 
 Documentation: [https://space-packet-parser.readthedocs.io/en/latest/](https://space-packet-parser.readthedocs.io/en/latest/)
 
-Space Packet Parser is a package for decoding CCSDS telemetry packets according to an XTCE packet structure definition. 
-It is based on the UML model of the XTCE spec and aims to support all but the most esoteric elements of the 
+Space Packet Parser is a package for decoding CCSDS telemetry packets according to an XTCE packet structure definition.
+It is based on the UML model of the XTCE spec and aims to support all but the most esoteric elements of the
 XTCE telemetry packet specification.
 
 Resources:

docs/source/benchmarking.md

@@ -2,11 +2,11 @@
 
 ## Full Packet Parsing Performance
 
-Benchmarking packet parsing is challenging because performance is greatly impacted by the complexity of the packet 
+Benchmarking packet parsing is challenging because performance is greatly impacted by the complexity of the packet
 structures being parsed. There are a few measures by which we can assess the performance of Space Packet Parser.
 
 > [!NOTE]
-> Throughout the Space Packet Parser repo and documentation space, 
+> Throughout the Space Packet Parser repo and documentation space,
 > B/kB means bytes/kilobytes and b/kb means bits/kilobits.*
 
 Common factors affecting performance:
@@ -17,23 +17,23 @@ Common factors affecting performance:
 
 ### Packets Per Second
 
-This is a metric we are often asked about. Unfortunately, the answer is that it depends on which packets are 
-being parsed: how many fields are in each packet and how much extra work the parser is doing to sort out complex 
+This is a metric we are often asked about. Unfortunately, the answer is that it depends on which packets are
+being parsed: how many fields are in each packet and how much extra work the parser is doing to sort out complex
 packet structures and evaluate calibrators.
 
 ### Kilobits Per Second
 
-This metric is often used when discussing data volumes and downlink bandwidths to make sure that a data processing 
-system can keep up with the data rate from a spacecraft in the time allowed for processing. This number is also 
-affected by packet structures. It will be high for simple packets containing large binary blobs and low for 
+This metric is often used when discussing data volumes and downlink bandwidths to make sure that a data processing
+system can keep up with the data rate from a spacecraft in the time allowed for processing. This number is also
+affected by packet structures. It will be high for simple packets containing large binary blobs and low for
 complex packets containing many small fields.
 
 ### Results
 
-These tests were run on an Apple Silicon M3 Max processor. 
+These tests were run on an Apple Silicon M3 Max processor.
 
-As a baseline, for relatively simple packets (these are JPSS-1 spacecraft geolocation packets containing attitude 
-and ephemeris data), we benchmarked using 7200 packets with a consistent size of 71B per packet. These packets contain 
+As a baseline, for relatively simple packets (these are JPSS-1 spacecraft geolocation packets containing attitude
+and ephemeris data), we benchmarked using 7200 packets with a consistent size of 71B per packet. These packets contain
 32-bit floats and integers of various sizes.
 
 - **26405-34620 packets per second**
@@ -89,9 +89,9 @@ Progress: [====================]100% [Elapsed: 0:00:00.264275, Parsed 511200 byt
 
 ## Parsing Individual Values Benchmarking
 
-In addition to the benchmarks discussed above, we also benchmarked the low level operations that make up most 
+In addition to the benchmarks discussed above, we also benchmarked the low level operations that make up most
 of the parsing work. The parser relies on two fundamental methods: `read_as_int(nbits)` and `read_as_bytes(nbits)`,
-each of which is capable of reading an arbitrary number of bits from a byte string. That is, the binary data being 
+each of which is capable of reading an arbitrary number of bits from a byte string. That is, the binary data being
 parsed need not be byte aligned or even an integer number of bytes.
 
 ```
@@ -108,9 +108,9 @@ test_benchmark__read_as_bytes__partial_bytes              441.6660 (1.91)     46
 
 The results are as expected:
 
-- The most efficient parsing is byte-aligned parsing of objects that are integer number of bytes in length. 
+- The most efficient parsing is byte-aligned parsing of objects that are integer number of bytes in length.
 - Parsing integers is slower than raw bytes due to the conversion from bytes to int.
-- The most expensive operation is parsing a bytes object that is an odd number of bits (e.g. 6 bits). This is due 
+- The most expensive operation is parsing a bytes object that is an odd number of bits (e.g. 6 bits). This is due
   to the padding operation required to return a bytes object from such a call.
 - The only surprise is that non-aligned integers parse faster than non-aligned full bytes. Ironically this is due
   to a check that we perform during byte parsing to return faster if the request _is_ byte aligned.

docs/source/changelog.md

@@ -1,12 +1,12 @@
 # Change Log
-This is a log of changes made to the library over time. For planned upcoming changes, please check the GitHub issue 
+This is a log of changes made to the library over time. For planned upcoming changes, please check the GitHub issue
 list and release milestones.
 
 ## Version Release Notes
 Release notes for the `space_packet_parser` library
 
 ### v6.0.0 (unreleased)
-- *BREAKING*: `XtcePacketDefinition` no longer accepts a file object as input. 
+- *BREAKING*: `XtcePacketDefinition` no longer accepts a file object as input.
   Use `spp.xtce.definitions.XtcePacketDefinition.from_document()` or `spp.load_xml()` instead.
 - *BREAKING*: Reorganization of the project into different submodules for more explicit handling
   of imports. There is now an `space_packet_parser.xtce` module with xtce representations separated
@@ -41,27 +41,27 @@ Release notes for the `space_packet_parser` library
 - If a packet definition parses too few bits, a UserWarning is now emitted instead of a logger warning.
 
 ### v5.0.0 (released)
-- *BREAKING*: Main API changed. No need to create separate definition and parser objects any more. Create only a 
+- *BREAKING*: Main API changed. No need to create separate definition and parser objects any more. Create only a
   definition from your XTCE document and instead of `my_parser.generator`, use `my_packet_definition.packet_generator`.
-- *BREAKING*: Removed CSV-based packet definition support. We may indirectly support this in the future via 
+- *BREAKING*: Removed CSV-based packet definition support. We may indirectly support this in the future via
   a utility for converting CSV definitions to XTCE.
 - *BREAKING*: Separated out logical pieces into separate modules rather than everything
   living within the xtcedef module. This means user imports may be different now.
 - *BREAKING*: Replace `bitstring` objects with native Python bytes objects
   - Remove dependency on the `bitstring` library
   - Much faster parsing speed
-  - Users that are passing `bitstring.ConstBitStream` objects to `generator` will need to pass a 
+  - Users that are passing `bitstring.ConstBitStream` objects to `generator` will need to pass a
     binary filelike object instead
 - *BREAKING*: The ``ParsedDataItem`` class has been removed and the derived values are being returned now.
   The ``raw_value`` is stored as an attribute on the returned object. The other items can be accessed
   through the packet definition object ``my_packet_definition.named_parameters["my_item"].short_description``
 - *BREAKING*: The return type of BinaryDataEncoding is now the raw bytes.
   To get the previous behavior you can convert the data to an integer and then format it as a binary string.
   ``f"{int.from_bytes(data, byteorder='big'):0{len(data)*8}b}"``
-- *BREAKING*: Removed `word_size` kwarg from packet generator method. 
+- *BREAKING*: Removed `word_size` kwarg from packet generator method.
   We expect all binary data to be integer number of bytes.
 - *BREAKING*: Changed `packet_generator` kwarg `skip_header_bits` to `skip_header_bytes`.
-- Fixed incorrect parsing of StringDataEncoding elements. Raw string values are now returned as byte buffers. 
+- Fixed incorrect parsing of StringDataEncoding elements. Raw string values are now returned as byte buffers.
   Derived string values contain python string objects.
 - The ``CCSDSPacket`` class is now a dictionary subclass, enabling direct lookup of items from the Packet itself.
 - A ``RawPacketData`` class has been added that is a subclass of bytes. It keeps track of the current
@@ -101,9 +101,9 @@ Release notes for the `space_packet_parser` library
 
 ## Historical Changes (`lasp_packets`)
 Changes documented in v3.0 and earlier correspond to development efforts undertaken before this library was
-moved to GitHub (it was previously known as `lasp_packets`). 
-None of the git history is available for these versions as the git history was truncated 
-in preparation for the move to Github to prevent accidental release of non-public example data which may be 
+moved to GitHub (it was previously known as `lasp_packets`).
+None of the git history is available for these versions as the git history was truncated
+in preparation for the move to Github to prevent accidental release of non-public example data which may be
 (but probably isn't) present in historical commits.
 
 ### v3.0 (released publicly)
@@ -135,7 +135,7 @@ in preparation for the move to Github to prevent accidental release of non-publi
 - Add support for BooleanExpression in a RestrictionCriteria element
 
 ### v1.3 (released internally)
-- Expand version compatiblity for python >=3.6, <4
+- Expand version compatibility for python >=3.6, <4
 
 ### v1.2 (released internally)
 - Remove unnecessary warning about float data types being IEEE formatted.
@@ -146,7 +146,7 @@ in preparation for the move to Github to prevent accidental release of non-publi
 - Add support for CSV-based packet definitions (contribution by Michael Chambliss).
 
 ### v1.0 (released internally)
-- Add support for all parameter types. 
+- Add support for all parameter types.
 - Add support for all data encodings.
 - Add support for calibrators and contextual calibrators.
 - Add support for variable length strings given by termination characters or preceding length fields.

docs/source/developers.md

@@ -1,12 +1,30 @@
 # Developer Documentation
+
 ## Installing Development Dependencies
-Poetry installs dev dependencies by default from the `poetry.lock` or `pyproject.toml` files. Just run 
+
+Poetry installs dev dependencies by default from the `poetry.lock` or `pyproject.toml` files.
+
 ```bash
 poetry install
 ```
 
+Alternatively, if you are using something other than Poetry for development you can install from
+the `dev` extras group.
+
+```bash
+pip install ".[dev]"
+```
+
+Once the development dependencies are installed, you can run
+
+```bash
+pre-commit install
+```
+
+to get pre-commit hooks to automatically run the linting and formatting checks for you before each commit.
+
 ## Testing
-Testing is run with `pytest` and the order is randomized by `pytest-randomly`. 
+Testing is run with `pytest` and the order is randomized by `pytest-randomly`.
 To run all tests, run
 ```bash
 pytest tests
@@ -18,15 +36,15 @@ docker-compose up --build && docker-compose down
 ```
 
 ## Building Documentation with Sphinx
-Documentation is automatically built on ReadTheDocs in response to every PR and release, 
+Documentation is automatically built on ReadTheDocs in response to every PR and release,
 but you can also build it locally with:
 ```bash
 # From docs directory
 make html && open build/html/index.html
 ```
 
 ## Making a Pull Request
-Feel free to fork this repo and submit a PR! 
+Feel free to fork this repo and submit a PR!
 - If you are working on an issue, link your PR to that issue.
 - All PRs should be destined for the `main` branch (trunk-based development).
 - Reviews are required before merging and our automated tests must pass.
@@ -45,7 +63,7 @@ That is,
 
 ### Preparing for Release
 1. Create a release candidate branch named according to the version to be released. This branch is used to polish
-   the release but is fundamentally not different from any other feature branch in trunk-based development. 
+   the release but is fundamentally not different from any other feature branch in trunk-based development.
    The naming convention is `release/X.Y.Z`.
 
 2. Bump the version of the package to the version you are about to release, either manually by editing `pyproject.toml`
@@ -56,30 +74,30 @@ That is,
 
 4. Update `changelog.md` to reflect that the version is now "released" and revisit `README.md` to keep it up to date.
 
-5. Open a PR to merge the release branch into main. This informs the rest of the team how the release 
-   process is progressing as you polish the release branch. You may need to rebase the release branch onto 
+5. Open a PR to merge the release branch into main. This informs the rest of the team how the release
+   process is progressing as you polish the release branch. You may need to rebase the release branch onto
    any recent changes to `main` and resolve any conflicts on a regular basis.
 
-6. When you are satisfied that the release branch is ready, merge the PR into `main`. 
+6. When you are satisfied that the release branch is ready, merge the PR into `main`.
 
-7. Check out the `main` branch, pull the merged changes, and tag the newly created merge commit with the 
-   desired version `X.Y.Z` and push the tag upstream. 
+7. Check out the `main` branch, pull the merged changes, and tag the newly created merge commit with the
+   desired version `X.Y.Z` and push the tag upstream.
 
 ### Automatic Release Process
-We use GitHub Actions for automatic release process that responds to pushes of git tags. When a tag matching 
-a semantic version (`[0-9]+.[0-9]+.[0-9]+*` or `test-release/[0-9]+.[0-9]+.[0-9]+*`) is pushed, 
-a workflow runs that builds the package, pushes the artifacts to PyPI or TestPyPI 
-(if tag is prefixed with `test-release`), 
-and creates a GitHub Release from the distributed artifacts. Release notes 
+We use GitHub Actions for automatic release process that responds to pushes of git tags. When a tag matching
+a semantic version (`[0-9]+.[0-9]+.[0-9]+*` or `test-release/[0-9]+.[0-9]+.[0-9]+*`) is pushed,
+a workflow runs that builds the package, pushes the artifacts to PyPI or TestPyPI
+(if tag is prefixed with `test-release`),
+and creates a GitHub Release from the distributed artifacts. Release notes
 are automatically generated from commit history and the Release name is taken from the basename of the tag.
 
 #### Official Releases
 Official releases are published to the public PyPI (even if they are release candidates like `1.2.3rc1`). This differs
-from test releases, which are only published to TestPyPI and are not published to GitHub at all. 
-If the semantic version has any suffixes (e.g. `rc1`), the release will be marked as 
+from test releases, which are only published to TestPyPI and are not published to GitHub at all.
+If the semantic version has any suffixes (e.g. `rc1`), the release will be marked as
 a prerelease in GitHub and PyPI.
 
-To trigger an official release, push a tag referencing the commit you want to release. The commit _MUST_ be on 
+To trigger an official release, push a tag referencing the commit you want to release. The commit _MUST_ be on
 the `main` branch. Never publish an official release from a commit that hasn't been merged to `main`!
 
 ```bash
@@ -90,10 +108,10 @@ git push origin X.Y.Z
 ```
 
 #### Test Releases
-Test releases are published to TestPyPI only and are not published on GitHub. Test releases are triggered by tags 
+Test releases are published to TestPyPI only and are not published on GitHub. Test releases are triggered by tags
 prefixed with `test-release`.
 
-To publish a test release, prefix the tag with `test-release`. This will prevent any publishing to the public PyPI 
+To publish a test release, prefix the tag with `test-release`. This will prevent any publishing to the public PyPI
 and will prevent the artifacts being published on GitHub.
 
 ```bash

docs/source/examples.md

@@ -3,7 +3,7 @@
 [Examples folder on Github](https://github.com/lasp/space_packet_parser/tree/main/examples)
 
 Examples on Github include:
-- [Basic quicklook tool](https://github.com/lasp/space_packet_parser/blob/main/examples/parsing_and_plotting_idex_waveforms_from_socket.py) 
+- [Basic quicklook tool](https://github.com/lasp/space_packet_parser/blob/main/examples/parsing_and_plotting_idex_waveforms_from_socket.py)
   using realtime packet parsing from a streaming socket
-- [CSV to XTCE](https://github.com/lasp/space_packet_parser/blob/main/examples/csv_to_xtce_conversion.py) 
+- [CSV to XTCE](https://github.com/lasp/space_packet_parser/blob/main/examples/csv_to_xtce_conversion.py)
   packet definition conversion and parsing