Merge branch 'next' for v1.7

Author: enchant97

.github/workflows/tests.yml

@@ -5,6 +5,7 @@ on:
     branches: [ main ]
   pull_request:
     branches: [ main, next ]
+  workflow_dispatch:
 
 jobs:
   lint:
@@ -19,14 +20,13 @@ jobs:
       uses: actions/cache@v3
       with:
         path: ~/.cache/pip
-        key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
+        key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements*.txt') }}
         restore-keys: |
           ${{ runner.os }}-pip-
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        pip install flake8
-        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
+        pip install -r requirements.txt -r requirements-dev.txt
     - name: Run Linter
       run: |
         # stop the build if there are Python syntax errors or undefined names
@@ -45,12 +45,17 @@ jobs:
       uses: actions/cache@v3
       with:
         path: ~/.cache/pip
-        key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
+        key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements*.txt') }}
         restore-keys: |
           ${{ runner.os }}-pip-
     - name: Install dependencies
       run: |
-        python -m pip install --upgrade pip
-        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
+        pip install -r requirements.txt -r requirements-dev.txt
     - name: Run Unit Tests
-      run: python -m unittest -v
+      run: mkdir reports && nose2
+    - name: Upload Test Reports
+      uses: actions/upload-artifact@v3
+      if: success() || failure()
+      with:
+        name: reports
+        path: reports/

.gitignore

@@ -9,3 +9,4 @@ data/
 __pycache__
 .venv/
 .coverage
+reports/

CHANGELOG.md

@@ -4,6 +4,23 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.7.0] - 2022-10-30
+### Added
+- Show expiry on paste screen
+- Configurable multi-tiered caching
+- CLI can remove empty cache folders
+- Strict paste id checking in URLs
+- Better general exception handling
+- Add human padding for paste id
+- Some more unit tests
+### Changed
+- Major code refactoring (pastes are no longer dependant on storage types, for future s3 object support)
+- Tidy REST API routes
+### Fixed
+- Fixed #53 Expiry set in UI always interpreted as UTC, by having a configurable timezone
+### Removed
+- Removed deprecated features, see #50
+
 ## [1.6.1] - 2022-10-09
 ### Fixed
 - #51, When override lexer is given in paste url incorrectly uses cached content

README.md

@@ -7,7 +7,7 @@
 
 A fast and minimal paste bin, written in Python using Quart.
 
-> **IMPORTANT NOTICE** Some deprecated features will be removed in V1.7, view [here](https://github.com/enchant97/hasty-paste/issues/50)
+> **IMPORTANT NOTICE** Some deprecated features have been removed in V1.7, view [here](https://github.com/enchant97/hasty-paste/issues/50)
 
 ## Features
 - Quickly paste and save, to share some text
@@ -20,13 +20,17 @@ A fast and minimal paste bin, written in Python using Quart.
 - Uses minimal resources
 - REST API
 - No Database needed, everything stored as flat-files
+- Caching (Internal & Redis)
 - Lightweight Docker image (uses Alpine Linux)
 
 ## Showcase
 [![Showcase Image](docs/assets/showcase.png)](docs/assets/showcase.png)
 
 ## Docs
-Docs are located in the [/docs](docs/index.md) directory.
+Docs are located in the [/docs](docs/index.md) directory. Or on the site: [enchantedcode.co.uk/hasty-paste](https://enchantedcode.co.uk/hasty-paste)
+
+## Hastily Paste It CLI
+This is a simple script allowing the creation of pastes from the command-line. You can download your version [here](hastily-paste-it/README.md).
 
 ## Branches
 | Name         | Description            | State         |

docs/assets/openapi.yml

@@ -0,0 +1,144 @@
+openapi: 3.0.0
+info:
+  title: Hasty Paste API
+  description: "A fast and minimal paste bin, written in Python using Quart."
+  version: 1.7.0
+
+paths:
+  /api/is-healthy:
+    get:
+      tags:
+        - Status
+      summary: "Check the health of server"
+      description: ""
+      responses:
+        200:
+          description: "Server is OK"
+  /api/pastes:
+    post:
+      tags:
+        - Paste
+      summary: ""
+      description: ""
+      requestBody: { $ref: "#/components/requestBodies/PasteCreate" }
+      responses:
+        201:
+          description: ""
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/Paste" }
+        400:
+          description: "required fields are missing, or invalid"
+  /api/pastes/:
+    get:
+      tags:
+        - Paste
+      summary: "Get all id's, requires ENABLE_PUBLIC_LIST to be True"
+      description: ""
+      responses:
+        200:
+          description: "Paste id's"
+        403:
+          description: "It has been disabled by administrator"
+  /api/pastes/simple:
+    post:
+      tags:
+        - Paste
+      summary: "Create a paste without any fancy features"
+      description: "Could be used easily with curl by a user."
+      responses:
+        201:
+          description: "paste id of created paste"
+  /api/pastes/{paste_id}:
+    parameters:
+      - name: paste_id
+        example: be4Q5k2K0L
+        schema:
+          type: string
+        in: path
+        required: true
+    get:
+      tags:
+        - Paste
+      summary: "Get a paste's raw content"
+      responses:
+        200:
+          description: "The paste raw content"
+        404:
+          description: "The provided paste id does not exist or has expired"
+  /api/pastes/{paste_id}/meta:
+    parameters:
+      - name: paste_id
+        example: be4Q5k2K0L
+        schema:
+          type: string
+        in: path
+        required: true
+    get:
+      tags:
+        - Paste
+      summary: "Get a paste's meta"
+      responses:
+        200:
+          description: "The paste's meta"
+          content:
+            application/json:
+              schema: { $ref: "#/components/schemas/Paste" }
+        404:
+          description: "The provided paste id does not exist or has expired"
+
+components:
+  schemas:
+    PasteCreate:
+      type: object
+      required:
+        - content
+      properties:
+        content:
+          type: string
+        expire_dt:
+          type: string
+          format: date-time
+        lexer_name:
+          type: string
+          example: python
+        long_id:
+          type: boolean
+          default: false
+        title:
+          type: string
+    Paste:
+      type: object
+      required:
+        - paste_id
+        - creation_dt
+      properties:
+        paste_id:
+          type: string
+          example: be4Q5k2K0L
+        creation_dt:
+          type: string
+          format: date-time
+        expire_dt:
+          type: string
+          format: date-time
+        lexer_name:
+          type: string
+          example: python
+        long_id:
+          type: boolean
+          default: false
+        title:
+          type: string
+        version:
+          type: integer
+          default: 1
+
+  requestBodies:
+    PasteCreate:
+      content:
+        application/json:
+          schema:
+            allOf: [{ $ref: "#/components/schemas/PasteCreate" }]
+      description: ""
+      required: true

docs/index.md

@@ -15,6 +15,7 @@ It is a place to quickly paste some text and share it. Mostly used for sharing d
 - Uses minimal resources
 - REST API
 - No Database needed, everything stored as flat-files
+- Caching (Internal & Redis)
 - Lightweight Docker image (uses Alpine Linux)
 
 ## Showcase

docs/install.md

@@ -18,6 +18,7 @@ All configs shown here should be given as environment variables.
 | Name                             | Description                                                        | Default       | Docker Default |
 | :------------------------------- | :----------------------------------------------------------------- | :------------ | :------------- |
 | PASTE_ROOT                       | Where the paste flat file system will be kept                      |               | /app/data      |
+| TIME_ZONE                        | The time-zone where your clients are (used in web UI)              | Europe/London | Europe/London  |
 | NEW_AT_INDEX                     | Index page displays new paste page instead                         | False         | False          |
 | ENABLE_PUBLIC_LIST               | Whether to enable public access for listing pastes                 | False         | False          |
 |                                  |                                                                    |               |                |
@@ -28,8 +29,8 @@ All configs shown here should be given as environment variables.
 | UI_DEFAULT__EXPIRE_TIME__DAYS    | Default days in ui for expiry if enabled                           | 0             | 0              |
 |                                  |                                                                    |               |                |
 | CACHE__ENABLE                    | Whether to enable caching of any type                              | True          | True           |
-| CACHE__MAX_INTERNAL_SIZE         | The max size of the internal cache                                 | 4             | 4              |
-| CACHE__REDIS_URI                 | Use redis for caching (disabled internal)                          | -             | -              |
+| CACHE__INTERNAL_MAX_SIZE         | The max size of the internal cache (<=0 to disable)                | 4             | 4              |
+| CACHE__REDIS_URI                 | Use redis for caching                                              | -             | -              |
 |                                  |                                                                    |               |                |
 | BRANDING__TITLE                  | Customise the app title                                            | -             | -              |
 | BRANDING__DESCRIPTION            | Customise the app description                                      | -             | -              |
@@ -64,6 +65,15 @@ e.g
 CACHE__REDIS_URI="redis://localhost:6379"
 ```
 
+### Cache
+Hasty Paste supports tired caching. For example both the internal cache and Redis cache can be used. The priority of the cache types are shown below:
+
+```
+Internal -> Redis -> Miss
+```
+
+If you are using multiple workers (set via `WORKERS`), each worker does **not** share memory. This means when using the internal cache, each cached item will be duplicated across workers (increasing memory usage). If this is the case you may want to use Redis and select a smaller internal cache size.
+
 ## With Docker (Recommended)
 This will assume you have both Docker and Docker Compose installed.
 
@@ -87,6 +97,7 @@ services:
       - 8000:8000
     environment:
       - "UI_DEFAULT__USE_LONG_ID=False"
+      - "TIME_ZONE=Europe/London"
 
 volumes:
   data:
@@ -113,6 +124,7 @@ services:
       - 8000:8000
     environment:
       - "UI_DEFAULT__USE_LONG_ID=False"
+      - "TIME_ZONE=Europe/London"
       - "CACHE__REDIS_URI=redis://redis:6379"
     depends_on:
       - redis