AWS Catalog

[leoloobeek]

Proposed changes

Currently Nuclei integrates with templates on an S3 bucket by downloading all files to disk, saving to the CustomS3TemplateDirectory, then reads the templates from disk. The disk I/O is not ideal for serverless workloads and this PR adds an AWS Catalog implementation similar to disk.DiskCatalog.

This will allow users to pull templates directly from S3 storage and the content is read directly from the S3 download without writing to disk.

I did not remove or alter the AWS S3 code in the customtemplates package to avoid impacting any current use cases that leverage the existing S3 implementation.

Usage

Replacing disk.NewCatalog() with aws.NewCatalog() will leverage AWS S3 for retrieving templates.

catalog := aws.NewCatalog("bucket-name")
// ...
cfg := loader.NewConfig(defaultOpts, &config.Config{}, catalog, executerOpts)
store, _ := loader.New(cfg)
store.Load()

Checklist

• Pull request is created against the dev branch
• All checks passed (lint, unit/integration/regression tests etc.) with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)