TypeScript AWS Lambda Issue

[michaelbarnes]

Hi, I'm trying to run my tests on AWS Lambda but I run into an issue when Artillery fails to build the TS code .

When I run the run-lambda comment like so:

artillery run-lambda \
   --region eu-west-1 \
   ./src/initial_sync.yml

I end up with this:

WARNING: You are currently running a version of TypeScript which is not officially supported by @typescript-eslint/typescript-estree.

You may find that it works just fine, or you may not.

SUPPORTED TYPESCRIPT VERSIONS: >=3.3.1 <5.2.0

YOUR TYPESCRIPT VERSION: 5.6.3

Please only submit bug reports when using the officially supported version.

=============
Test bundle contents:
┌─────────────────────────┬──────┬───────┐
│ Name                    │ Type │ Notes │
├─────────────────────────┼──────┼───────┤
│ initial_sync.yml        │ file │       │
├─────────────────────────┼──────┤       │
│ processors/powersync.ts │ file │       │
├─────────────────────────┼──────┤       │
│ libs/auth.ts            │ file │       │
├─────────────────────────┼──────┤       │
│ libs/ndjson.ts          │ file │       │
├─────────────────────────┼──────┤       │
│ libs/sync.ts            │ file │       │
├─────────────────────────┼──────┤       │
│ libs/utils.ts           │ file │       │
└─────────────────────────┴──────┴───────┘

 - Lambda function: artilleryio-v2-0-23-arm64-f8c43bf3e9c12c5376caa22b1d6092df
 - Region: eu-west-1
 - AWS account: 026763473155
[a866b382-a8b9-418b-b869-8d93dd4ac83c]: A Lambda function has exited with an error. Reason: ArtilleryError
{
  stdout: 'Test run id: tfdfb_5xjgtf7mnpajpm63nrdk5pft8cq6e_er47\n',
  stderr: '✘ [ERROR] Could not resolve "dotenv/config"\n' +
    '\n' +
    '    processors/powersync.ts:1:7:\n' +
    "      1 │ import 'dotenv/config'; // Load environment variables from a .env file\n" +
    '        ╵        ~~~~~~~~~~~~~~~\n' +
    '\n' +
    '  You can mark the path "dotenv/config" as external to exclude it from the bundle, which will remove this error and leave the unresolved path in the bundle.\n' +
    '\n' +
    '✘ [ERROR] Could not resolve "jose"\n' +
    '\n' +
    '    libs/auth.ts:1:22:\n' +
    "      1 │ import * as jose from 'jose';\n" +
    '        ╵                       ~~~~~~\n' +
    '\n' +
    '  You can mark the path "jose" as external to exclude it from the bundle, which will remove this error and leave the unresolved path in the bundle.\n' +
    '\n' +
    '/artillery/packages/artillery/lib/util/prepare-test-execution-plan.js:209\n' +
    '    throw new Error(`Failed to compile Typescript processor\\n${error.message}`);\n' +
    '          ^\n' +
    '\n' +
    'Error: Failed to compile Typescript processor\n' +
    'Build failed with 2 errors:\n' +
    'libs/auth.ts:1:22: ERROR: Could not resolve "jose"\n' +
    'processors/powersync.ts:1:7: ERROR: Could not resolve "dotenv/config"\n' +
    '    at replaceProcessorIfTypescript (/artillery/packages/artillery/lib/util/prepare-test-execution-plan.js:209:11)\n' +
    '    at prepareTestExecutionPlan (/artillery/packages/artillery/lib/util/prepare-test-execution-plan.js:116:19)\n' +
    '    at async RunCommand.runCommandImplementation (/artillery/packages/artillery/lib/cmds/run.js:178:20)\n' +
    '    at async RunCommand.run (/artillery/packages/artillery/lib/cmds/run.js:43:5)\n' +
    '    at async RunCommand._run (/artillery/node_modules/@oclif/core/lib/command.js:181:22)\n' +
    '    at async Config.runCommand (/artillery/node_modules/@oclif/core/lib/config/config.js:456:25)\n' +
    '    at async Object.run (/artillery/node_modules/@oclif/core/lib/main.js:96:16)\n' +
    '\n' +
    'Node.js v22.14.0\n'
}

When I run tsc --version globally it outputs Version 5.1.6 and I've fixed the TS version is my package.json to 5.1.6.
The tests run totally fine locally.

I can't exclude these packages as I need them in my processor, can someone help?

[michaelbarnes]

Changed my dev setup. Basically build the TS locally and then point to the dist folder in the processors section of the YML e.g.

config:
  ...
  processor: '../dist/processors/powersync.js' 

This works but now after running a few load tests on AWS Lamda, I now get this error:

A Lambda function has exited with an error. Reason: TestDataSyncFailure
{
  err: {
    message: "ENOSPC: no space left on device, mkdir '/tmp/test_data/tnghk_yfyg6ac7ywqrhdkcmxxawznt49mmq_663p'",
    stack: "Error: ENOSPC: no space left on device, mkdir '/tmp/test_data/tnghk_yfyg6ac7ywqrhdkcmxxawznt49mmq_663p'\n" +
      '    at Object.mkdirSync (node:fs:1364:26)\n' +
      '    at syncTestData (/artillery/a9-handler-dependencies.js:11:8)\n' +
      '    at Runtime.handler (/artillery/a9-handler-index.js:70:11)\n' +
      '    at Runtime.handleOnceNonStreaming (file:///artillery/node_modules/aws-lambda-ric/dist/index.mjs:1400:29)'
  }
}
[f0b4a1d5-9630-4143-9c83-7cdab7567bd4]: A Lambda function has exited with an error. Reason: InstallDependenciesFailure
{
  err: {
    message: "ENOENT: no such file or directory, chdir '/tmp/test_data/ty5mf_6zbe896hfn9mcth7kpjrf8549bezw_eddc' -> '/tmp/test_data/tnghk_yfyg6ac7ywqrhdkcmxxawznt49mmq_663p'",
    stack: "Error: ENOENT: no such file or directory, chdir '/tmp/test_data/ty5mf_6zbe896hfn9mcth7kpjrf8549bezw_eddc' -> '/tmp/test_data/tnghk_yfyg6ac7ywqrhdkcmxxawznt49mmq_663p'\n" +
      '    at process.wrappedChdir [as chdir] (node:internal/bootstrap/switches/does_own_process_state:128:14)\n' +
      '    at installNpmDependencies (/artillery/a9-handler-dependencies.js:47:11)\n' +
      '    at Runtime.handler (/artillery/a9-handler-index.js:85:11)\n' +
      '    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)'
  }
}
[f0b4a1d5-9630-4143-9c83-7cdab7567bd4]: A Lambda function has exited with an error. Reason: ArtilleryError
{
  stdout: 'Test run id: tnghk_yfyg6ac7ywqrhdkcmxxawznt49mmq_663p\n' +
    "worker error, id: 1 Error: Cannot find module 'jose'\n" +
    'Require stack:\n' +
    '- /tmp/test_data/ty5mf_6zbe896hfn9mcth7kpjrf8549bezw_eddc/dist/libs/auth.js\n' +
    '- /tmp/test_data/ty5mf_6zbe896hfn9mcth7kpjrf8549bezw_eddc/dist/processors/powersync.js\n' +
    '- /artillery/packages/core/lib/runner.js\n' +
    '- /artillery/packages/core/index.js\n' +
    '- /artillery/packages/artillery/lib/artillery-global.js\n' +
    '- /artillery/packages/artillery/lib/platform/local/worker.js\n' +
    '    at Function._resolveFilename (node:internal/modules/cjs/loader:1225:15)\n' +
    '    at Function._load (node:internal/modules/cjs/loader:1055:27)\n' +
    '    at TracingChannel.traceSync (node:diagnostics_channel:322:14)\n' +
    '    at wrapModuleLoad (node:internal/modules/cjs/loader:220:24)\n' +
    '    at Module.require (node:internal/modules/cjs/loader:1311:12)\n' +
    '    at require (node:internal/modules/helpers:136:16)\n' +
    '    at Object.<anonymous> (/tmp/test_data/ty5mf_6zbe896hfn9mcth7kpjrf8549bezw_eddc/dist/libs/auth.js:36:27)\n' +
    '    at Module._compile (node:internal/modules/cjs/loader:1554:14)\n' +
    '    at Object..js (node:internal/modules/cjs/loader:1706:10)\n' +
    '    at Module.load (node:internal/modules/cjs/loader:1289:32) {\n' +
    "  code: 'MODULE_NOT_FOUND',\n" +
    '  requireStack: [\n' +
    "    '/tmp/test_data/ty5mf_6zbe896hfn9mcth7kpjrf8549bezw_eddc/dist/libs/auth.js',\n" +
    "    '/tmp/test_data/ty5mf_6zbe896hfn9mcth7kpjrf8549bezw_eddc/dist/processors/powersync.js',\n" +
    "    '/artillery/packages/core/lib/runner.js',\n" +
    "    '/artillery/packages/core/index.js',\n" +
    "    '/artillery/packages/artillery/lib/artillery-global.js',\n" +
    "    '/artillery/packages/artillery/lib/platform/local/worker.js'\n" +
    '  ]\n' +
    '}\n',
  stderr: ''
}

I'm using HTTP streams and downloading around 10mb per vUser.
What do I need to do to the buckets to avoid ENOSPC: no space left on device?

[hassy]

AWS Lambda has a 512 mb storage limit per function. Not sure why it's happening, since your test doesn't seem to have any external npm dependencies. Have you tried running the same test on AWS Fargate?