Trouble validating settings and mqtt

[mikerenna]

Im trying to set this up and I'm struggling with the following error:
2022-08-05T12:43:59-05:00 [Settings] Unable to read the secrets file: ENOENT: no such file or directory, open '/run/secrets/secrets'.
2022-08-05T12:43:59-05:00 [Settings] Loaded settings from /run/secrets/settings

I don't see anything happening in mqtt allthough i have it enabled (would have expected connect messages in log or in mqtt). I'm ultimately trying to determine if the docker setup is correct. the above message makes me think the settings file did not load correctly but i can console to the /run/secrets/ folder referenced above and viewing the files there, they appear to be my settings and triggers files. Any thoughts on the error or why i might not see mqtt message of any kind related to startup or any status?

Thanks in advance
Mike

[neilenns]

Please post a copy of your docker compose configuration

…

________________________________ From: mikerenna ***@***.***> Sent: Friday, August 5, 2022 10:58:08 AM To: neilenns/node-deepstackai-trigger ***@***.***> Cc: Subscribed ***@***.***> Subject: [neilenns/node-deepstackai-trigger] Trouble validating settings and mqtt (Issue #456) Im trying to set this up and I'm struggling with the following error: 2022-08-05T12:43:59-05:00 [Settings] Unable to read the secrets file: ENOENT: no such file or directory, open '/run/secrets/secrets'. 2022-08-05T12:43:59-05:00 [Settings] Loaded settings from /run/secrets/settings I don't see anything happening in mqtt allthough i have it enabled (would have expected connect messages in log or in mqtt). I'm ultimately trying to determine if the docker setup is correct. the above message makes me think the settings file did not load correctly but i can console to the /run/secrets/ folder referenced above and viewing the files there, they appear to be my settings and triggers files. Any thoughts on the error or why i might not see mqtt message of any kind related to startup or any status? Thanks in advance Mike — Reply to this email directly, view it on GitHub<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fneilenns%2Fnode-deepstackai-trigger%2Fissues%2F456&data=05%7C01%7C%7C03f7ef58a6a648069d2008da770c0ed0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637953190922152068%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=jvXHoKJHk6s7kY0Jb0etf4gngB8VdHf8W%2BiWEA3wXys%3D&reserved=0>, or unsubscribe<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACIVHFX7DPFWS7SMV6N2T3DVXVI3BANCNFSM55W22EXA&data=05%7C01%7C%7C03f7ef58a6a648069d2008da770c0ed0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637953190922152068%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=66xFf%2Fq0kPtdsmvP5IMal0yCA4z7rPERyPNZCUICuWY%3D&reserved=0>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[mikerenna]

here you go. trying to pull them off of a local folder. this is running QNAP container station for docker: copy/pasted flattened the formatting.

version: "3.8"
services:
trigger:
volumes:
- '/share/blue_iris/aiinput:/aiinput'
environment:
- TZ=America/Chicago
ports:
- 4242:4242
secrets:
- triggers
- settings
image: ghcr.io/neilenns/node-deepstackai-trigger:latest
restart: always
depends_on:
- deepstack-ai

deepstack-ai:
image: deepquestai/deepstack:latest
restart: always
volumes:
- localstorage:/datastore
environment:
- VISION-DETECTION=True

volumes:
localstorage:

secrets:
settings:
# This should point to the location of the settings.json configuration file
file: '/share/Container/deepstackai/settings.json'

triggers:
# This should point to the location of the triggers.json configuration file
file: '/share/Container/deepstackai/triggers.json'

[mikerenna]

strange thing is that it appears that its picking up those files and creating them in the run/secrets/ location. if i view them through console they appear correct. so maybe its working but not sure how to tell. nothing in mqtt to indicate any connection. so i'm wondering if something is wrong.

[neilenns]

Are you sure the paths to those files exist on your local machine? These two lines should be the local path:

file: '/share/Container/deepstackai/settings.json'
file: '/share/Container/deepstackai/triggers.json'

Also if they are shared to whatever machine is running Docker that might cause a problem? Not sure. In my case they are just local files in the same folder as my docker-compose.yml file.

[neilenns]

I looked at my config more closely and I don't think I'm even using the secrets, I just have a volume mounted. Try doing it that way instead:

services:
  trigger:
    volumes:
      - /share/blue_iris/aiinput:/aiinput
      - /share/Container/deepstackai:/config

[mikerenna]

That looks more in line with other containers I run with a mounted config folder. I'll give that a go. Thank you.

…

________________________________ From: Neil Enns ***@***.***> Sent: Friday, August 5, 2022 2:31:05 PM To: neilenns/node-deepstackai-trigger ***@***.***> Cc: mikerenna ***@***.***>; Author ***@***.***> Subject: Re: [neilenns/node-deepstackai-trigger] Trouble validating settings and mqtt (Issue #456) I looked at my config more closely and I don't think I'm even using the secrets, I just have a volume mounted. Try doing it that way instead: services: trigger: volumes: - /share/blue_iris/aiinput:/aiinput - /share/Container/deepstackai:/config — Reply to this email directly, view it on GitHub<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fneilenns%2Fnode-deepstackai-trigger%2Fissues%2F456%23issuecomment-1206739157&data=05%7C01%7C%7C248e3653aaa8420ffb0f08da7710a8c3%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637953210708159937%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=UP3ZwNBL7sTqkSlAX9TjjMkikLHvSu85aqMyOS9YjT4%3D&reserved=0>, or unsubscribe<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAKWHHRQTWWPJTGJ6ONYD75TVXVMWTANCNFSM55W22EXA&data=05%7C01%7C%7C248e3653aaa8420ffb0f08da7710a8c3%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637953210708159937%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=B3omR2s0CtWuypAoNYUqDi%2BROTBZZGP8lFdbkn0a5lA%3D&reserved=0>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[mikerenna]

unfortunately no difference. From this is seems like its at least picking up the settings file from where I put it. (i added a config subfolder on my share location). Is there more I should see here? Any way to tell if its running properly? I'm assuming that something is wrong because there are no entries at all in my MQTT host and i was expecting them from startup at least. No indication here in the log that its connected. no indication that triggers file was loaded either.

2022-08-08T10:30:06-05:00 [Main] ****************************************
2022-08-08T10:30:06-05:00 [Main] Starting up version 5.8.4
2022-08-08T10:30:06-05:00 [Main] Timezone offset is 300
2022-08-08T10:30:06-05:00 [Main] Current time is Mon Aug 08 2022 10:30:06 GMT-0500 (Central Daylight Time)
2022-08-08T10:30:06-05:00 [Settings] Unable to read the settings file: ENOENT: no such file or directory, open '/run/secrets/settings'.
2022-08-08T10:30:06-05:00 [Settings] Unable to load file /run/secrets/settings.
2022-08-08T10:30:06-05:00 [Settings] Unable to read the secrets file: ENOENT: no such file or directory, open '/config/secrets.json'.

[mikerenna]

as a follow-up, i modified the mqtt host as a test and immediately got an error in the log. So its clearly loading the settings file but no indication that it successfully connects to the mqtt server. is there a setting that forces it to publish a status message in mqtt that i'm missing? A way to confirm that the triggers file is being picked up? Testing triggers, there is no action in mqtt either which seems related. Thanks in advance for any ideas.

[neilenns]

Enable verbose logging and see if it shows any messages for images getting processed. https://github.com/neilenns/node-deepstackai-trigger/wiki/Configuration

…

________________________________ From: mikerenna ***@***.***> Sent: Tuesday, August 9, 2022 4:23:29 AM To: neilenns/node-deepstackai-trigger ***@***.***> Cc: Neil Enns ***@***.***>; Comment ***@***.***> Subject: Re: [neilenns/node-deepstackai-trigger] Trouble validating settings and mqtt (Issue #456) as a follow-up, i modified the mqtt host as a test and immediately got an error in the log. So its clearly loading the settings file but no indication that it successfully connects to the mqtt server. is there a setting that forces it to publish a status message in mqtt that i'm missing? A way to confirm that the triggers file is being picked up? Testing triggers, there is no action in mqtt either which seems related. Thanks in advance for any ideas. — Reply to this email directly, view it on GitHub<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fneilenns%2Fnode-deepstackai-trigger%2Fissues%2F456%23issuecomment-1209253751&data=05%7C01%7C%7C923790e5b9a54f8384a708da79f99628%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637956410132482167%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=7XsvnPrRF3%2FWZxq0u%2F7CQstJWYeqA%2F54COuq4HgwklM%3D&reserved=0>, or unsubscribe<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACIVHFWAPMTYKBZSOT4ARDDVYI5TDANCNFSM55W22EXA&data=05%7C01%7C%7C923790e5b9a54f8384a708da79f99628%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637956410132482167%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Er08fnsjnhiMm%2FZAbI1erJyvVsU5ykQgsDAh7V6BXfQ%3D&reserved=0>. You are receiving this because you commented.Message ID: ***@***.***>

[mikerenna]

Verbose was already set to true. Should i have been seeing more than the above in the log as a result? it seems like its loading the settings file (which i reason by the fact that when i change the mqtt host to a bad port, i see an error on restart) but if its not picking up the verbose flag, I couldn't explain that either. Sorry for the trouble. This seems like an exceptionally useful tool. I appreciate you taking your time to share ideas here. -Mike

[neilenns]

Post the entire set of log messages you get from start of container to when it stops writing messages. Thanks!

…

________________________________ From: mikerenna ***@***.***> Sent: Tuesday, August 9, 2022 7:41:29 AM To: neilenns/node-deepstackai-trigger ***@***.***> Cc: Neil Enns ***@***.***>; Comment ***@***.***> Subject: Re: [neilenns/node-deepstackai-trigger] Trouble validating settings and mqtt (Issue #456) Verbose was already set to true. Should i have been seeing more than the above in the log as a result? it seems like its loading the settings file (which i reason by the fact that when i change the mqtt host to a bad port, i see an error on restart) but if its not picking up the verbose flag, I couldn't explain that either. Sorry for the trouble. This seems like an exceptionally useful tool. I appreciate you taking your time to share ideas here. -Mike — Reply to this email directly, view it on GitHub<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fneilenns%2Fnode-deepstackai-trigger%2Fissues%2F456%23issuecomment-1209471866&data=05%7C01%7C%7C92bbc5fe98784fc88c7208da7a153f83%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637956528920314846%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=R39KlQsXEFd5sS8nxqnN3tz7RVYSYUXsMliGzam%2FCGI%3D&reserved=0>, or unsubscribe<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACIVHFXU4N7WPZF3GYG62ZDVYJUZTANCNFSM55W22EXA&data=05%7C01%7C%7C92bbc5fe98784fc88c7208da7a153f83%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637956528920314846%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MGKiUdU8YDBRGeNqslTL36xTmQcmWpjoE9NINf%2FG3sk%3D&reserved=0>. You are receiving this because you commented.Message ID: ***@***.***>

[mikerenna]

just checked that blue iris is dropping jpgs to the designated folder but no action in log. In fact, log is still just showing the three times I restarted it yesterday. (this is the log view from portainer for the trigger container. deepstack shows even less). You can see in the log where I changed the port on mqtt to see if i could force a failure.
2022-08-08T10:30:06-05:00 [Main] ****************************************
2022-08-08T10:30:06-05:00 [Main] Starting up version 5.8.4
2022-08-08T10:30:06-05:00 [Main] Timezone offset is 300
2022-08-08T10:30:06-05:00 [Main] Current time is Mon Aug 08 2022 10:30:06 GMT-0500 (Central Daylight Time)
2022-08-08T10:30:06-05:00 [Settings] Unable to read the settings file: ENOENT: no such file or directory, open '/run/secrets/settings'.
2022-08-08T10:30:06-05:00 [Settings] Unable to load file /run/secrets/settings.
2022-08-08T10:30:06-05:00 [Settings] Unable to read the secrets file: ENOENT: no such file or directory, open '/config/secrets.json'.
2022-08-08T10:30:06-05:00 [Settings] Loaded settings from /config/settings.json
2022-08-08T16:35:16-05:00 [Main] Shutting down.
2022-08-08T16:35:17-05:00 [Main] ****************************************
2022-08-08T16:35:17-05:00 [Main] Starting up version 5.8.4
2022-08-08T16:35:17-05:00 [Main] Timezone offset is 300
2022-08-08T16:35:17-05:00 [Main] Current time is Mon Aug 08 2022 16:35:17 GMT-0500 (Central Daylight Time)
2022-08-08T16:35:17-05:00 [Settings] Unable to read the settings file: ENOENT: no such file or directory, open '/run/secrets/settings'.
2022-08-08T16:35:17-05:00 [Settings] Unable to load file /run/secrets/settings.
2022-08-08T16:35:17-05:00 [Settings] Unable to read the secrets file: ENOENT: no such file or directory, open '/config/secrets.json'.
2022-08-08T16:35:17-05:00 [Settings] Loaded settings from /config/settings.json
2022-08-08T16:35:17-05:00 [Main] [MQTT] Unable to connect: connect ECONNREFUSED 192.168.0.14:1884
2022-08-08T16:35:17-05:00 [Main] ****************************************
2022-08-08T16:35:17-05:00 [Main] Startup failed due to errors. For troubleshooting assistance see https://github.com/danecreekphotography/node-deepstackai-trigger/wiki/Troubleshooting.
2022-08-08T16:35:17-05:00 [Main] Startup reattempt 1 of 5 in 30 seconds.
2022-08-08T16:35:47-05:00 [Main] ****************************************
2022-08-08T16:35:47-05:00 [Main] Starting up version 5.8.4
2022-08-08T16:35:47-05:00 [Main] Timezone offset is 300
2022-08-08T16:35:47-05:00 [Main] Current time is Mon Aug 08 2022 16:35:47 GMT-0500 (Central Daylight Time)
2022-08-08T16:35:47-05:00 [Settings] Unable to read the settings file: ENOENT: no such file or directory, open '/run/secrets/settings'.
2022-08-08T16:35:47-05:00 [Settings] Unable to load file /run/secrets/settings.
2022-08-08T16:35:47-05:00 [Settings] Unable to read the secrets file: ENOENT: no such file or directory, open '/config/secrets.json'.
2022-08-08T16:35:47-05:00 [Settings] Loaded settings from /config/settings.json
2022-08-08T16:36:19-05:00 [Main] Shutting down.
2022-08-08T16:36:20-05:00 [Main] ****************************************
2022-08-08T16:36:20-05:00 [Main] Starting up version 5.8.4
2022-08-08T16:36:20-05:00 [Main] Timezone offset is 300
2022-08-08T16:36:20-05:00 [Main] Current time is Mon Aug 08 2022 16:36:20 GMT-0500 (Central Daylight Time)
2022-08-08T16:36:20-05:00 [Settings] Unable to read the settings file: ENOENT: no such file or directory, open '/run/secrets/settings'.
2022-08-08T16:36:20-05:00 [Settings] Unable to load file /run/secrets/settings.
2022-08-08T16:36:20-05:00 [Settings] Unable to read the secrets file: ENOENT: no such file or directory, open '/config/secrets.json'.
2022-08-08T16:36:20-05:00 [Settings] Loaded settings from /config/settings.json

[mikerenna]

i had removed all the secrets references and went to your suggestion for a volume mount for /config. That didn't appear to make a difference. I don't know why there are still secret references here unless that's all just in the code. When I added a /config subfolder to my mount it changed the last line in the log, which i would have expected. that much seems ok.

[neilenns]

Looks like it is up and running. It isn't seeing any images dropped for processing, you should see log entries for each image it tries to analyze. I'm guessing your mount of the folder for images, or the config pointing to that folder, isn't correct.

…

________________________________ From: mikerenna ***@***.***> Sent: Tuesday, August 9, 2022 7:49:16 AM To: neilenns/node-deepstackai-trigger ***@***.***> Cc: Neil Enns ***@***.***>; Comment ***@***.***> Subject: Re: [neilenns/node-deepstackai-trigger] Trouble validating settings and mqtt (Issue #456) i had removed all the secrets references and went to your suggestion for a volume mount for /config. That didn't appear to make a difference. I don't know why there are still secret references here unless that's all just in the code. When I added a /config subfolder to my mount it changed the last line in the log, which i would have expected. that much seems ok. — Reply to this email directly, view it on GitHub<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fneilenns%2Fnode-deepstackai-trigger%2Fissues%2F456%23issuecomment-1209482905&data=05%7C01%7C%7C4996e51e14c541bb662308da7a165799%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637956533619684797%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=59aBfr%2FPhhwbc%2FZCrj69%2BhX2ba3KLNNhCKcSJ%2B5sw2M%3D&reserved=0>, or unsubscribe<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACIVHFRFM3VONN2URJW4EODVYJVWZANCNFSM55W22EXA&data=05%7C01%7C%7C4996e51e14c541bb662308da7a165799%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637956533619684797%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=CVaaZsM9oyBH5MtZ5G%2B1W5xBJYaKBFMBtIAmPYLo0Es%3D&reserved=0>. You are receiving this because you commented.Message ID: ***@***.***>

[mikerenna]

i don't know. I've tried a myriad of things to troubleshoot the picture folder. The sample config files use a relative path in the mask ("/aiinput/Cat*.jpg). What is that relative of? My aiinput folder is almost identical to your example. settings file appears to be being picked up properly from a similar mount (similar to your example, with just an additional /config subfolder on the share). i've tried with the relative path in triggers and without. I tried moving my picture folder round. I'm not sure how to troubleshoot. Pictures are landing in the folder, and there is never any indication in the log that anything past start up has ever happened. is there any other way to validate things are running? Is there no mqtt status message in the log that should happen at startup? or any way to tell that triggers was loaded? (log shows settings being loaded, although some goofy message on that, but no mention of triggers). Verbose flag didn't make any difference in logging from what i can say. Sort of at a loss.