Regex in Substitute not working

[jibz89]

NetBox version
What version of NetBox are you currently running?
v4.0.11

Describe the bug
A clear and concise description of what the bug is.
plugin is not adding the substitute to template, it just adds the jinja2 variable name like {{ cisco_exclude_banner }}

To Reproduce
Steps to reproduce the behavior:

1. Go to 'Config Diff Plugin'
2. Click on 'Platform Settings'
3. Click on 'Add'
4. Click 'Platform' choose IOS (if you don't have platform go back to Device and create Platform)
5. For Driver type in cisco_iosxe
6. For Command 'show running-config'
7. Now Click on 'Create' to save Platform Settings
8. Next under Config Management Click on 'Substitutes'
9. Click 'Add'
10. Choose Platform settings create earlier (ex: IOS XE cisco_iosxe)
11. Name something like 'cisco_exclude_banner'
12. Regexp 'banner login [\S\s]+^C'
13. Click 'Create' to save Substitute
14. Now Click on 'Provisioning' and then 'Config Templates'
15. Create Template type Name ex: test
16. Template Code enter the Substitute as jinja2 variable like {% raw %}{{ cisco_exclude_banner }}{% endraw %}
17. Click on Devices and Edit and Choose the Config template for device (Device must be Active status and have Primary IP and Platform IOS)
18. Now go back to Config Diff Plugin and Click 'Configuration Requests'
19. Click 'Add' Find the device and Click 'Create' to Save
20. Now click on 'Collect diffs'
21. No Error

Expected behavior
A clear and concise description of what you expected to happen.
the matched config was not added into rendered output instead just the variable gets added {{ cisco_exclude_banner }}

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.
sample_text_config.txt

[miaow2]

Hi @jibz89, thanks for opening the issue.

I fix this bug in 2.7.0 version.

Please be caution with regex, because regex in exclude_snmp_traps matches snmp-server lines and lines after that section

[jibz89]

substitutes

Rendered diff

Template

So I removed the snmp traps from the template but the banner substitutes is still not working (see screenshot). i also try creating another template with same code to exclude banner. i tested the regex in regex101.
how can i get into the dev env of this plugin to see what its actually doing ?

thanks for looking into this issue.

[jibz89]

exclude_snmp_traps regex (snmp-server enable traps(?: .*$\n?)+)+ is only matching only lines starting with 'snmp-server enable traps'

[miaow2]

Hi @jibz89 , please share with me next data, so I can test plugin on your data, maybe I miss something and that's why can't catch the bug:

• actual config of the device (remove sensitive data)
• how rendered config in netbox looks like for the device (also remove sensitive data)
• what platform is assigned to the device
• substitutes data: name, regex and platform setting
• diff result from config compliance

how can i get into the dev env of this plugin to see what its actually doing ?

in my env Netbox is installed how it is described in official docs.
then git clone plugins repo, activate netbox virtualvenv and go to the plugins directory and run pip install -e .

after that you will get locally installed version of the plugin, you can do change in it and see it

[jibz89]

• actual config of the device (remove sensitive data)
  

actual config of device.txt

• how rendered config in netbox looks like for the device (also remove sensitive data)
  

rendered config in netbox.txt

• what platform is assigned to the device
  

Device Platform settings are
Platform: IOS XE
Driver: cisco_iosxe
Command: show running-config

• substitutes data: name, regex and platform setting
  

Subtitutes Data:
Name: cisco_exclude_all_before_line
REGEXP: [.*\n\S\s]+^C\n!
Platform Settings: IOS XE cisco_iosxe

Name: cisco_exclude_all_except_line_and_space
REGEXP: ^(?!line\b)(?!\s).+$
Platform Settings: IOS XE cisco_iosxe

• diff result from config compliance
  

I notice if I click on "Collect diffs" button in Configuration Requests in does not get the regex right and just prints out the variable name in the diff.
Go to device and click on Config Compliance (screen shot below)

However, if I go to scripts and choose the device and check “Commit changes” I am getting the expected result, the regex is being captured and removed from the config diff.

thank you for looking into this issue.

[miaow2]

Hi @jibz89 thanks for your data,

I've checked your config and regexp and cisco_exclude_all_before_line does not match anything

because of that I've got an error, when running Collect diffs

Traceback (most recent call last):
  File "/home/miaow/learning/netbox_config_diff/netbox_config_diff/configurator/base.py", line 130, in _collect_one_diff
    actual_config, rendered_config = await conn.render_substituted_config(
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miaow/learning/netbox_config_diff/netbox_config_diff/configurator/platforms.py", line 340, in render_substituted_config
    return CONFIG, self._render_substituted_config(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/miaow/learning/netbox_config_diff/netbox_config_diff/configurator/platforms.py", line 295, in _render_substituted_config
    raise TemplateError(msg)
scrapli_cfg.exceptions.TemplateError: substitution pattern cisco_exclude_all_before_line was unable to find a match in the target configsource

Do you provide correct regexp [.*\n\S\s]+^C\n!?

[jibz89]

this is the correct regexp

[.*\n\S\s]+line con 0\n

thanks,