fix: prompt-builder - jinja2 template set vars still shows required

[srini047]

Related Issues

• fixes PromptBuilder incorrectly marks internally set Jinja2 variables as required inputs when using `required_variables='*' #9916

Proposed Changes:

Previously we checked only the undeclared variables and left the set variables.
Now, we find all the variables assigned using set in the template and push it to the variables list.

This behaviour would be seen in multiple components namely:

• PromptBuilder
• ChatPromptBuilder
• LLMMetadataExtractor
• LLMDocumentContentExtractor

How did you test it?

Run the exisiting suite and added test case for the this scenario.

Notes for the reviewer

• Is this logic for addressing the problem is fine.
• Then, do let me know if there's any other component (other than ones mentioned above) as well that needs to be addressed with this logic.

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I documented my code
• I ran pre-commit hooks and fixed any issue

[vercel]

@srini047 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

[coveralls]

Pull Request Test Coverage Report for Build 19114177728

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 22 unchanged lines in 7 files lost coverage.
• Overall coverage remained the same at 92.248%

---

Files with Coverage Reduction	New Missed Lines	%
components/embedders/sentence_transformers_sparse_document_embedder.py	1	98.41%
components/embedders/sentence_transformers_sparse_text_embedder.py	1	98.08%
components/embedders/sentence_transformers_document_embedder.py	2	97.01%
components/embedders/sentence_transformers_text_embedder.py	2	96.49%
dataclasses/chat_message.py	2	99.33%
dataclasses/streaming_chunk.py	3	95.77%
tools/toolset.py	11	84.71%

Totals
Change from base Build 19078717113:	0.0%
Covered Lines:	13531
Relevant Lines:	14668

---

💛 - Coveralls

[sjrl]

With the above listed changes I think we could then extend this to the other components you mention. It may be wise to add this as a utility method in haystack/utils/ either in misc.py or a new file called jinja2.py

[sjrl]

hey @srini047 do you think you'll have time to work on this, this week?

[srini047]

hey @srini047 do you think you'll have time to work on this, this week?

Yes @sjrl will continue on this today.

[sjrl]

Hey @srini047 could you add a release note for your PR following the instructions here?

Also we still want to bring this change to ChatPromptBuilder.