When replacing text, don't include <think> block

[Banbury]

Deepseek R1 generates a block of text surrounded by <think> tags, before generating the real answer. When using an action, that replaces the selection in Obsidian, this block currently is included, and I have to delete it manually. It would be nice, if the block wouldn't be included in the first place or deleted at the end of the operation.

[omfgroflmfaol]

This sounds like a great opportunity to add support for structured outputs:

image source: langchain

(replacing "name" -> "reasoning" and "interest" -> "answer" in the image above)

While it would of course also be possible to add a checkbox option to "show/hide reasoning" which just filters out substrings of the accumulated text enclosed in <think> ``</think> tags, the usuage of think tags is not universal for LLMs capable of reasoning. It would be more elegant to provide full customizability via structured outputs. This would also enable advanced custom formatting of outputs (like specifying whether one wants parts of the answer as lists, headings, tags, valid yaml properties etc.)

A structured output format is usually defined with a JSON Schema. Such a JSON Schema could be provided by the user in a text field as part of the object constructed by the LocalGPTAction Interface, and then passed as a parameter of the API request. This would ensure that the response of the model adheres to this format.

In a second step, one would also need some parsing function to be able to customize which parts of the JSON Schema are to be included in the final response. This can be implemented by having a parsing function go through all elements of the JSON formatted response, concatenating all elements in order, but filtering out JSON objects with a property called "hide". This works because if we define a part of the schema to have this property, the corresponding part of the answer will have this property.

Because not all LLMs support this, one would need a fallback error handler where the JSON Schema is passed not as an argument to the API, but as a string that is appended to the prompt, with the instruction "follow this JSON Schema in your response" or something similar (which also works sometimes).

More info: https://platform.openai.com/docs/guides/structured-outputs

[pfrankov]

@omfgroflmfaol thank you for the suggestion but it will make the plugin more complicated. I'll just add "Thinking" loader.