So chatgpt is still limited to the 4096 token context length?

[aliencaocao]

I saw the code here https://github.com/acheong08/ChatGPT/blob/main/src/revChatGPT/Official.py#L282 and it seems to me that this is exactly the same as gpt-3 in the sense that you are still limited to past 4096 tokens of chat history, since you have to prepend the chat history to the prompt. So what exactly has changed, besides using a different model perhaps fine-tuned for using chat history as context?
I am looking for a solution that does not have such context limit, but so far even chatgpt seems like this. If this is how chatgpt achieves contextual awareness, then I can just implement my own function to prepend chat history to the prompt and send to GPT-3. I am quite confused about why people prefer this than GPT-3.

I also took a look at the API requests on the webpage, and it is not obvious to me that they sent the entire conversation per request. So how did you come up with this implementation? Did you actually find evidence of openAI doing this to their prompts, or you are just trying it out and it happened to work?

[acheong08]

yes

[acheong08]

There is a small difference which is that the conversations endpoint puts it through a few layers of processing before passing it to completions, probably to compress it via another model

[aliencaocao]

that is very surprising to hear. Thanks for the answer

[tzekid]

Sorry to necro this discussion, but how do you guys compress the text for longer conversations? Just recursively ask chatgpt to summarise the discussion and feed that into a system or assistant message?

[acheong08]

Not implemented but based on some work by @waylaidwanderer, you can do that via embeddings for long term memory

[tzekid]

That's a good point of reference. Just discovered OpenAI's Embeddings, might look for something on-device for my use case.
Thanks!