Think about storing original images in the history #90

New issue

Open

opened 2025-10-17 01:02:48 +03:00 by skobkin · 0 comments

skobkin commented

2025-10-17 01:02:48 +03:00

Owner

Idea

LLM response quality could be improved if we:

Store original images in the history
Provide them to the LLM with all the history altogether each time it needs it
Remove early/lazy image-to-text conversion

Why

Currently when LLM replies to the request, it can rely only on the old image-to-text representation which were made without current context of user request.

If LLM has the image while processing current request, it can achieve better results by analyzing the image with consideration of it.

Pros

Better response quality
No requests for image recognition until we're starting to process user's request

Cons

Image recognition result caching will be removed
Images could be processed multiple times

## Idea LLM response quality could be improved if we: - Store original images in the history - Provide them to the LLM with all the history altogether each time it needs it - Remove early/lazy image-to-text conversion ## Why Currently when LLM replies to the request, it can rely only on the old image-to-text representation which were made without current context of user request. If LLM has the image while processing current request, it can achieve better results by analyzing the image with consideration of it. ## Pros - Better response quality - No requests for image recognition until we're starting to process user's request ## Cons - Image recognition result caching will be removed - Images could be processed multiple times

skobkin added the

research needed

enhancement

labels

2025-10-17 01:02:48 +03:00

skobkin self-assigned this

2025-10-17 01:02:48 +03:00

skobkin referenced this issue

2025-10-17 01:02:59 +03:00

Improvements part 1 #89

skobkin pinned this

2025-10-17 02:03:42 +03:00