I have recently been experimenting with local LLMs using Ollama, OpenClaw, Hermes, and a small homemade harness.
The usual approach to long-term memory is to save information in files or a database and insert the relevant parts into the model’s context. This works, but it is not really the model learning. Replace the memory files, and the model immediately becomes its old self again.
What interests me more is whether a local model could gradually train on its history with a user.
The harness would collect conversations, corrections, accepted solutions, failed attempts, preferences, and the model’s own later reflections. Instead of treating all of this as permanent prompt context, it would periodically turn the most useful examples into training data.
The model could then be fine-tuned locally. Over time, it might internalize how the user communicates, which explanations work, what their projects look like, and which mistakes it has already made.
This would create two forms of continuity.
External memory would provide immediate and editable recall. Training would provide slower, deeper adaptation. The eventual goal would be for the model to rely less on being told who the user is because some of that relationship had become encoded in the model itself.
The difficult part is deciding what deserves to be learned.
Training indiscriminately on every conversation would reinforce misunderstandings, temporary preferences, bad answers, and duplicated information. Repeated fine-tuning could also damage capabilities the model already has.
The harness would therefore need to behave more like an editor than a recorder. It should select interactions with clear outcomes, preserve older examples, test each new adapter against previous versions, and reject updates that make the model worse.
I am especially interested in combining this with an internal reasoning loop. Between conversations, the model could revisit unresolved questions, critique previous answers, and generate candidate lessons from its own history.
The dream is a local model which, over time, becomes truely my own.