Skip to content

feat: integrate SMRITI local-first memory layer - #124

Open
shivamtyagi18 wants to merge 1 commit into
farzaa:mainfrom
shivamtyagi18:feature/smriti-memory-integration
Open

feat: integrate SMRITI local-first memory layer#124
shivamtyagi18 wants to merge 1 commit into
farzaa:mainfrom
shivamtyagi18:feature/smriti-memory-integration

Conversation

@shivamtyagi18

@shivamtyagi18 shivamtyagi18 commented Jul 19, 2026

Copy link
Copy Markdown

Overview

This PR integrates a local-first long-term memory layer (SMRITI) into Clicky and adds support for alternative vision models (Google Gemini, xAI Grok, and local Ollama) to allow testing with free-tier and offline backends. It also introduces a macOS Speech Synthesizer fallback to prevent voice errors when ElevenLabs credits run out.

Key Features

1. SMRITI Memory Layer Integration

  • Proactive Context Retrieval: Queries SMRITI's /recall REST API on push-to-talk release, matching and appending the top 5 relevant long-term memories to the system prompt of the active LLM.
  • Dynamic Hot-Reloading: The local REST daemon checks the file modification time (mtime) of palace.json on incoming queries, immediately syncing new memories written on disk by other concurrent agents (Claude Code, Cursor, Gemini CLI) while maintaining zero-latency by reusing the pre-warmed HuggingFace embeddings model.
  • Active Encoding: Automatically saves conversation turns as User: <prompt>\nClicky: <response> to the shared database.
  • Debugging Logs: Prints recalled memories directly to the Xcode console with 🧠 SMRITI Recalled markers for developer transparency.

2. Multi-Model Support (Gemini, Grok, Ollama)

  • New Settings UI: Redesigns the settings panel model picker to support Sonnet 3.5, Gemini 3.5 Flash, Grok 2 Vision, and Local Ollama.
  • Unified OpenAI Client: Implements OpenAICompatibleAPI.swift to handle standard OpenAI-style vision payloads (base64 image URLs) and stream SSE tokens.
  • Secure Worker Proxy: Extends the Cloudflare Worker with a /openai-proxy route to append xAI and Google AI Studio keys securely, routing Gemini keys via query parameters (?key=) to bypass Vertex AI's v1main enterprise routing.
  • Custom Local Models: Displays a text entry row when Ollama is selected to customize local model targets (defaults to llama3.2-vision).

3. Fail-safe TTS Fallback

  • If ElevenLabs returns a 401 Unauthorized (invalid/missing key) or 429 Too Many Requests (out of credits), Clicky automatically falls back to the Mac's local NSSpeechSynthesizer to read the actual model response aloud.

Changed Files

  • leanring-buddy/CompanionManager.swift: Orchestrates model routing, local TTS fallbacks, and SMRITI daemon lifecycle.
  • leanring-buddy/CompanionPanelView.swift: Renders the new model picker UI and Ollama custom model entry field.
  • leanring-buddy/OpenAICompatibleAPI.swift [NEW]: Unified OpenAI spec request-builder and SSE stream parser.
  • scripts/smriti_local_api.py [NEW]: SMRITI REST API server with palace database hot-reloading.
  • worker/src/index.ts: Secure proxy routing for Claude, ElevenLabs, Gemini, and Grok.
  • AGENTS.md: Updated architecture design documentation.
Screenshot 2026-07-18 at 8 44 35 PM Screenshot 2026-07-19 at 12 12 05 AM

@shivamtyagi18

Copy link
Copy Markdown
Author

Hi @farzaa added integration with Smriti-Memcore local memory for Clicky. May be test it out and share your feedback. I can enhance Smriti for Clicky requirements. This is a great product. Thank you for sharing. Would love to contribute and bring pleasure to end users :)

…odel support

- Integrates SMRITI local-first long-term memory layer with hot-reloading sync, turn encoding, and console logging of recalled memories.
- Exposes local SMRITI HTTP API daemon on port 7798 with global storage at ~/.smriti/global.
- Implements hot-reloading in local REST daemon to dynamically sync concurrent database changes made by other tools (Claude Code, Cursor, Gemini).
- Adds native support for alternative models: Google Gemini 3.5 Flash, xAI Grok 2 Vision, and Local Ollama.
- Implements generic OpenAICompatibleAPI client in Swift to handle OpenAI-style vision payloads and SSE streaming.
- Updates Cloudflare Worker with a secure /openai-proxy endpoint to append GEMINI_API_KEY and XAI_API_KEY env secrets.
- Integrates a macOS Speech Synthesizer fallback to read responses aloud if ElevenLabs runs out of credits or has an invalid key.
- Redesigns model picker in settings UI to support Sonnet, Gemini, Grok, and Ollama, adding a text input to customize local Ollama model names.
- Fixes transcription token URL configuration to use the deployed Cloudflare Worker domain.
- Self-updated AGENTS.md to match the new architecture.
@shivamtyagi18
shivamtyagi18 force-pushed the feature/smriti-memory-integration branch from 6a0ce1a to 8611688 Compare July 19, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant