perf: eliminate N+1 queries and add bounded caches in hsg_query

[Copilot]

📋 Description

The hsg_query hot path issued up to 4N sequential DB round-trips per query (N = candidate memory count) due to per-record fetches in the scoring loop, tag matching, vector retrieval, and feedback updates. Also, cache and sal_cache were unbounded Maps that grew indefinitely on long-running servers.

Core fixes

Batch DB fetching (db.ts, vector_store.ts, postgres.ts, valkey.ts)

• Added get_mems_by_ids to both SQLite (WHERE id IN (?,...)) and PostgreSQL (WHERE id = ANY($1)) backends
• Added getVectorsByIds to VectorStore interface + all implementations:
  • PostgreSQL: single IN/ANY query
  • Valkey: constructs all vec:{sector}:{id} keys upfront, fetches with a single pipeline round-trip

hsg_query loop refactor (hsg.ts)

Before: 4N+ sequential DB calls inside the scoring loop

// Called N times each:
const m = await q.get_mem.get(mid);                   // N queries
const mvf = await calc_multi_vec_fusion_score(mid, …); // N queries (fetched vectors internally)
const tag_match = await compute_tag_match_score(mid, qtk); // N queries (re-fetched same memory)
const msec = await vector_store.getVectorsById(mid);  // N queries (vectors fetched again)

After: 2 batch queries before the loop, pure in-memory scoring

const [all_mems_arr, vec_map] = await Promise.all([
    q.get_mems_by_ids.all(ids_arr),          // 1 query
    vector_store.getVectorsByIds(ids_arr),    // 1 query
]);
// Loop uses mem_map.get(mid) and vec_map.get(mid) — zero DB calls

Other improvements

• calc_multi_vec_fusion_score: accepts pre-fetched vecs[] instead of fetching by ID; now synchronous
• compute_tag_match_score: accepts tags_json directly instead of re-fetching by memory_id; now synchronous
• Feedback score updates: reads from pre-fetched map + Promise.all for concurrent writes (was N sequential fetches + N sequential writes)
• Linked-node reinforcement: single get_mems_by_ids batch + Promise.all concurrent updates
• Added CACHE_MAX = 500 and SAL_CACHE_MAX = 2000 with FIFO eviction — matches existing pattern used by seg_cache

🔄 Type of Change

• 🐛 Bug fix (non-breaking change which fixes an issue)
• ✨ New feature (non-breaking change which adds functionality)
• 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
• 📚 Documentation update
• 🎨 Style/UI changes
• ♻️ Code refactoring
• ⚡ Performance improvements
• 🧪 Test updates
• 🔧 Build/CI changes

🧪 Testing

• I have tested this change locally
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

📱 Screenshots (if applicable)

N/A

🔍 Code Review Checklist

• Code follows the project's coding standards
• Self-review of the code has been performed
• Code is properly commented, particularly in hard-to-understand areas
• Changes generate no new warnings
• Any dependent changes have been merged and published

🚀 Deployment Notes

No schema changes. No config changes required. Drop-in replacement — existing behavior is preserved.

📋 Additional Context

All changes are backward-compatible. The VectorStore interface gains a new getVectorsByIds method — any custom implementations outside this repo will need to add it.

---

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.