Search & Recommendation Engine
AI / MLProductionDiscovery platformSearch and recommendation system with Elasticsearch full-text retrieval, ML-powered ranking and embedding, collaborative filtering, and real-time click event feedback loop.
Recommended for: Commerce teams tuning search relevance
Scope Notes
Intended for Commerce teams tuning search relevance at Production complexity and Discovery platform scope. Treat the graph as a starting shape rather than a fixed stack: keep the topology and the signals below, then swap providers to match your platform.
Review Checklist
Use Case
E-commerce product search, content discovery, personalised feeds, marketplace ranking
Best Fit Scenarios
- Catalogues needing faceted full-text search
- Personalised ranking with ML re-ranking
- Teams that need a lexical fallback when ML is down
Stack Breakdown
Architecture Layers
Components by Category
frontend
backend
database
external
async
Why This Topology Works
The Catalog Service is the source of truth and the Indexer is the only writer to Elasticsearch, so the index cannot diverge from the catalog. Search and recommendation APIs read the index and re-rank with the ML model, and a lexical fallback keeps search working when inference is unavailable. Clicks are attributed through the Reco API before reaching Kafka.
Scaling Notes
Elasticsearch shards by index and Redis caches user profiles and recent recommendations. ML inference is the ranking bottleneck, so cache embeddings and keep the lexical fallback warm.
Observability
Track search latency P95, indexing lag behind catalog changes, inference time and recommendation CTR. Compare ranked versus lexical results during A/B tests.
Typical Bottlenecks
- Index freshness versus catalog change rate
- ML inference latency on the query path
- Behaviour event quality and bot traffic
Async Flow and Reliability
Queries are synchronous through the APIs, while index sync and behaviour capture are asynchronous. Catalog changes flow through the Indexer, and click events are buffered in Kafka for offline evaluation.
Upgrade Path
Add a relational catalog datastore behind the Catalog Service and a training pipeline that consumes behaviour events, then add API auth and per-client rate limiting in front of the public search endpoints.