Product search & taxonomy

Once a manuscript has become a Product — a sellable thing built on a Work, its Expressions and Manifestations — the next problem is finding it again. A reader on packtpub.com types "go microservices"; a bookseller looks up an exact ISBN; a recommendation panel wants "products like this one". All three are the same underlying question asked in different ways, and the platform answers them with a single search facility over the product catalogue.

Product search works at the product level, not the block level. The Content Lake indexes individual blocks of content so a reader can jump to the exact paragraph that answers a question; product search indexes each product as one thing, because a storefront wants to return the book, not paragraph 14 of chapter 3. The two share the same machinery and the same query vocabulary — the difference is only the grain of what a result points at.

The second half of this page is about a deliberate boundary. The platform gives every product an embedding — a position in a shared vector space that captures what the product is about — and exposes those vectors through the API. What it deliberately does not do is decide which category a product belongs in. Taxonomy is a client concern. packtpub.com organises by language and skill level; gamedevassembly.com organises by game engine and discipline; a subscription app organises by learning path. The same product can sit in different categories on different storefronts, all driven by the one embedding the platform provides.

How products are found

Product search follows the same hybrid architecture as the Content Lake search pipeline: it runs two ways of matching in parallel and merges them by rank. Keyword matching (BM25) finds products whose title, subtitle, description, or author text contains the words in the query — precise, literal, unbeatable for a known title or ISBN. Vector matching finds products whose meaning is close to the query even when no words overlap, so a search for "container orchestration" surfaces a Kubernetes book that never uses that exact phrase. Because product records are short and well-titled, that merged ranking is usually the final ranking — a deeper re-ranking pass is available on request for relevance-critical surfaces, rather than a cost every query pays.

And crucially, search matches what each storefront shows. A catalogue can override a product's title, description, or price for its own audience (catalogue overrides), and the index carries those resolved presentations: a product retitled "Microservices mit Go" for the German storefront is findable by that title there, while packtpub.com matches the original. A storefront scopes its searches to its catalogue and gets its own presentation back — same product, same underlying meaning, the right words for that shop window.

flowchart LR
    Q["Query<br/>'go microservices'"] --> KW["Keyword match<br/>BM25 over title,<br/>description, authors"]
    Q --> VEC["Vector match<br/>nearest embeddings"]
    KW --> BLEND["Blend &amp; re-rank"]
    VEC --> BLEND
    BLEND --> R["Ranked products<br/>prod_go_micro · prod_k8s · …"]

Every product carries a small set of indexed attributes — enough to match on and to filter by, without duplicating the full product record. The product_model attribute is worth calling out: it records how the product's formats are sourced — artifact (all uploaded distributables), rendered (all generated from the Content Lake), or mixed — and is derived from the format sources, not set by hand. See Product types for what that sourcing means.

The last row is worth pausing on. For Content Lake-backed products, the index rolls up the knowledge-graph topics the content actually covers — with a salience threshold, so it means substantially about, not mentions once in passing. "Products about Kubernetes" becomes a single filtered search, and a "topics covered" facet comes with it. Finding the exact passages about Kubernetes remains Content Lake search's job — the two indexes answer different grains of the same question.

Search modes

One query facility serves the three questions from the opening. The search_mode parameter picks the retrieval strategy:

• hybrid (default) — blends vector similarity and BM25 full-text matching. The right choice for general product discovery, where a reader's words may or may not match the catalogue's wording.
• keyword — BM25 full-text ranking only. Best for known-item retrieval: an exact title, an author name, an ISBN. No semantic fuzz, just the literal match.
• vector — approximate nearest-neighbour only. Best for meaning-first queries where wording is unreliable.
• similar — "products like these": give it up to ten product IDs and it ranks the rest of the catalogue by closeness to them — the "customers also viewed" rail in a single call, no text query at all.

Filtering, pagination and re-ranking

All three modes support the same refinements, following the patterns in the Content Lake search specification:

• Filtering narrows the candidate set natively during retrieval — by product_type, language, state, tags, price range, and the other indexed attributes — improving both speed and relevance. Product search is fast by construction: the whole catalogue is a small index, and a filtered hybrid query targets well under a quarter of a second.
• Facets, counts, and ordering ride the same request: the counts behind a storefront's filter rail, the exact total, and attribute orderings — "newest", "lowest price", and "best selling" powered by daily-refreshed sales and readership data, which can also gently boost popular products inside a relevance search.
• Pagination uses search sessions. A search returns a session ID (srch_); re-sending the same query with it fetches the next page with everything already seen automatically excluded — no offsets, no duplicates. Ordered browse lists page by a stateless cursor instead. See search sessions.
• Re-ranking is an opt-in second scoring pass over the top candidates, following the Content Lake's re-ranking approach — for the surfaces that want maximum precision and can spare the extra fraction of a second.

POST /v0/products/search
Authorization: Bearer <token>

{
  "query": "go microservices",
  "search_mode": "hybrid",
  "catalogue_id": "cat_packtpub",
  "filters": {
    "product_types": ["BOOK"],
    "languages": ["en"],
    "states": ["PUBLISHED"]
  },
  "limit": 20
}

{
  "search_id": "srch_go_micro_a1b2",
  "results": [
    {
      "score": 0.91,
      "product": {
        "product_id": "prod_go_micro",
        "title": "Go Microservices",
        "language": "en",
        "price": 29990000,
        "currency": "USD"
      },
      "highlight": "<<Go>> <<Microservices>>"
    }
  ],
  "cumulative_returned": 20,
  "has_more": true
}

Taxonomy is a client concern

Search finds a product when someone is looking for it. Taxonomy is the other half of discovery: the browsable structure of categories and subcategories a storefront lays over its catalogue so people can find products without knowing what to search for. The platform draws a hard line here — it provides the raw material for taxonomy, and leaves the taxonomy itself entirely to the client application.

The reason is that no single category scheme fits every destination.

flowchart TD
    P["One product<br/>prod_go_micro<br/>(one embedding)"]
    P --> S1["packtpub.com<br/>Category: Go · Intermediate"]
    P --> S2["gamedevassembly.com<br/>Category: Backend Tools"]
    P --> S3["Subscription app<br/>Path: Cloud-Native Track"]

The same product sits in Go / Intermediate on packtpub.com, under Backend Tools on gamedevassembly.com, and inside a Cloud-Native learning track on a subscription app — all at once, and all driven by the one embedding the platform already computed. None of those schemes is more correct than the others; they answer to different audiences, and they change on their own schedules. Baking any of them into the platform would force every storefront to inherit one shop's idea of how the world is organised.

What the platform provides

The platform gives every product a position in a shared vector space — its embedding — and exposes those vectors through the API so a client can build whatever categorisation it likes on top. A client can:

1. Retrieve one embedding — fetch the vector for a single product, for real-time classification or an on-the-fly similarity lookup.
2. Batch-export embeddings — export the vectors for a whole catalogue or a filtered slice (say, all published English books) to build or retrain a client-side taxonomy model.
3. Nearest-neighbour queries — use vector search mode to find the products closest to a given vector, to populate a category page or a recommendation rail.

How the client turns those vectors into categories — k-means clustering, a trained classifier, or plain manual curation — is entirely its own business. The platform stores, manages, and enforces no taxonomy of its own.

In practice a storefront scopes its taxonomy to its catalogue — there is no point classifying products it does not carry — so the batch-export endpoint accepts a catalogue ID and returns embeddings for exactly that product set.

Related pages

• Products overview — the Work → Expression → Manifestation → Product ladder and the product lifecycle.
• Product types — where product_model (artifact / rendered / mixed) comes from.
• Catalogues — the product set a storefront's taxonomy is scoped to.
• Content Lake search — the block-level hybrid search this page mirrors, and the source of the shared session, filter, and re-ranking patterns.
• Explorer — Chapter VI — search, visualised.