How search works
Hybrid search combines exact, keyword and semantic matching, and each result says why it matched. How it stays silent, how it was tuned, and what the retrieval evaluation shows and does not show.
SKaaS finds skills by task, not by name. A search runs three channels at once and fuses their rankings.
What does hybrid search combine?
The exact channel matches a skill's name or identifier. The keyword channel is full-text search over the name, description and search text, with English stemming, so "rotating" and "rotation" meet. A name is a few words, and a match in a field that short scores high whatever the word is, so the name carries a lower weight than the description and the search text. The semantic channel compares the meaning of your query with the meaning of each skill using embeddings. The three rankings are fused into one list by weighted reciprocal rank fusion, and a small positive-intent term, which counts the words that a query shares with a skill's name and description, breaks ties. Ask for a mode: hybrid (the default), lexical or semantic. The exact channel always joins a hybrid search.
Why did this result match?
Each hit carries a "why this matched" explanation: which channels found it, its rank in each, and the matched terms. When the semantic channel drives a result, the library labels it "close in meaning to your query". Raw scores are not shown as authority.
What does the search skip?
A search only returns skills the caller may read, in a state that allows search. Deprecated, archived and quarantined skills are left out. In the public catalog, a public skill appears only after a moderator has approved it, so an unreviewed public skill is left out of public search too. Supporting files are not searched in the baseline: search covers each skill's routing metadata and guide text.
Can search say there is no match?
Yes. A client can set a minimum confidence, and the response then reports that it abstained instead of returning a weak guess. The prompt hook uses the same rule to stay silent. A hit counts as evidence when the query is its name, when its semantic similarity reaches the threshold of the embedding space, or when it shares at least two stemmed terms with the query. A response that abstains says so, and returns no hits.
The similarity threshold belongs to an embedding space, because models spread their similarities differently. A model that packs unrelated texts at 0.55 needs a higher threshold than one that puts them at 0.25. The default is 0.44. A deployment sets its own with SKAAS_EMBEDDING_SIMILARITY_THRESHOLD.
How does a client learn the threshold of a deployment?
The threshold and the embedding space belong to the deployment, so a client that routes a prompt or checks a skill on the person's machine should use the same numbers as the service. Two answers carry them. GET /v1/whoami has a search member with semantic_threshold, which is null when the deployment sets none and the default applies, and embedding_space, which is null when the deployment has no embedding provider. The read MCP server reports the same values in the search member of skills_status, as embedding_space and semantic_threshold. A service of an earlier version leaves semantic_threshold out, and a client then uses the default. The skaas test command reads it for its offline run: see the command-line tool.
Is semantic search always on?
No. A self-hosted deployment without an embedding provider runs keyword and exact matching only, and the response says that the semantic channel is disabled. A degraded result always says that it is degraded.
Which embedding models are known?
The package that makes embeddings keeps a profile for each model family that the evaluation ran. A profile holds the prefix that the model's authors say to put before a document and before a query, and, for the variants that were measured, the silence threshold of its space.
| Model family | Query prefix | Measured variants | Silence threshold |
|---|---|---|---|
nomic-embed-text |
search_query: (documents get search_document: ) |
nomic-embed-text, :latest, :v1.5 |
0.55 |
mxbai-embed-large |
Represent this sentence for searching relevant passages: |
mxbai-embed-large, :latest, :335m |
0.50 |
qwen3-embedding |
An instruction line that asks for the skill that fits the task | qwen3-embedding:4b |
0.55 |
A model of a known family that is another size or version than the measured one gets no threshold from the profile and keeps the default, because a threshold that was measured on one variant says nothing about another. A model of another family has no profile. The thresholds are valid only with the prefixes of the table. With the openai-compatible provider, the server looks up the model name in these profiles. It adds the prefixes of a known family to every document and every query, and it uses the measured threshold of a measured variant. A threshold that you set with SKAAS_EMBEDDING_SIMILARITY_THRESHOLD wins over the measured one. Vectors made with a prefix are not the vectors of the same model without one, so such a space carries a different name, and an artifact that holds a vector of the unprefixed space is embedded again. The voyage and hash providers have no profile, and for them the server adds no prefix and applies no measured threshold. A deployment that runs another model, or a variant that was not measured, keeps the default of 0.44 until you set a threshold chosen on its own prompts, and the readiness answer of the server notes that it uses the default. A threshold is a property of a space and of a corpus.
The rule that chose the thresholds read the development cases only: the highest value in steps of 0.01 at which the share of right silences stayed at or above 0.97 over those cases.
What does the evaluation say?
The retrieval evaluation runs the real search path against a corpus of 100 artifacts (90 skills and 10 references) in 12 domains, with 500 cases: 300 that have an answer, 100 hard negatives and 100 that should get silence. Every fifth case of each kind is held out, which is 100 cases. The tuning read the other 400 only, and the held-out cases were run afterwards, once for each of three embedding models. Every number below is from the committed reports in eval/reports/.
Two labels of caution come first. A language model wrote the labels, and no person has reviewed them. All numbers come from one run on one machine, with the database on the same machine, so the timings do not describe a hosted deployment.
What the tuning changed:
| Setting | Before | After |
|---|---|---|
| Weight of the positive-intent term | 0.04 | 0.005 |
| Weights of name, description, search text | 6, 3, 1 | 0.5, 1, 1 |
| Analyzer of the keyword index | lucene.standard |
lucene.english |
| Silence threshold of the space | 0.52 for nomic-embed-text, else 0.44 |
0.55, 0.52 and 0.55 for the three models |
On the 320 development cases that have an answer, the tuning captures record the before and after figures for each model. These are development measurements, separate from the held-out check below.
| Model | MRR@10 before | MRR@10 after | Recall@5 before | Recall@5 after |
|---|---|---|---|---|
nomic-embed-text |
67.5 | 78.3 | 86.6 | 90.6 |
mxbai-embed-large |
70.0 | 81.2 | 89.1 | 92.2 |
qwen3-embedding:4b |
74.6 | 85.3 | 92.2 | 95.6 |
The live store measurement on the development cases gives MRR@10 and Recall@5 of 78.0 and 90.6 for the first model, 81.3 and 92.2 for the second, and 85.1 and 95.6 for the third. Channel results can differ slightly between the tuning capture and the later store run.
On the held-out cases, with the three models in the order nomic-embed-text, mxbai-embed-large and qwen3-embedding:4b:
| Gate | Result | Status |
|---|---|---|
| Recall@5 is at least 0.90 | 0.950, 0.963 and 0.963 | Passes |
| Abstention precision is at least 0.95 | 1.000 for all three | Passes |
| Warm search p95 is at most 400 ms | Passes on this machine. 27.8 ms with the first model | Passes |
| No metric falls more than one point below the term-overlap baseline | Abstention recall is 0.60, 0.70 and 0.70 against 0.95 for the baseline | Fails |
| The chosen embedding is within one point of the automated baseline | Did not run: needs a hosted baseline | Not run |
| Hybrid search p95 is at most 350 ms on 5,000 documents | Did not run: needs 5,000 documents and a durable-object host | Not run |
The gate that fails says that on one metric, staying silent when no skill fits, the search does worse than a simple count of shared terms: it is silent on 12, 14 or 14 of the 20 held-out cases that expect silence, and the baseline is silent on 19. Every other metric is far above that baseline on the held-out cases. For the first model, Recall@1 is 0.650 against 0.388, Recall@5 is 0.950 against 0.700 and the abstention precision is 1.000 against 0.463. No threshold of any gate was moved to make a gate pass.
Read the margins with the noise in mind. The held-out split has 80 cases that have an answer, so Recall@5 of 0.950 has a 95 percent interval of about 0.88 to 0.98. With 20 held-out cases that expect silence, one case is five points of recall.
The report names the likely causes in the order of the evidence. The embedding space does not separate the two kinds of query well enough, and the similarity of the best hit tells them apart better with a stronger model. The second way through, on two shared terms, has a ceiling, because five of the 20 held-out cases that expect silence get an answer whatever the threshold is. And the labels and the comparison itself are limited. What would close the gap, in the report's order: a stage that decides whether any skill fits, with a reranker or a small judge; the best embedding model that the deployment can run; reviewed labels with abstention cases written by people from real prompts; and a threshold chosen on the deployment's own prompts. On the development cases, qwen3-embedding:4b is the best of the three for retrieval by a wide margin, so a deployment that runs a local model should prefer it, and on silence the three are within the noise of 20 cases.
The tuning has a cost. A deployment that uses the hash provider, which carries no meaning, got worse, because the larger tie-break term had been propping up a semantic channel that was noise. A deployment with no embedding model uses the keyword channel and is not affected. A model whose space is as poor as a hash is.