Skip to content

Conversation

@PatrickSys
Copy link
Owner

Summary

  • Evaluation framework: Frozen fixture set with reproducible methodology (committed before code changes per AGENTS.md integrity rules)
  • Search quality: RRF fusion (k=60), intent-aware query classification (5 types), hard test-file filtering, import-graph proximity reranking, file-level deduplication
  • Embedding upgrade: @xenova/transformers v2 → @huggingface/transformers v3 (granite model, 8192 context)
  • Chunk optimization: 100→50 lines, overlap 10→0

BREAKING CHANGE

Re-indexing required after upgrade. Existing .codebase-context/ indices from v1.5.x are incompatible.

Test plan

  • pnpm build succeeds
  • pnpm test — 135/135 passing
  • No README.md changes in this PR
  • No eval percentage claims in any file

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your free trial has ended. If you'd like to continue receiving code reviews, you can add a payment method here.

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 08d243d956

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@PatrickSys PatrickSys force-pushed the feat/v1.6.0-search-quality branch from 08d243d to 91b6633 Compare February 10, 2026 21:12
@PatrickSys PatrickSys force-pushed the feat/v1.6.0-search-quality branch from 91b6633 to bc836d1 Compare February 10, 2026 21:26
@PatrickSys
Copy link
Owner Author

@codex review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bc836d15be

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@PatrickSys PatrickSys force-pushed the feat/v1.6.0-search-quality branch from bc836d1 to 26aa3f7 Compare February 10, 2026 21:45
@PatrickSys
Copy link
Owner Author

@codex review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 26aa3f7f2a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@PatrickSys PatrickSys force-pushed the feat/v1.6.0-search-quality branch from 26aa3f7 to f89e7a3 Compare February 10, 2026 21:57
…nker

- Reciprocal Rank Fusion (k=60, rank-based scoring)
- Intent-aware query classification (5 types)
- Hard test-file filtering (0% spec contamination)
- Import-graph proximity reranking
- File-level deduplication
- Chunk optimization (100→50 lines, 0 overlap)
- Embedding upgrade (@huggingface/transformers v3)

BREAKING CHANGE: Re-indexing required due to model and chunking changes
@PatrickSys PatrickSys force-pushed the feat/v1.6.0-search-quality branch from f89e7a3 to 5346766 Compare February 10, 2026 22:02
@PatrickSys
Copy link
Owner Author

@codex review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 534676663a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

const reranked = scored.map(({ result, crossScore }) => ({
...result,
// Normalize cross-encoder score to 0-1 range for the top slot
score: crossScore

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Normalize reranker logits before overwriting result scores

This writes raw cross-encoder logits into SearchResult.score, but downstream quality logic (assessSearchQuality) interprets scores as normalized confidence values (for example, low-score and separation thresholds). When reranking is triggered on ambiguous queries (enabled by default), negative or unbounded logits can incorrectly downgrade otherwise good results to low_confidence and distort confidence signals returned to users. Convert reranker outputs to a bounded confidence scale (or store them separately) before replacing score.

Useful? React with 👍 / 👎.

Comment on lines +585 to +586
if (this.isTemplateOrStyleFile(chunk.filePath)) {
combinedScore *= 0.75;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Restrict template/style demotion to behavioral queries

The template/style penalty currently applies to every query, even though the comment says it is for behavioral queries. This means style- or template-focused searches (for example, layout/theme/CSS questions) always get a 25% penalty on the most relevant .html/.scss files, which can push unrelated implementation files above the correct targets and reduce retrieval quality.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant