perf(monitor): 4 fewer queries w/ equivalent scans #95
+292
−528
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This combines queries for `count` (live), `future_count`, and `erroring_count` into a combined query for "live" jobs. It also combines `locked_count`/`working_count` and `workable_count` into a combined query for "pending" jobs. The reason this is still two queries and not one is because on PostgreSQL, `locked_count, `working_count`, and `workable_count` all require heap access (i.e. cannot be `Index Only` scans) -- because `locked_at` is not part of any index. Given this, we benefit from restricting the scan to as few rows as necessary, and `jobs.pending` is exactly the combined superset that supports all three counts. Meanwhile, the "live" job query is an `Index Only` scan and can theoretically benefit from not having to hit the heap. (Though unfortunately it sometimes does require heap visits, depending on how recently a given page has been VACUUMed.) /no-platform stack-info: PR: #95, branch: smudge/stack/6
smudge
force-pushed
the
smudge/stack/5
branch
from
c22e230 to
c4546d6
Compare
smudge
force-pushed
the
smudge/stack/6
branch
from
f799f7b to
3082edc
Compare
This was referenced Feb 10, 2026
smudge
force-pushed
the
smudge/stack/6
branch
from
3082edc to
b473e61
Compare
Open
smudge
force-pushed
the
smudge/stack/6
branch
from
b473e61 to
247b4b3
Compare
smudge
force-pushed
the
smudge/stack/6
branch
from
247b4b3 to
4648ab9
Compare
smudge
force-pushed
the
smudge/stack/5
branch
from
aebb0b0 to
16461fc
Compare
smudge
added a commit
that referenced
this pull request
Feb 10, 2026
This combines queries for `count` (live), `future_count`, and `erroring_count` into a combined query for "live" jobs. It also combines `locked_count`/`working_count` and `workable_count` into a combined query for "pending" jobs. The reason this is still two queries and not one is because on PostgreSQL, `locked_count, `working_count`, and `workable_count` all require heap access (i.e. cannot be `Index Only` scans) -- because `locked_at` is not part of any index. Given this, we benefit from restricting the scan to as few rows as necessary, and `jobs.pending` is exactly the combined superset that supports all three counts. Meanwhile, the "live" job query is an `Index Only` scan and can theoretically benefit from not having to hit the heap. (Though unfortunately it sometimes does require heap visits, depending on how recently a given page has been VACUUMed.) /no-platform stack-info: PR: #95, branch: smudge/stack/6
smudge
force-pushed
the
smudge/stack/6
branch
from
4648ab9 to
e1c6a8d
Compare
smudge
force-pushed
the
smudge/stack/5
branch
from
16461fc to
6d37d93
Compare
smudge
added a commit
that referenced
this pull request
Feb 10, 2026
This combines queries for `count` (live), `future_count`, and `erroring_count` into a combined query for "live" jobs. It also combines `locked_count`/`working_count` and `workable_count` into a combined query for "pending" jobs. The reason this is still two queries and not one is because on PostgreSQL, `locked_count, `working_count`, and `workable_count` all require heap access (i.e. cannot be `Index Only` scans) -- because `locked_at` is not part of any index. Given this, we benefit from restricting the scan to as few rows as necessary, and `jobs.pending` is exactly the combined superset that supports all three counts. Meanwhile, the "live" job query is an `Index Only` scan and can theoretically benefit from not having to hit the heap. (Though unfortunately it sometimes does require heap visits, depending on how recently a given page has been VACUUMed.) /no-platform stack-info: PR: #95, branch: smudge/stack/6
smudge
force-pushed
the
smudge/stack/6
branch
from
e1c6a8d to
2fcc09e
Compare
smudge
force-pushed
the
smudge/stack/5
branch
from
6d37d93 to
7d7eaf4
Compare
smudge
added a commit
that referenced
this pull request
Feb 11, 2026
This combines queries for `count` (live), `future_count`, and `erroring_count` into a combined query for "live" jobs. It also combines `locked_count`/`working_count` and `workable_count` into a combined query for "pending" jobs. The reason this is still two queries and not one is because on PostgreSQL, `locked_count, `working_count`, and `workable_count` all require heap access (i.e. cannot be `Index Only` scans) -- because `locked_at` is not part of any index. Given this, we benefit from restricting the scan to as few rows as necessary, and `jobs.pending` is exactly the combined superset that supports all three counts. Meanwhile, the "live" job query is an `Index Only` scan and can theoretically benefit from not having to hit the heap. (Though unfortunately it sometimes does require heap visits, depending on how recently a given page has been VACUUMed.) /no-platform stack-info: PR: #95, branch: smudge/stack/6
smudge
force-pushed
the
smudge/stack/6
branch
from
2fcc09e to
2a963ff
Compare
smudge
force-pushed
the
smudge/stack/6
branch
from
2a963ff to
665facb
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stacked PRs:
perf(monitor): 4 fewer queries w/ equivalent scans
This combines queries for
count(live),future_count, anderroring_countinto a combined query for "live" jobs.It also combines
locked_count/working_countandworkable_countinto a combined query for "pending" jobs.
The reason this is still two queries and not one is because on
PostgreSQL,
locked_count,working_count, andworkable_countall require heap access (i.e. cannot beIndex Onlyscans) -- becauselocked_atis not part of any index. Given this, we benefit from restricting the scan to as few rows as necessary, andjobs.pending` isexactly the combined superset that supports all three counts.
Meanwhile, the "live" job query is an
Index Onlyscan and cantheoretically benefit from not having to hit the heap. (Though
unfortunately it sometimes does require heap visits, depending on how
recently a given page has been VACUUMed.)
/no-platform