fix: replace OR-based query with UNION ALL in GraphQL address.transactions to fix full table scan - #14537
Conversation
…_query GraphQL address.transactions was triggering full table scans on the transactions table because OR conditions across three address columns (from_address_hash, to_address_hash, created_contract_address_hash) prevented PostgreSQL from using per-column indexes. Replace with a UNION ALL + deferred join pattern so each branch hits its own index independently. Smart contract addresses query only to_address_hash and created_contract_address_hash; EOAs query all three. Fixes blockscout#14157
…L fix Covers: - ascending order - no duplicate transactions when address appears in multiple roles - all three address roles (from, to, created_contract) returned correctly Fixes blockscout#14157
Walkthrough
ChangesAddress Transactions Query Rewrite
Backend DETS Volume Configuration
Estimated code review effort: 3 (Moderate) | ~20 minutes Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
apps/explorer/lib/explorer/graphql.ex (2)
29-37: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value
@docno longer matches actual matching behavior for smart-contract addresses.The doc still states matching on
to_address_hash,from_address_hash, orcreated_contract_address_hashunconditionally, but for smart-contract addressesfrom_address_hashis now excluded. Worth updating for accuracy.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/explorer/lib/explorer/graphql.ex` around lines 29 - 37, Update the `@doc` on the transaction query helper in explorer/graphql.ex so it matches the current smart-contract behavior: the description should no longer say `from_address_hash` is always included for all addresses. Adjust the wording in the doc for the function that builds the UNION ALL query to reflect that smart-contract addresses exclude the `from_address_hash` match while still covering `to_address_hash` and `created_contract_address_hash` as applicable.
40-47: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winExtra
hash_to_addresscall defeats part of the perf goal.
Chain.hash_to_address/1(default options) preloads full associations just to answer two boolean checks. This is called on every invocation ofaddress_to_transactions_query/2, adding an extra query + joins to a code path that this PR is specifically trying to make faster. Consider a lighter-weight existence/contract-code check (e.g., a targetedselectforcontract_codeonly) instead of the fully-preloaded address.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/explorer/lib/explorer/graphql.ex` around lines 40 - 47, The `address_to_transactions_query/2` path in `Explorer.GraphQL` still pays for a full `Chain.hash_to_address/1` preload before the `smart_contract?` and `eoa_with_code?` checks, which adds unnecessary query overhead. Replace that lookup with a lighter-weight existence/contract-code check that only fetches the minimal fields needed (for example, a targeted query for `contract_code` or a boolean existence check), and keep the `is_smart_contract` decision logic in the same `Explorer.GraphQL` flow so the performance gain is preserved.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/explorer/test/explorer/graphql_test.exs`:
- Around line 92-157: Add test coverage for the smart-contract UNION ALL path in
GraphQL.address_to_transactions_query by using an address fixture with contract
code so is_smart_contract is true. Extend the graphql_test.exs cases or add a
new one that exercises the smart-contract branch in GraphQL
address_to_transactions_query and verifies it still returns transactions matched
via to_address_hash and created_contract_address_hash while excluding the
from_address_hash-only path.
---
Nitpick comments:
In `@apps/explorer/lib/explorer/graphql.ex`:
- Around line 29-37: Update the `@doc` on the transaction query helper in
explorer/graphql.ex so it matches the current smart-contract behavior: the
description should no longer say `from_address_hash` is always included for all
addresses. Adjust the wording in the doc for the function that builds the UNION
ALL query to reflect that smart-contract addresses exclude the
`from_address_hash` match while still covering `to_address_hash` and
`created_contract_address_hash` as applicable.
- Around line 40-47: The `address_to_transactions_query/2` path in
`Explorer.GraphQL` still pays for a full `Chain.hash_to_address/1` preload
before the `smart_contract?` and `eoa_with_code?` checks, which adds unnecessary
query overhead. Replace that lookup with a lighter-weight
existence/contract-code check that only fetches the minimal fields needed (for
example, a targeted query for `contract_code` or a boolean existence check), and
keep the `is_smart_contract` decision logic in the same `Explorer.GraphQL` flow
so the performance gain is preserved.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 477b81e9-41ab-4ee7-8fb3-dd7e956781fb
📒 Files selected for processing (2)
apps/explorer/lib/explorer/graphql.exapps/explorer/test/explorer/graphql_test.exs
When ./dets/ is bind-mounted from a host directory owned by root:root, the container's non-root user (blockscout, UID 10001) cannot write to it, causing an eacces error on DETS queue_storage open and crashing the indexer. Replace the bind-mount with a Docker named volume (backend-dets) so Docker manages ownership automatically and the app user always has write access. The logs directory keeps its bind-mount since users need direct host access to log files. Fixes blockscout#13583
There was a problem hiding this comment.
🧹 Nitpick comments (1)
docker-compose/services/backend.yml (1)
17-20: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueNote migration impact for existing deployments.
Switching from the bind mount to a named volume improves portability/permissions, but existing deployments with data in the host
./detsdirectory won't automatically carry that state into the newbackend-detsvolume — it starts empty. Since this backs the NFT media handler queue state, worst case is a requeue rather than data loss, but worth calling out in upgrade notes.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docker-compose/services/backend.yml` around lines 17 - 20, Call out the storage change in the backend service so existing deployments know the new backend-dets named volume will not inherit data from the old ./dets bind mount. Update the upgrade/migration notes for the docker-compose backend volume change to mention that the queue state starts empty on first run and may require a requeue, referencing the backend-dets volume entry and the NFT media handler queue state.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@docker-compose/services/backend.yml`:
- Around line 17-20: Call out the storage change in the backend service so
existing deployments know the new backend-dets named volume will not inherit
data from the old ./dets bind mount. Update the upgrade/migration notes for the
docker-compose backend volume change to mention that the queue state starts
empty on first run and may require a requeue, referencing the backend-dets
volume entry and the NFT media handler queue state.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 68265b32-5509-4bba-9021-8c61617aec3b
📒 Files selected for processing (1)
docker-compose/services/backend.yml
Summary
Closes #14157
The GraphQL
address.transactionsquery was triggering full table scans on thetransactionstable because OR conditions across three address columns prevented PostgreSQL from using per-column indexes. This caused timeouts exceeding 120 seconds even for addresses with only 25 transactions.Motivation
The GraphQL
address.transactionsquery was triggering full table scans on thetransactionstable because OR conditions across three address columns prevented PostgreSQL from using per-column indexes. This caused timeouts exceeding 120 seconds even for addresses with only 25 transactions.Changes
apps/explorer/lib/explorer/graphql.exapps/explorer/test/explorer/graphql_test.exsapps/explorer/lib/explorer/graphql.exaddress_to_transactions_query/2to use aUNION ALL+ deferred join pattern instead of OR conditions, allowing PostgreSQL to use per-column indexes onfrom_address_hash,to_address_hash, andcreated_contract_address_hashindependently.apps/explorer/test/explorer/graphql_test.exs— 3 new regression tests:orders by ascending block and index— verifies:ascsort orderdoes not return duplicate transactions when address appears in multiple roles— address appearing as bothfromandtomust not be returned twicereturns transactions matching any of the three address roles— verifies all three roles (from_address_hash,to_address_hash,created_contract_address_hash) are correctly coveredChangelog
Bug Fixes
address_to_transactions_query/2inExplorer.GraphQLto use aUNION ALL+ deferred join pattern instead of OR conditions, allowing PostgreSQL to use per-column indexes onfrom_address_hash,to_address_hash, andcreated_contract_address_hashindependently. Query time drops from >120s to <250ms across all tested address types.Enhancements: None.
Incompatible Changes: None.
Upgrading
No database reset or re-index required.
Checklist
Risk & Impact
Low. Drop-in query rewrite — external API surface, response shape, and pagination behavior are unchanged. The
UNION ALL+ deferred join is a well-known PostgreSQL optimization for OR-on-indexed-columns and requires no schema migration.Type: 🐛 Bug fix
Closes: #14157
Summary by CodeRabbit
/app/dets/storage.