A blockchain node stores data in a format optimized for consensus and verification, not for querying. If you want to know the current balance of an Ethereum address, a node can tell you. But if you want to know all the transactions that address has ever made, or all the tokens it holds, or its interaction history with a specific smart contract, you need an indexer.
Indexers work by reading every block from a blockchain, decoding the transactions and events, and storing the processed data in a database optimized for the types of queries users actually want to run. This is conceptually similar to how a search engine crawls the web and builds an index. The raw HTML of every webpage exists, but search engines make it useful by indexing and organizing it.
The Graph is the most widely known decentralized indexing protocol. It allows developers to define subgraphs that specify exactly what data to extract from a blockchain and how to store it. When a relevant event occurs on-chain, the subgraph mapping functions process the event and update the stored data. Applications then query the subgraph using GraphQL rather than interacting with the blockchain directly.
The indexing process is computationally intensive. For Ethereum, an indexer needs to process every transaction in every block since genesis, decode the input data and event logs, and maintain a running state of all relevant data. A full Ethereum archive node stores over 15 terabytes of data. Processing this into useful indexed formats requires significant computing resources and can take days or weeks for a complete historical sync.
Event logs are the primary data source for most indexers. When a smart contract emits an event, like a Transfer event for ERC-20 tokens, it creates a log entry that indexers can efficiently filter and process. Well-designed contracts emit events for every significant state change, making indexing straightforward. Contracts that modify state without emitting events are harder to index because the indexer needs to trace transaction execution to detect changes.
Different indexing services optimize for different use cases. Dune Analytics provides a SQL interface to indexed blockchain data, making it accessible to analysts who know SQL but not Solidity. Nansen combines on-chain indexing with wallet labeling, connecting addresses to known entities. Flipside provides curated datasets for specific protocols. Each makes different choices about what to index and how to present it.
Real-time indexing presents additional challenges. Block production on Ethereum happens every 12 seconds, and an indexer needs to process each new block data and update its database before the next block arrives. During periods of high activity, a single block might contain thousands of relevant events. The indexer needs to handle this load while also serving query traffic from applications.
Reorg handling is a subtle but critical aspect of indexing. Blockchains occasionally reorganize, meaning a previously processed block gets replaced by a different block at the same height. An indexer needs to detect these reorgs and roll back any data changes from the replaced block. Failing to handle reorgs correctly can lead to inconsistent data that shows transactions or state changes that never actually happened.
For traders and analysts, indexers are what make on-chain analysis practical. Without them, monitoring whale wallets, tracking token flows, analyzing DeFi protocol usage, or building trading signals from on-chain data would require running your own infrastructure at significant cost. The quality and coverage of the indexer you rely on directly affects the quality of your analysis.