Refine data documentation in /developers/docs/data-and-analytics/ #13352
Conversation
andrewhong5297
requested review from
corwintines,
minimalsm,
pettinarip and
wackerow
as code owners
❌ Deploy Preview for ethereumorg failed. Why did it fail? →
|
|
This issue is stale because it has been open 30 days with no activity. |
github-actions
bot
added
the
Status: Stale
There was a problem hiding this comment.
Hey @andrewhong5297! So sorry for the delays on this. Circling back, gave it a read and left my thoughts.... Most of my code suggestions are just clerical stuff, but a few that I'd like addressed before we pull this in.
- (Clerical fixes)
- "Recent" verbiage... would reword some stuff to be a bit more future-proof
- 2024->2025, anything critical we should add?
- Page length... split into two?
- Empty further reading
- Thoughts on repeating block explorer listings? Would prefer to have one list of these on their dedicated page and link to that
Otherwise I think this is great, and a really nice update to the page. Would love to get this in soon. Would you be available for any updates @andrewhong5297? Thanks again!
| --- | ||
| title: Data and analytics | ||
| description: How to get on-chain analytics and data for use in your dapps | ||
| title: Data and Analytics |
There was a problem hiding this comment.
(We use sentence casing for all titles)
| title: Data and Analytics | |
| title: Data and analytics |
| ## Introduction {#Introduction} | ||
|
|
||
| As utilization of the network continues to grow, an increasing amount of valuable information will exist in the on-chain data. As the volume of data rapidly increases, calculating and aggregating this information to report upon or drive a dapp can become a time and process heavy endeavor. | ||
| As you get into crypto, you're likely to see stats, charts, and APIs across dozens of different websites (and github repos). This is a stark contrast from traditional tech or finance where there is just one trusted source (i.e. Bloomberg). This is due to the open data nature of the industry, where all data can be freely downloaded from client nodes. |
There was a problem hiding this comment.
| As you get into crypto, you're likely to see stats, charts, and APIs across dozens of different websites (and github repos). This is a stark contrast from traditional tech or finance where there is just one trusted source (i.e. Bloomberg). This is due to the open data nature of the industry, where all data can be freely downloaded from client nodes. | |
| As you get into crypto, you're likely to see stats, charts, and APIs across dozens of different websites (and GitHub repos). This is a stark contrast from traditional tech or finance where there is just one trusted source (i.e. Bloomberg). This is due to the open data nature of the industry, where all data can be freely downloaded from client nodes. |
| As you get into crypto, you're likely to see stats, charts, and APIs across dozens of different websites (and github repos). This is a stark contrast from traditional tech or finance where there is just one trusted source (i.e. Bloomberg). This is due to the open data nature of the industry, where all data can be freely downloaded from client nodes. | ||
|
|
||
| Leveraging existing data providers can expedite development, produce more accurate results, and reduce ongoing maintenance efforts. This will enable a team to concentrate on the core functionality their project is trying to provide. | ||
| However, that data is stored in hex/binary form and is hardly legible or usable. As an ecosystem, we've had to work together to share queries, tools, tables, and labor to make crypto data accessible for everyone. Each year, we see giant leaps in how we can combine onchain/offchain data in analytics and applications. |
There was a problem hiding this comment.
Maybe just a note on how it's also not indexed by default or easily searchable?
There was a problem hiding this comment.
ah, see it's covered below, that'll do
| However, that data is stored in hex/binary form and is hardly legible or usable. As an ecosystem, we've had to work together to share queries, tools, tables, and labor to make crypto data accessible for everyone. Each year, we see giant leaps in how we can combine onchain/offchain data in analytics and applications. | ||
|
|
||
| ## Prerequisites {#prerequisites} | ||
| In this overview page, we'll cover all the parts of the crypto data stack as they stand today (last updated in 2024). |
There was a problem hiding this comment.
@andrewhong5297 Certainly apologize for delays here... anything critical that has developed since that you'd like to add now that we're in 2025?
| In this overview page, we'll cover all the parts of the crypto data stack as they stand today (last updated in 2024). | ||
|
|
||
| You should understand the basic concept of [Block Explorers](/developers/docs/data-and-analytics/block-explorers/) in order to better understand using them in the data analytics context. In addition, familiarize yourself with the concept of an [index](/glossary/#index) to understand the benefits they add to a system design. | ||
| ## Crypto Data Landscape |
There was a problem hiding this comment.
(Custom header ids needed for all headers—related to our translations)
| ## Crypto Data Landscape | |
| ## Crypto Data Landscape {#crypto-data-landscape} |
| - DeFiLlama adapters saw [230 pull requests from 150 contributors across 559 files](https://github.com/DefiLlama/DefiLlama-Adapters/pulse/monthly) out of about 3700 total files. | ||
| - Dune spellbook saw [127 pull requests from 42 contributors across 779 files](https://github.com/duneanalytics/spellbook/pulse/monthly) out of about 3700 total files. | ||
|
|
||
| You can probably consider every 2-3 files to be a new protocol/table. So 200+ tables shifting around every month, and that number is only increasing with new chains and startups. If you’ve never worked in GitHub or data tables before, let me tell you that’s a ton of work to manage. |
There was a problem hiding this comment.
Similar to above... "let me tell you that’s a ton of work to manage." would reword
Some general ethereum.org style guide tips: https://ethereum.org/en/contributing/style-guide/#best-practices
There was a problem hiding this comment.
Seeing a word count around 1800
https://ethereum.org/en/contributing/style-guide/#best-practices
Generally try to keep things closer to 1000, 1500 max... perhaps we could split the content of this page into two separate pages?
|
|
||
| You can probably consider every 2-3 files to be a new protocol/table. So 200+ tables shifting around every month, and that number is only increasing with new chains and startups. If you’ve never worked in GitHub or data tables before, let me tell you that’s a ton of work to manage. | ||
|
|
||
| ### Back Onchain: Putting the data back into contracts using Zero Knowledge (ZK) tech |
There was a problem hiding this comment.
| ### Back Onchain: Putting the data back into contracts using Zero Knowledge (ZK) tech | |
| ### Back onchain: Putting the data back into contracts using zero knowledge (ZK) tech {#back-onchain} |
| ## Block explorers {#block-explorers} | ||
|
|
||
| Many [Block Explorers](/developers/docs/data-and-analytics/block-explorers/) offer [RESTful](https://www.wikipedia.org/wiki/Representational_state_transfer) [API](https://www.wikipedia.org/wiki/API) gateways that will provide developers visibility into real-time data on blocks, transactions, miners, accounts, and other on-chain activity. | ||
|
|
||
| Developers can then process and transform this data to give their users unique insights and interactions with the [blockchain](/glossary/#blockchain). For example, [Etherscan](https://etherscan.io) provides execution and consensus data for every 12s slot. |
There was a problem hiding this comment.
As mentioned above; this would be the third place we talk about block explorers.... Glad we didn't place a list here, but I would also not link to Etherscan. We should direct to our block-explorers page
Would probably ditch the "blockchain" glossary link here as well... feels out-of-place for this deep in the docs.
| ## Further Reading {#further-reading} | ||
|
|
||
| - []() | ||
| - []() |
wackerow
changed the title
wackerow
changed the title
|
We'll also need to clear merge conflicts, but wouldn't worry about too much about that just yet; could make the existing comments harder to follow. |
github-actions
bot
removed
the
Status: Stale
|
This issue is stale because it has been open 30 days with no activity. |
github-actions
bot
added
the
Status: Stale
2 participants
Related to issue #9144