Skip to content

Commit 0a378e6

Browse files
ffakenzch1bonoonio
authored andcommitted
Add blockfrost mode to hydra-chain-observer (#1631)
<!-- Describe your change here --> 🥶 Added **Blockfrost Mode** to `hydra-chain-observer`. 🥶 The *network id* and *block time* are derived from the configured `BLOCKFROST_TOKEN_PATH`. 🥶 Implemented a naive roll-forward approach: - 🧊 We start following the chain from a given block hash or the tip (latest block). - 🧊 We check if the current block is within the safe zone to be processed, using the "number of block confirmations" > Based on some [reference](https://cardano.stackexchange.com/questions/8760/what-is-your-comfort-level-for-number-of-confirmations-and-why) from a not-so-stranger on the internet. - 🧊 From the transaction hashes of the block, we fetch the transactions in CBOR representations. - 🧊 We then deserialise them into Cardano API transactions, allowing us to collect head observations by reusing existing code. - 🧊 Finally, using the next block hash information from the block, we repeat the process. 🥶 Note: If any "retriable error" occurs during roll-forward, we wait based on the known *block time* before restarting using latest known fetched block and UTxO view (collected observations). --- <!-- Consider each and tick it off one way or the other --> * [x] CHANGELOG updated or not needed * [x] Documentation updated or not needed * [x] Haddocks updated or not needed * [x] No new TODOs introduced or explained herafter --------- Co-authored-by: Sebastian Nagel <[email protected]> Co-authored-by: Noon <[email protected]>
1 parent aaaa6e8 commit 0a378e6

File tree

21 files changed

+742
-280
lines changed

21 files changed

+742
-280
lines changed

.github/workflows/explorer/docker-compose.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,8 @@ services:
2323
ports:
2424
- "80:8080"
2525
command:
26-
[ "--node-socket", "/data/node.socket"
26+
[ "direct"
27+
, "--node-socket", "/data/node.socket"
2728
, "--testnet-magic", "2"
2829
, "--api-port", "8080"
2930
# NOTE: Block in which current master scripts were published

CHANGELOG.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,8 @@ changes.
4040
- Overall this results in transactions still to be submitted once per client,
4141
but requires signifanctly less book-keeping on the client-side.
4242

43+
- Add **Blockfrost Mode** to `hydra-chain-observer`, to follow the chain via Blockfrost API.
44+
4345
## [0.19.0] - 2024-09-13
4446

4547
- Tested with `cardano-node 9.1.1` and `cardano-cli 9.2.1.0`
@@ -54,7 +56,6 @@ changes.
5456

5557
- Add a demo mode to hydra-cluster to facilitate network resiliance tests [#1552](https://github.com/cardano-scaling/hydra/pull/1552)
5658

57-
5859
## [0.18.1] - 2024-08-15
5960

6061
- New landing page and updated documentation style. [#1560](https://github.com/cardano-scaling/hydra/pull/1560)

cabal.project

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,8 @@ repository cardano-haskell-packages
1212

1313
-- See CONTRIBUTING.md for information about when and how to update these.
1414
index-state:
15-
, hackage.haskell.org 2024-09-23T15:45:50Z
16-
, cardano-haskell-packages 2024-09-20T19:39:13Z
15+
, hackage.haskell.org 2024-09-25T13:28:12Z
16+
, cardano-haskell-packages 2024-09-23T21:46:49Z
1717

1818
packages:
1919
hydra-prelude

flake.lock

Lines changed: 3 additions & 3 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

hydra-chain-observer/README.md

Lines changed: 28 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,37 @@
11
# Hydra Chain Observer
22

3-
A small executable which connects to a chain like the `hydra-node`, but puts any
4-
observations as traces onto `stdout`.
3+
A lightweight executable designed to connect to a blockchain, such as the `hydra-node`, and streams chain observations as traces to `stdout`.
4+
It supports two modes of operation: **Direct** connection to a node via socket, and connection through **Blockfrost** API.
55

6-
To run, pass a `--node-socket`, corresponding network id and optionally
7-
`--start-chain-from`. For example:
6+
## Direct Mode
7+
8+
To run the observer in Direct Mode, provide the following arguments:
9+
- `--node-socket`: path to the node socket file.
10+
- network id: `--testnet-magic` (with magic number) for the testnet or `--mainnet` for the mainnet.
11+
- (optional) `--start-chain-from`: specify a chain point (SLOT.HEADER_HASH) to start observing from.
12+
13+
For example:
814

915
``` shell
10-
hydra-chain-observer \
16+
hydra-chain-observer direct \
1117
--node-socket testnets/preprod/node.socket \
1218
--testnet-magic 1 \
1319
--start-chain-from "41948777.5d34af0f42be9823ebd35c2d83d5d879c5615ac17f7158bb9aa4ef89072455a7"
1420
```
21+
22+
23+
## Blockfrost Mode
24+
25+
To run the observer in Blockfrost Mode, provide the following arguments:
26+
- `--project-path`: file path to your Blockfrost project API token hash.
27+
> expected to be prefixed with environment (e.g. testnetA3C2E...)
28+
- (optional) `--start-chain-from`: specify a chain point (SLOT.HEADER_HASH) to start observing from.
29+
30+
For example:
31+
32+
``` shell
33+
hydra-chain-observer blockfrost \
34+
--project-path $PROJECT_TOKEN_HASH_PATH \
35+
--start-chain-from "41948777.5d34af0f42be9823ebd35c2d83d5d879c5615ac17f7158bb9aa4ef89072455a7"
36+
```
37+

hydra-chain-observer/exe/Main.hs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
module Main where
22

3-
import Hydra.ChainObserver (defaultObserverHandler)
43
import Hydra.ChainObserver qualified
4+
import Hydra.ChainObserver.NodeClient (defaultObserverHandler)
55
import Hydra.Prelude
66

77
main :: IO ()

hydra-chain-observer/hydra-chain-observer.cabal

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,17 +65,24 @@ library
6565
hs-source-dirs: src
6666
ghc-options: -haddock
6767
build-depends:
68+
, base16-bytestring
69+
, blockfrost-client >=0.9.1.0
6870
, hydra-cardano-api
6971
, hydra-node
7072
, hydra-plutus
7173
, hydra-prelude
7274
, hydra-tx
75+
, io-classes
7376
, optparse-applicative
7477
, ouroboros-network-protocols
78+
, retry
7579

7680
exposed-modules:
81+
Hydra.Blockfrost.ChainObserver
7782
Hydra.ChainObserver
83+
Hydra.ChainObserver.NodeClient
7884
Hydra.ChainObserver.Options
85+
Hydra.Ouroborus.ChainObserver
7986

8087
executable hydra-chain-observer
8188
import: project-config
Lines changed: 227 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,227 @@
1+
{-# LANGUAGE DuplicateRecordFields #-}
2+
3+
module Hydra.Blockfrost.ChainObserver where
4+
5+
import Hydra.Prelude
6+
7+
import Blockfrost.Client (
8+
BlockfrostClientT,
9+
runBlockfrost,
10+
)
11+
import Blockfrost.Client qualified as Blockfrost
12+
import Control.Concurrent.Class.MonadSTM (
13+
MonadSTM (readTVarIO),
14+
newTVarIO,
15+
writeTVar,
16+
)
17+
import Control.Retry (constantDelay, retrying)
18+
import Data.ByteString.Base16 qualified as Base16
19+
import Hydra.Cardano.Api (
20+
ChainPoint (..),
21+
HasTypeProxy (..),
22+
Hash,
23+
NetworkId (..),
24+
NetworkMagic (..),
25+
SerialiseAsCBOR (..),
26+
SlotNo (..),
27+
Tx,
28+
UTxO,
29+
serialiseToRawBytes,
30+
)
31+
import Hydra.Cardano.Api.Prelude (
32+
BlockHeader (..),
33+
)
34+
import Hydra.Chain.Direct.Handlers (convertObservation)
35+
import Hydra.ChainObserver.NodeClient (
36+
ChainObservation (..),
37+
ChainObserverLog (..),
38+
NodeClient (..),
39+
ObserverHandler,
40+
logOnChainTx,
41+
observeAll,
42+
)
43+
import Hydra.Logging (Tracer, traceWith)
44+
import Hydra.Tx (IsTx (..))
45+
46+
data APIBlockfrostError
47+
= BlockfrostError Text
48+
| DecodeError Text
49+
| NotEnoughBlockConfirmations Blockfrost.BlockHash
50+
| MissingBlockNo Blockfrost.BlockHash
51+
| MissingNextBlockHash Blockfrost.BlockHash
52+
deriving (Show, Exception)
53+
54+
runBlockfrostM ::
55+
(MonadIO m, MonadThrow m) =>
56+
Blockfrost.Project ->
57+
BlockfrostClientT IO a ->
58+
m a
59+
runBlockfrostM prj action = do
60+
result <- liftIO $ runBlockfrost prj action
61+
case result of
62+
Left err -> throwIO (BlockfrostError $ show err)
63+
Right val -> pure val
64+
65+
blockfrostClient ::
66+
Tracer IO ChainObserverLog ->
67+
FilePath ->
68+
Integer ->
69+
NodeClient IO
70+
blockfrostClient tracer projectPath blockConfirmations = do
71+
NodeClient
72+
{ follow = \startChainFrom observerHandler -> do
73+
prj <- Blockfrost.projectFromFile projectPath
74+
75+
Blockfrost.Block{_blockHash = (Blockfrost.BlockHash genesisBlockHash)} <-
76+
runBlockfrostM prj (Blockfrost.getBlock (Left 0))
77+
78+
Blockfrost.Genesis
79+
{ _genesisActiveSlotsCoefficient
80+
, _genesisSlotLength
81+
, _genesisNetworkMagic
82+
} <-
83+
runBlockfrostM prj Blockfrost.getLedgerGenesis
84+
85+
let networkId = fromNetworkMagic _genesisNetworkMagic
86+
traceWith tracer ConnectingToExternalNode{networkId}
87+
88+
chainPoint <-
89+
case startChainFrom of
90+
Just point -> pure point
91+
Nothing -> do
92+
toChainPoint <$> runBlockfrostM prj Blockfrost.getLatestBlock
93+
94+
traceWith tracer StartObservingFrom{chainPoint}
95+
96+
let blockTime = realToFrac _genesisSlotLength / realToFrac _genesisActiveSlotsCoefficient
97+
98+
let blockHash = fromChainPoint chainPoint genesisBlockHash
99+
100+
stateTVar <- newTVarIO (blockHash, mempty)
101+
void $
102+
retrying (retryPolicy blockTime) shouldRetry $ \_ -> do
103+
loop tracer prj networkId blockTime observerHandler blockConfirmations stateTVar
104+
`catch` \(ex :: APIBlockfrostError) ->
105+
pure $ Left ex
106+
}
107+
where
108+
shouldRetry _ = \case
109+
Right{} -> pure False
110+
Left err -> pure $ isRetryable err
111+
112+
retryPolicy blockTime = constantDelay (truncate blockTime * 1000 * 1000)
113+
114+
-- | Iterative process that follows the chain using a naive roll-forward approach,
115+
-- keeping track of the latest known current block and UTxO view.
116+
-- This process operates at full speed without waiting between calls,
117+
-- favoring the catch-up process.
118+
loop ::
119+
(MonadIO m, MonadThrow m, MonadSTM m) =>
120+
Tracer m ChainObserverLog ->
121+
Blockfrost.Project ->
122+
NetworkId ->
123+
DiffTime ->
124+
ObserverHandler m ->
125+
Integer ->
126+
TVar m (Blockfrost.BlockHash, UTxO) ->
127+
m a
128+
loop tracer prj networkId blockTime observerHandler blockConfirmations stateTVar = do
129+
current <- readTVarIO stateTVar
130+
next <- rollForward tracer prj networkId observerHandler blockConfirmations current
131+
atomically $ writeTVar stateTVar next
132+
loop tracer prj networkId blockTime observerHandler blockConfirmations stateTVar
133+
134+
-- | From the current block and UTxO view, we collect Hydra observations
135+
-- and yield the next block and adjusted UTxO view.
136+
rollForward ::
137+
(MonadIO m, MonadThrow m) =>
138+
Tracer m ChainObserverLog ->
139+
Blockfrost.Project ->
140+
NetworkId ->
141+
ObserverHandler m ->
142+
Integer ->
143+
(Blockfrost.BlockHash, UTxO) ->
144+
m (Blockfrost.BlockHash, UTxO)
145+
rollForward tracer prj networkId observerHandler blockConfirmations (blockHash, utxo) = do
146+
block@Blockfrost.Block
147+
{ _blockHash
148+
, _blockConfirmations
149+
, _blockNextBlock
150+
, _blockHeight
151+
} <-
152+
runBlockfrostM prj $ Blockfrost.getBlock (Right blockHash)
153+
154+
-- Check if block within the safe zone to be processes
155+
when (_blockConfirmations < blockConfirmations) $
156+
throwIO (NotEnoughBlockConfirmations _blockHash)
157+
158+
-- Check if block contains a reference to its next
159+
nextBlockHash <- maybe (throwIO $ MissingNextBlockHash _blockHash) pure _blockNextBlock
160+
161+
-- Search block transactions
162+
txHashes <- runBlockfrostM prj . Blockfrost.allPages $ \p ->
163+
Blockfrost.getBlockTxs' (Right _blockHash) p Blockfrost.def
164+
165+
-- Collect CBOR representations
166+
cborTxs <- traverse (runBlockfrostM prj . Blockfrost.getTxCBOR) txHashes
167+
168+
-- Convert to cardano-api Tx
169+
receivedTxs <- mapM toTx cborTxs
170+
let receivedTxIds = txId <$> receivedTxs
171+
let point = toChainPoint block
172+
traceWith tracer RollForward{point, receivedTxIds}
173+
174+
-- Collect head observations
175+
let (adjustedUTxO, observations) = observeAll networkId utxo receivedTxs
176+
let onChainTxs = mapMaybe convertObservation observations
177+
forM_ onChainTxs (traceWith tracer . logOnChainTx)
178+
179+
blockNo <- maybe (throwIO $ MissingBlockNo _blockHash) (pure . fromInteger) _blockHeight
180+
let observationsAt = HeadObservation point blockNo <$> onChainTxs
181+
182+
-- Call observer handler
183+
observerHandler $
184+
if null observationsAt
185+
then [Tick point blockNo]
186+
else observationsAt
187+
188+
-- Next
189+
pure (nextBlockHash, adjustedUTxO)
190+
191+
-- * Helpers
192+
193+
isRetryable :: APIBlockfrostError -> Bool
194+
isRetryable (BlockfrostError _) = True
195+
isRetryable (DecodeError _) = False
196+
isRetryable (NotEnoughBlockConfirmations _) = True
197+
isRetryable (MissingBlockNo _) = True
198+
isRetryable (MissingNextBlockHash _) = True
199+
200+
toChainPoint :: Blockfrost.Block -> ChainPoint
201+
toChainPoint Blockfrost.Block{_blockSlot, _blockHash} =
202+
ChainPoint slotNo headerHash
203+
where
204+
slotNo :: SlotNo
205+
slotNo = maybe 0 (fromInteger . Blockfrost.unSlot) _blockSlot
206+
207+
headerHash :: Hash BlockHeader
208+
headerHash = fromString . toString $ Blockfrost.unBlockHash _blockHash
209+
210+
fromNetworkMagic :: Integer -> NetworkId
211+
fromNetworkMagic = \case
212+
0 -> Mainnet
213+
magicNbr -> Testnet (NetworkMagic (fromInteger magicNbr))
214+
215+
toTx :: MonadThrow m => Blockfrost.TransactionCBOR -> m Tx
216+
toTx (Blockfrost.TransactionCBOR txCbor) =
217+
case decodeBase16 txCbor of
218+
Left decodeErr -> throwIO . DecodeError $ "Bad Base16 Tx CBOR: " <> decodeErr
219+
Right bytes ->
220+
case deserialiseFromCBOR (proxyToAsType (Proxy @Tx)) bytes of
221+
Left deserializeErr -> throwIO . DecodeError $ "Bad Tx CBOR: " <> show deserializeErr
222+
Right tx -> pure tx
223+
224+
fromChainPoint :: ChainPoint -> Text -> Blockfrost.BlockHash
225+
fromChainPoint chainPoint genesisBlockHash = case chainPoint of
226+
ChainPoint _ headerHash -> Blockfrost.BlockHash (decodeUtf8 . Base16.encode . serialiseToRawBytes $ headerHash)
227+
ChainPointAtGenesis -> Blockfrost.BlockHash genesisBlockHash

0 commit comments

Comments
 (0)