Magic-Router development

[bmuddha]

General Overview

The development of a utility service is required to streamline the differentiation between the Base Layer Blockchain (BLB) and Ephemeral Rollups (ER) during request processing. This service will enable client requests to be automatically routed to the appropriate destination based on predefined rules and contextual information extracted from the request itself.

The routing functionality encompasses two distinct scenarios:

1. Selecting between BLB and ER.
2. Selecting among multiple ERs.

At this stage, it appears that the second type of routing—selecting among multiple ERs—is best managed on the client side, as it requires significant context to ensure correct proxying. Consequently, the initial development phase will focus on the first scenario: determining the final destination of a request between the BLB and ER.

---

Routing Logic

The routing logic primarily hinges on the delegation status of accounts. Specifically:

• If the account involved in the request is delegated, the request should be routed to the ER.
• If the account is not delegated, the request should be sent to the BLB.

To achieve optimal performance, the router must have access to real-time delegation information and maintain an up-to-date record of all delegated accounts. One potential implementation approach involves maintaining a persistent WebSocket (WS) connection pool to the base layer, subscribing to updates on all accounts involved in delegation-related transactions. This setup would enable near real-time updates of account status changes.

---

Supported Request Types

sendTransaction

Routing will depend on the accounts referenced in the transaction's instructions.

simulateTransaction

Uses the same logic as sendTransaction.

getAccountInfo

Routing is based on the public key of the requested account. If the account is marked as delegated, the request is routed to the ER; otherwise, it is sent to the BLB.

getBalance

Follows the same logic as getAccountInfo.

getMultipleAccounts

Requests can be split and routed to both ER and BLB, depending on the delegation status of each account.

getTokenAccountsBalance

Uses the same logic as getAccountInfo or getBalance.

---

Unsupported Request Types

getProgramAccounts, getTokenAccountsByOwner, getTokenAccountsByDelegate

These requests are challenging to support due to the need to fetch data from both BLB and ER, which could result in inconsistent state responses.

getLargestAccount, getLargestTokenAccounts

Encounter similar issues as getProgramAccounts.

---

Ambiguous Support

getTransaction, getSignatureStatuses

Routing these requests based on transaction signatures is unclear, as the target could be on the BLB, an ER, or non-existent.

getSignaturesForAddress

Regardless of an account's delegation status, transactions could exist on both BLB and ER. Aggregating and organizing these transactions may be impractical and could lead to significant complexity.

---

Requests Exclusively Supported by BLB

Requests such as getSupply, getSlot, and getLeaderSchedule are inherently tied to the base layer and do not require delegation-aware routing.

---

Potential Challenges

Race Conditions

sendTransaction Followed by getAccountInfo

When a sendTransaction containing a delegation instruction is immediately followed by a getAccountInfo for the same account, the router might mistakenly query the BLB before receiving an update about the account's delegation status. This is an edge case but can be addressed by rechecking the account's status after receiving the response and re-querying the ER if necessary.

Consecutive sendTransaction Requests

A related issue arises when a delegation transaction is immediately followed by another transaction targeting the newly delegated account. If the delegation fails, the routing logic may require complex fallback rules depending on the nature of the failure.

---

WebSocket Support

It remains unclear whether the router should support WebSocket subscriptions (or a subset of the Solana specification). Managing dynamic subscriptions and recreating them on different endpoints based on delegation status could introduce significant complexity and potential errors.

[bmuddha]

Account Delegation Status

The primary technique utilized by the routing logic will be based on the
account's delegation status. For this, the router needs to maintain a lookup
table for each account it encounters, categorizing it as "delegated" or "to be
delegated". These lookup tables should be synchronized with the base layer with
minimal latency.

High-Level Routing Algorithm

1. Accept incoming JSON-RPC request
2. Check if the request is supported
3. Extract request metadata:
   1. Referenced public keys
4. If the request is sendTransaction, extract all account keys to be
   delegated (if any)
   4. If the request is sendTransaction and contains delegation
   instructions, subscribe and confirm WebSocket (WS) subscription for each
   account to be delegated
   5. Check the status of each account referenced in the request
5. Based on the delegation status, route the request to the appropriate
   endpoint (chain or ER)
   7. If applicable, verify that the owner of the account in the response
   corresponds to the expected value, i.e., it should not be equal to the
   Delegation program, regardless of the response source
   8. If the verification in step 7 fails, reroute the request to the correct
   endpoint. If the account is not found on the ER, route it to the base
   layer, and if the owner of the account is the Delegation program, resend
   the request to the ER

Low-Level Routing

For redundancy, the router should maintain several connection pools (both
for HTTP and WebSocket), and load balance between them:

• For HTTP requests, the reqwest library (based on hyper) will be used,
  which manages its own connection pool and load balances between them
• It should also be possible to provide several different endpoints for
  establishing connections, so that multiple connection pools will be
  maintained for each, and the router will load balance between them using
  a weighted uniform distribution
• In case of a request failure due to an I/O error, a configurable retry
  policy can be employed
• For WebSocket connections, similar logic will apply, except the pool will be
  maintained at the router level (not by a library)
  • If a WebSocket connection fails, it will mark all the accounts for which
    it maintains subscriptions as in a "limbo" state, indicating that their
    status may be stale. After that, it will try to reestablish the
    connection and recreate all subscriptions, followed by refetching all
    those accounts to get their current state
  • Account status records marked as potentially stale will trigger the
    router to fetch them from the mainnet and follow steps 7-8 in the
    higher-level routing for final state resolution

Thoughts on Latency

Regular cases when states are fetched from expected sources should have
relatively low latency. However, edge cases that can arise from race conditions
or indeterminate account statuses due to WebSocket connection failure can
introduce additional latency due to multiple requests being performed. This is
a compromise that sacrifices latency for consistency.

@GabrielePicco @thlorenz that's my current approach to implemenation of routing logic, ideas and constructive criticism are welcome