Improve initial data inventory requests

[HenrikJannsen]

For a fresh node it takes several minutes to receive all inventory data.

We should analyze where are the bottlenecks, if timeouts are an issue (I saw 2 seed node connections timed out) and add data size to the logs of the data types. Maybe Consider to reduce size of response to make responses more reliable.

Here is an example data set:

Accumulated_Data.csv

[HenrikJannsen]

Here is the accumulated count of all items of the same type:

AddAuthenticatedDataRequest

• AuthorizedBondedRole: 21
• AuthorizedProofOfBurnData: 1752
• UserProfile: 358 + 2093 + 4 = 2455
• ReleaseNotification: 1 + 1 = 2
• AuthorizedAlertData: 6 + 1 = 7
• AuthorizedSignedWitnessData: 590
• AuthorizedBondedReputationData: 150
• AuthorizedAccountAgeData: 976
• AuthorizedOracleNode: 2
• BannedUserProfileData: 32 + 1 = 33
• AuthorizedTimestampData: 1794 + 1734 + 1 = 3529
• AuthorizedMarketPriceData: 10 + 1 = 11
• CommonPublicChatMessage: 242
• CommonPublicChatMessageReaction: 27
• BisqEasyOfferbookMessage: 123
• BisqEasyOfferbookMessageReaction: 1 + 1 = 2

---

RemoveAuthenticatedDataRequest

• AuthorizedBondedRole: 3
• BannedUserProfileData: 2 + 1 = 3
• AuthorizedAlertData: 10
• AuthorizedOracleNode: 3
• UserProfile: 34
• BisqEasyOfferbookMessage: 369 + 1 = 370
• CommonPublicChatMessage: 27 + 1 = 28
• ReleaseNotification: 5
• CommonPublicChatMessageReaction: 1 + 1 = 2

---

AddMailboxRequest

• BisqEasyOpenTradeMessage: 388 + 386 = 774
• MediatorsResponse: 2
• MediationRequest: 7
• BisqEasyTakeOfferResponse: 36
• BisqEasyRejectTradeMessage: 36
• BisqEasyConfirmFiatReceiptMessage: 14
• BisqEasyReportErrorMessage: 4
• AckMessage: 64
• ReportToModeratorMessage: 4
• TwoPartyPrivateChatMessage: 113
• BisqEasyTakeOfferRequest: 18
• BisqEasyOpenTradeMessageReaction: 4
• BisqEasyCancelTradeMessage: 82
• BisqEasyBtcAddressMessage: 12
• AuthorizeAccountAgeRequest: 2
• BisqEasyAccountDataMessage: 49
• BisqEasyConfirmBtcSentMessage: 16

---

RemoveMailboxRequest

• BisqEasyOpenTradeMessage: 2646 + 521 = 3167
• MediationRequest: 10
• BisqEasyConfirmFiatSentMessage: 78
• MediatorsResponse: 6
• TwoPartyPrivateChatMessage: 258
• BisqEasyCancelTradeMessage: 67
• BisqEasyBtcAddressMessage: 172
• AckMessage: 147
• AuthorizeSignedWitnessRequest: 16
• BisqEasyRejectTradeMessage: 100
• AuthorizeTimestampRequest: 32
• TwoPartyPrivateChatMessageReaction: 4
• BisqEasyAccountDataMessage: 179
• AuthorizeAccountAgeRequest: 32
• BisqEasyReportErrorMessage: 30
• BisqEasyConfirmBtcSentMessage: 82
• ReportToModeratorMessage: 10
• BisqEasyOpenTradeMessageReaction: 50
• BisqEasyConfirmFiatReceiptMessage: 100
• BisqEasyTakeOfferRequest: 243
• BisqEasyTakeOfferResponse: 102

---

Let me know if you need further clarification!

[HenrikJannsen]

Data from logs:

13:30:51.984
Received 1999.6 KB of inventory data from: ljtn3q7fkb7nqo6umkz36qip5fb46y43hajlkjl5zncctzii5wcsthyd.onion:1000 after 11.53 sec.;
21 item(s) of AddAuthenticatedDataRequest.AuthorizedBondedRole
1752 item(s) of AddAuthenticatedDataRequest.AuthorizedProofOfBurnData
358 item(s) of AddAuthenticatedDataRequest.UserProfile
1 item(s) of AddAuthenticatedDataRequest.ReleaseNotification
6 item(s) of AddAuthenticatedDataRequest.AuthorizedAlertData
590 item(s) of AddAuthenticatedDataRequest.AuthorizedSignedWitnessData
150 item(s) of AddAuthenticatedDataRequest.AuthorizedBondedReputationData
976 item(s) of AddAuthenticatedDataRequest.AuthorizedAccountAgeData
2 item(s) of AddAuthenticatedDataRequest.AuthorizedOracleNode
32 item(s) of AddAuthenticatedDataRequest.BannedUserProfileData

Received 1999.786 KB of inventory data from: ljtn3q7fkb7nqo6umkz36qip5fb46y43hajlkjl5zncctzii5wcsthyd.onion:1000 after 38.04 sec.;
1794 item(s) of AddAuthenticatedDataRequest.AuthorizedTimestampData
2093 item(s) of AddAuthenticatedDataRequest.UserProfile
1 item(s) of AddAuthenticatedDataRequest.ReleaseNotification
1 item(s) of AddAuthenticatedDataRequest.AuthorizedAlertData
1 item(s) of AddAuthenticatedDataRequest.BannedUserProfileData

Received 1998.985 KB of inventory data from: ljtn3q7fkb7nqo6umkz36qip5fb46y43hajlkjl5zncctzii5wcsthyd.onion:1000 after 170.8 sec.;
3 item(s) of RemoveAuthenticatedDataRequest.AuthorizedBondedRole
2 item(s) of RemoveAuthenticatedDataRequest.BannedUserProfileData
10 item(s) of RemoveAuthenticatedDataRequest.AuthorizedAlertData
3 item(s) of RemoveAuthenticatedDataRequest.AuthorizedOracleNode
34 item(s) of RemoveAuthenticatedDataRequest.UserProfile
369 item(s) of RemoveAuthenticatedDataRequest.BisqEasyOfferbookMessage
27 item(s) of RemoveAuthenticatedDataRequest.CommonPublicChatMessage
5 item(s) of RemoveAuthenticatedDataRequest.ReleaseNotification
1 item(s) of RemoveAuthenticatedDataRequest.CommonPublicChatMessageReaction
27 item(s) of AddAuthenticatedDataRequest.CommonPublicChatMessageReaction
242 item(s) of AddAuthenticatedDataRequest.CommonPublicChatMessage
1 item(s) of AddAuthenticatedDataRequest.BisqEasyOfferbookMessageReaction
1734 item(s) of AddAuthenticatedDataRequest.AuthorizedTimestampData
10 item(s) of AddAuthenticatedDataRequest.AuthorizedMarketPriceData
123 item(s) of AddAuthenticatedDataRequest.BisqEasyOfferbookMessage
388 item(s) of AddMailboxRequest.BisqEasyOpenTradeMessage
2 item(s) of AddMailboxRequest.MediatorsResponse
7 item(s) of AddMailboxRequest.MediationRequest

Received 1999.765 KB of inventory data from: ljtn3q7fkb7nqo6umkz36qip5fb46y43hajlkjl5zncctzii5wcsthyd.onion:1000 after 104.42 sec.;
1 item(s) of RemoveAuthenticatedDataRequest.BisqEasyOfferbookMessage
1 item(s) of RemoveAuthenticatedDataRequest.CommonPublicChatMessage
1 item(s) of RemoveAuthenticatedDataRequest.CommonPublicChatMessageReaction
1 item(s) of AddAuthenticatedDataRequest.BisqEasyOfferbookMessageReaction
1 item(s) of AddAuthenticatedDataRequest.AuthorizedTimestampData
2646 item(s) of RemoveMailboxRequest.BisqEasyOpenTradeMessage
10 item(s) of RemoveMailboxRequest.MediationRequest
78 item(s) of RemoveMailboxRequest.BisqEasyConfirmFiatSentMessage
6 item(s) of RemoveMailboxRequest.MediatorsResponse
36 item(s) of AddMailboxRequest.BisqEasyTakeOfferResponse
36 item(s) of AddMailboxRequest.BisqEasyRejectTradeMessage
14 item(s) of AddMailboxRequest.BisqEasyConfirmFiatReceiptMessage
4 item(s) of AddMailboxRequest.BisqEasyReportErrorMessage
64 item(s) of AddMailboxRequest.AckMessage
4 item(s) of AddMailboxRequest.ReportToModeratorMessage
113 item(s) of AddMailboxRequest.TwoPartyPrivateChatMessage
18 item(s) of AddMailboxRequest.BisqEasyTakeOfferRequest
386 item(s) of AddMailboxRequest.BisqEasyOpenTradeMessage
4 item(s) of AddMailboxRequest.BisqEasyOpenTradeMessageReaction
82 item(s) of AddMailboxRequest.BisqEasyCancelTradeMessage
12 item(s) of AddMailboxRequest.BisqEasyBtcAddressMessage
2 item(s) of AddMailboxRequest.AuthorizeAccountAgeRequest
49 item(s) of AddMailboxRequest.BisqEasyAccountDataMessage
16 item(s) of AddMailboxRequest.BisqEasyConfirmBtcSentMessage

13:37:23.176
Received 518.982 KB of inventory data from: ljtn3q7fkb7nqo6umkz36qip5fb46y43hajlkjl5zncctzii5wcsthyd.onion:1000 after 75.24 sec.;
4 item(s) of AddAuthenticatedDataRequest.UserProfile
1 item(s) of AddAuthenticatedDataRequest.AuthorizedMarketPriceData
258 item(s) of RemoveMailboxRequest.TwoPartyPrivateChatMessage
67 item(s) of RemoveMailboxRequest.BisqEasyCancelTradeMessage
521 item(s) of RemoveMailboxRequest.BisqEasyOpenTradeMessage
172 item(s) of RemoveMailboxRequest.BisqEasyBtcAddressMessage
147 item(s) of RemoveMailboxRequest.AckMessage
16 item(s) of RemoveMailboxRequest.AuthorizeSignedWitnessRequest
100 item(s) of RemoveMailboxRequest.BisqEasyRejectTradeMessage
32 item(s) of RemoveMailboxRequest.AuthorizeTimestampRequest
4 item(s) of RemoveMailboxRequest.TwoPartyPrivateChatMessageReaction
179 item(s) of RemoveMailboxRequest.BisqEasyAccountDataMessage
32 item(s) of RemoveMailboxRequest.AuthorizeAccountAgeRequest
30 item(s) of RemoveMailboxRequest.BisqEasyReportErrorMessage
82 item(s) of RemoveMailboxRequest.BisqEasyConfirmBtcSentMessage
10 item(s) of RemoveMailboxRequest.ReportToModeratorMessage
50 item(s) of RemoveMailboxRequest.BisqEasyOpenTradeMessageReaction
100 item(s) of RemoveMailboxRequest.BisqEasyConfirmFiatReceiptMessage
243 item(s) of RemoveMailboxRequest.BisqEasyTakeOfferRequest
102 item(s) of RemoveMailboxRequest.BisqEasyTakeOfferResponse

ca 6:30 min!