[CAP-69] String/Bytes conversion host functions

[dmkozh]

This is a discussion thread for CAP-69 - String/Bytes conversion host functions

CAP-69

It has been noticed by the contract developers that it's currently impossible to get a character from a String host object without allocating a vector on a guest side and copying the string into it (for that worth host only provides a way to get a string length and copy it to/from the linear memory). That increases the cost of the contract unnecessarily, as it otherwise didn't need to use alloc at all.

The issue is really an oversight in the initial Soroban host API design StringObject and BytesObject are essentially the same data type (modulo semantic annotation), but only BytesObject has a rich set of host functions available.

The solution proposed here introduces only the conversion host functions in order to provide a way to perform all the operations on the host side without bloating the host interface with duplicate functions.

[MonsieurNicolas]

Can you expand on what is broken if this CAP does not get implemented/quantify impact on the network (as it seems to be an optimization CAP)?

[dmkozh]

I think it's hard to quantify the 'network impact' of not implementing the host functions. This closes a gap that we had in the initial host API (strings were added towards the end of the development) and simplifies the conversions and host API access significantly. I don't think it should be necessary to convert the whole string to the linear memory (which may also linking alloc), just to get a character from it. But it's hard to predict exactly how many contracts will actually use this (which is generally true for most of the host functions).

Our general intention with the host functions is to allow contracts to manipulate larger/dynamically allocated objects entirely on the host side. These conversions just follow the same philosophy.

We could duplicate the bytes operations for strings as well, but I think that's a bit too cumbersome. The proposed solution is also similar to the Rust std library that allows performing most of the operations only on bytes of the string.