RoCEv2 OTG model [latest]

[satyamsinghKey]

Issue: #403

Redocly View:
https://redocly.github.io/redoc/?url=https://raw.githubusercontent.com/open-traffic-generator/models/dev2_rocev2/artifacts/openapi.yaml&nocors#tag/Configuration/operation/set_config

Objective: Add support for RoCEv2 protocol

Sample Code:
This sample snippet will be constantly updated.
`

 #CLEAR CONFIG
    config = api.config()
#api.set_config(config)
print (str(config.ports))
p1 = config.ports.add(name="tx", location=utils.settings.ports[0])
p2 = config.ports.add(name="rx", location=utils.settings.ports[1])

d1, d2 = config.devices.device(name="rocev2_1").device(name="rocev2_2")
eth1, eth2 = d1.ethernets.add(), d2.ethernets.add()
eth1.connection.port_name, eth2.connection.port_name = p1.name, p2.name
eth1.mac, eth2.mac = "00:00:00:00:00:11", "00:00:00:00:00:22"
ip1, ip2 = eth1.ipv4_addresses.add(), eth2.ipv4_addresses.add()
eth1.name, eth2.name = "eth1", "eth2"
ip1.name, ip2.name = "ip1", "ip2"

ip1.address = "10.1.1.1"
ip2.address = "10.1.1.2"

ip1.prefix = 24
ip2.prefix = 24

ip1.gateway = ip2.address
ip2.gateway = ip1.address

rocev2_1, rocev2_2 = d1.rocev2, d2.rocev2
rocev2_1_int, rocev2_2_int = rocev2_1.ipv4_interfaces.add(), rocev2_2.ipv4_interfaces.add()
rocev2_1_int.ipv4_name, rocev2_2_int.ipv4_name = ip1.name, ip2.name
rocev2_1_int.ib_mtu, rocev2_2_int.ib_mtu = 1027, 1027
rocev2_1_peer, rocev2_2_peer = rocev2_1_int.peers.add(), rocev2_2_int.peers.add()
rocev2_1_peer.name, rocev2_2_peer.name = "RoCEv2 1", "RoCEv2 2"
rocev2_1_peer.destination_ip_address, rocev2_2_peer.destination_ip_address = ip2.address, ip1.address

peer1_qp_1 = rocev2_1_peer.qps.add()
peer2_qp_1 = rocev2_2_peer.qps.add()
peer1_qp_1.qp_name = "QP_1"
peer2_qp_1.qp_name = "QP_2"

peer1_qp_1.connection_type.choice = "reliable_connection"
peer1_qp_1.connection_type.reliable_connection.source_qp_number = 33
peer1_qp_1.connection_type.reliable_connection.dscp = 27

peer2_qp_1.connection_type.choice = "reliable_connection"
peer2_qp_1.connection_type.reliable_connection.source_qp_number = 34
peer2_qp_1.connection_type.reliable_connection.dscp = 28

peer1 = config.stateful_flows.rocev2.add()
tx_port1 = peer1.tx_ports.add()
tx_port1.port_name = "tx"
tx_port1.transmit_type.choice = "target_line_rate"

#flow_1
peer1_flow1 = tx_port1.transmit_type.target_line_rate.flows.add()
peer1_flow1.tx_endpoint = peer1_qp_1.qp_name
peer1_flow1.name = "QP_2"
peer1_flow1.rocev2_verb.choice = "send_with_immediate"
peer1_flow1.rocev2_verb.send_with_immediate.immediate_data = "bb"
peer1_flow1.message_size_unit = "KB"

#flow_2
peer2_flow1 = tx_port1.transmit_type.target_line_rate.flows.add()
peer2_flow1.tx_endpoint = peer2_qp_1.qp_name
peer2_flow1.name = "QP_1"
peer2_flow1.rocev2_verb.choice = "write_with_immediate"
peer2_flow1.rocev2_verb.write_with_immediate.immediate_data = "aa"
peer2_flow1.message_size_unit = "MB"

#RoCEv2 Protocol Port Settings
per_port_option1 = config.options.per_port_options.add()
per_port_option2 = config.options.per_port_options.add()
per_port_option1.port_name = "tx"
per_port_option2.port_name = "rx"
protocol1 = per_port_option1.protocols.add()
protocol2 = per_port_option2.protocols.add()
protocol1.choice = "rocev2"
protocol2.choice = "rocev2"
protocol1.rocev2.cnp.choice = "ip_dscp"
protocol1.rocev2.cnp.ip_dscp.value = 49
protocol1.rocev2.connection_type.choice = "reliable_connection"
protocol1.rocev2.connection_type.reliable_connection.enable_retransmission_timeout = False
protocol1.rocev2.connection_type.reliable_connection.retransmission_timeout_value = 10
protocol1.rocev2.dcqcn_settings.alpha_g = 1020
protocol1.rocev2.dcqcn_settings.initial_alpha = 1000
protocol1.rocev2.dcqcn_settings.maximum_rate_decrement_at_time = 12

#once all are configured, push the config to IxNetwork
api.set_config(config)

#Starting Protocols
control_state = api.control_state()
control_state.protocol.all.state = control_state.protocol.all.START
api.set_control_state(control_state)

#Fetching and Verifying Protocol stats...
utils.wait_for(
    lambda: results_ok(api), "stats to be as expected", timeout_seconds=20
)

#Start Traffic, in case of RoCEv2 it can only be done once the protocol is up
utils.start_traffic(api, b2b_raw_config)

#Fetch Traffic stats
//Yet to add
   node : rocev2_flow.per_port.qp_name
              rocev3_flow.per_port.column_names

#"Stopping Protocols
control_state = api.control_state()
control_state.protocol.all.state = control_state.protocol.all.STOP
api.set_control_state(control_state)

def results_ok(api):
   req = api.metrics_request()
   req.rocev2_ipv4.choice = "per_peer"
   req.rocev2_ipv4.per_peer.column_names = ["session_state"]
   results = api.get_metrics(req)
time.sleep(10)
ok = []
for r in results.rocev2_ipv4_per_peer_metrics:
    ok.append(r.session_state == "up")
return all(ok)

`

NOTE:

• A dummy field exists in options.per_port_options.port_settings.protocols.dummy_field. Please ignore this field for now, as it is present due to a compilation error we have identified and are currently investigating.

• Some sections/fields do not have descriptions, and you may come across the placeholder text "Description here". If you are familiar with these fields, please suggest appropriate descriptions.

• This PR is created new due to major modification required in the model.

• Link to view previous PR : (RoCEv2 OTG Model #405)

[kunkeysight]

As discussed, PFC Rx stats are important for RoCEv2 testing. Please add them in port stats.

[apratimmukherjee]

As discussed, PFC Rx stats are important for RoCEv2 testing. Please add them in port stats.

@kunkeysight : This should be added in a separate PR since it is an additional feature as far as I understand.
I think it might be better to add them in a separate pfc_port_stats with some of the relevant port stats + pfc stats to not pollute simple existing port_stats with 20-30 new stats that are not relevant to all free ixia-c users or full google FP eco-system.
Also should consult openconfig modeling for pfc config/stats before doing this.

I think might be better to first complete RoCEv2 control plane + traffic based on this PR with dev builds to allow merging of PR and initial delivery . And then separately implement PFC tx/rx stats in whichever order that is needed. Can happen in parallel as well .

[apratimmukherjee]

@satyamsinghKey -> Comments on the example and Introduction.
#User needs to provide message size, rocev2_verb in order to bring QPs up. Atleast these 3 are required to bring protocols up
#populate these parameters from
QPs should come up even if
a) flows are not created for them .
b) They are not explicitly set -> They look like optional params with default values.
So user should not have to be mandatorily providing them .
Also below the redocli link good to share the different points in the tree that need to be looked at in the redocli view :
e.g.

1. device/rocev2 for all QP configuration
2. stateful_flows ( /rocev2) for new way to configure stateful flows [ maybe brief explanation why we needed this e.g. packet size/structure etc. are not user controlled but based on qp config + stats are different and specific to roce ]
3. options/... per port options for per port rocev2 options
4. get_metric path
   [ Probably nothing in get_states today ]

[apratimmukherjee]

One more round of review since it is probably getting closer to merge and dont see any new reviews/review-requests.
I think at least some of the documentation still needs significant improvement esp. for flow side and some typos / extra fields probably need a re-check before we can get to merge-able state.

[apratimmukherjee]

This description is wrong. Needs to be fixed . On the lines of something like :
Specification of the traffic profile for flows from QPs on this port.
Currently , only target_line_rate mode is supported. When this mode is selected, all the configured QPs on this port will send data packets as per configured message_size with the messages from different flows/QPs being interleaved ( this is JUST an example ... needs exact expected technical specification in terms of RoCEv2 which should be provided by someone with proper understanding of how to set expectations for this mode in a generic manner ) with each other such that the cumulative rate for all packets being transmitted across all flows never exceed the target_line_rate.value ( exact field must be specified , these are just examples of what sort of info I would expect in this text ) . The actual rate achieved will depend on congestion encountered by DUT which can lead to decrease of actual packet rate achieved due to congestion reduction mechanisms such as DCQCN ( triggered by ECN bits indicating congestions from the DUT ) or due to PFC congestion notifications from DUT.

[apratimmukherjee]

Some responses to comments received.

[apratimmukherjee]

Blocking comments handled. So removing blocking tag.
Some changes still expected in Descriptions.

[kunkeysight]

Approved.