Skip to content

Commit 294a5b9

Browse files
aireillyaireilly
committed
Adding docs for 3 card T-GM
1 parent 5c19307 commit 294a5b9

13 files changed

+486
-30
lines changed

‎_topic_maps/_topic_map.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -1504,10 +1504,10 @@ Topics:
15041504
- Name: Using Stream Control Transmission Protocol
15051505
File: using-sctp
15061506
Distros: openshift-enterprise,openshift-origin
1507-
- Name: Using Precision Time Protocol hardware
1507+
- Name: Using PTP hardware
15081508
Dir: ptp
15091509
Topics:
1510-
- Name: About Precision Time Protocol in OpenShift cluster nodes
1510+
- Name: About PTP in OpenShift cluster nodes
15111511
File: about-ptp
15121512
- Name: Configuring PTP devices
15131513
File: configuring-ptp

‎images/561_OpenShift_Using_PTP_network_0124.png

-179 KB
Binary file not shown.

‎images/openshift-ptp-3-card-grandmaster.png

466 KB
Loading

‎images/openshift-ptp-using-dual-nic-ptp.png

108 KB
Loading

‎modules/nw-ptp-configuring-linuxptp-services-as-grandmaster-clock-dual-nic.adoc

+10-9
Original file line numberDiff line numberDiff line change
@@ -4,21 +4,22 @@
44

55
:_mod-docs-content-type: PROCEDURE
66
[id="configuring-linuxptp-services-as-grandmaster-clock-dual-nic_{context}"]
7-
= Configuring linuxptp services as a grandmaster clock for dual E810 NICs
7+
= Configuring linuxptp services as a grandmaster clock for dual E810 Westport Channel NICs
88

9-
You can configure the `linuxptp` services (`ptp4l`, `phc2sys`, `ts2phc`) as a grandmaster clock (T-GM) for dual E810 NICs by creating a `PtpConfig` custom resource (CR) that configures the host NICs.
9+
You can configure the `linuxptp` services (`ptp4l`, `phc2sys`, `ts2phc`) as a grandmaster clock (T-GM) for 2 E810 NICs by creating a `PtpConfig` custom resource (CR) that configures the NICs.
1010

11-
You can configure the `linuxptp` services as a T-GM for the following dual E810 NICs:
11+
You can configure the `linuxptp` services as a T-GM for the following E810 NICs:
1212

13-
* Intel E810-XXVDA4T Westport Channel NICs
14-
* Intel E810-CQDA2T Logan Beach NICs
13+
* Intel E810-XXVDA4T Westport Channel NIC
14+
* Intel E810-CQDA2T Logan Beach NIC
1515
16-
For distributed RAN (D-RAN) use cases, you can configure PTP for dual-NICs as follows:
16+
For distributed RAN (D-RAN) use cases, you can configure PTP for 2 NICs as follows:
1717

18-
* NIC one is synced to the global navigation satellite system (GNSS) time source.
19-
* NIC two is synced to the 1PPS timing output provided by NIC one. This configuration is provided by the PTP hardware plugin in the `PtpConfig` CR.
18+
* NIC 1 is synced to the global navigation satellite system (GNSS) time source.
19+
* NIC 2 is synced to the 1PPS timing output provided by NIC one. This configuration is provided by the PTP hardware plugin in the `PtpConfig` CR.
2020
21-
The dual-NIC PTP T-GM configuration uses a single instance of `ptp4l` and one `ts2phc` process reporting two `ts2phc` instances, one for each NIC.
21+
The 2-card PTP T-GM configuration uses one instance of `ptp4l` and one instance of `ts2phc`.
22+
The `ptp4l` and `ts2phc` programs are each configured to operate on two PTP hardware clocks (PHCs), one for each NIC.
2223
The host system clock is synchronized from the NIC that is connected to the GNSS time source.
2324

2425
[NOTE]

‎modules/nw-ptp-configuring-linuxptp-services-as-grandmaster-clock-three-nic.adoc

+106
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/ptp/configuring-ptp.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="configuring-linuxptp-services-as-grandmaster-clock-three-nic_{context}"]
7+
= Configuring linuxptp services as a grandmaster clock for 3 E810 NICs
8+
9+
You can configure the `linuxptp` services (`ptp4l`, `phc2sys`, `ts2phc`) as a grandmaster clock (T-GM) for 3 E810 NICs by creating a `PtpConfig` custom resource (CR) that configures the NICs.
10+
11+
You can configure the `linuxptp` services as a T-GM with 3 NICs for the following E810 NICs:
12+
13+
* Intel E810-XXVDA4T Westport Channel NIC
14+
* Intel E810-CQDA2T Logan Beach NIC
15+
16+
For distributed RAN (D-RAN) use cases, you can configure PTP for 3 NICs as follows:
17+
18+
* NIC 1 is synced to the Global Navigation Satellite System (GNSS)
19+
* NICs 2 and 3 are synced to NIC 1 with 1PPS faceplate connections
20+
21+
Use the following example `PtpConfig` CRs as the basis to configure `linuxptp` services as a 3-card Intel E810 T-GM.
22+
23+
.Prerequisites
24+
25+
* For T-GM clocks in production environments, install 3 Intel E810 NICs in the bare-metal cluster host.
26+
27+
* Install the OpenShift CLI (`oc`).
28+
29+
* Log in as a user with `cluster-admin` privileges.
30+
31+
* Install the PTP Operator.
32+
33+
.Procedure
34+
35+
. Create the `PtpConfig` CR. For example:
36+
37+
.. Save the following YAML in the `three-nic-grandmaster-clock-ptp-config.yaml` file:
38+
+
39+
.PTP grandmaster clock configuration for 3 E810 NICs
40+
[%collapsible]
41+
====
42+
[source,yaml]
43+
----
44+
include::snippets/ptp_PtpConfigThreeCardGmWpc.yaml[]
45+
----
46+
====
47+
+
48+
[NOTE]
49+
====
50+
Set the value for `ts2phc.nmea_serialport` to `/dev/gnss0`.
51+
====
52+
53+
.. Create the CR by running the following command:
54+
+
55+
[source,terminal]
56+
----
57+
$ oc create -f three-nic-grandmaster-clock-ptp-config.yaml
58+
----
59+
60+
.Verification
61+
62+
. Check that the `PtpConfig` profile is applied to the node.
63+
64+
.. Get the list of pods in the `openshift-ptp` namespace by running the following command:
65+
+
66+
[source,terminal]
67+
----
68+
$ oc get pods -n openshift-ptp -o wide
69+
----
70+
+
71+
.Example output
72+
[source,terminal]
73+
----
74+
NAME READY STATUS RESTARTS AGE IP NODE
75+
linuxptp-daemon-74m3q 3/3 Running 3 4d15h 10.16.230.7 compute-1.example.com
76+
ptp-operator-5f4f48d7c-x6zkn 1/1 Running 1 4d15h 10.128.1.145 compute-1.example.com
77+
----
78+
79+
.. Check that the profile is correct. Run the following command, and examine the logs of the `linuxptp` daemon that corresponds to the node you specified in the `PtpConfig` profile:
80+
+
81+
[source,terminal]
82+
----
83+
$ oc logs linuxptp-daemon-74m3q -n openshift-ptp -c linuxptp-daemon-container
84+
----
85+
+
86+
.Example output
87+
[source,terminal]
88+
----
89+
ts2phc[2527.586]: [ts2phc.0.config:7] adding tstamp 1742826342.000000000 to clock /dev/ptp11 <1>
90+
ts2phc[2527.586]: [ts2phc.0.config:7] adding tstamp 1742826342.000000000 to clock /dev/ptp7 <1>
91+
ts2phc[2527.586]: [ts2phc.0.config:7] adding tstamp 1742826342.000000000 to clock /dev/ptp14 <1>
92+
ts2phc[2527.586]: [ts2phc.0.config:7] nmea delay: 56308811 ns
93+
ts2phc[2527.586]: [ts2phc.0.config:6] /dev/ptp14 offset 0 s2 freq +0 <2>
94+
ts2phc[2527.587]: [ts2phc.0.config:6] /dev/ptp7 offset 0 s2 freq +0 <2>
95+
ts2phc[2527.587]: [ts2phc.0.config:6] /dev/ptp11 offset 0 s2 freq -0 <2>
96+
I0324 14:25:05.000439 106907 stats.go:61] state updated for ts2phc =s2
97+
I0324 14:25:05.000504 106907 event.go:419] dpll State s2, gnss State s2, tsphc state s2, gm state s2,
98+
I0324 14:25:05.000906 106907 stats.go:61] state updated for ts2phc =s2
99+
I0324 14:25:05.001059 106907 stats.go:61] state updated for ts2phc =s2
100+
ts2phc[1742826305]:[ts2phc.0.config] ens4f0 nmea_status 1 offset 0 s2
101+
GM[1742826305]:[ts2phc.0.config] ens4f0 T-GM-STATUS s2 <3>
102+
----
103+
<1> `ts2phc` is updating the PTP hardware clock.
104+
<2> Estimated PTP device offset between PTP device and the reference clock is 0 nanoseconds.
105+
The PTP device is in sync with the leader clock.
106+
<3> T-GM is in a locked state (s2).

‎modules/nw-ptp-grandmaster-clock-class-reference.adoc

-3
Original file line numberDiff line numberDiff line change
@@ -25,9 +25,6 @@ For example, the PRTC is traceable to a GNSS time source.
2525
|T-GM clock is in `HOLDOVER` mode, and within holdover specification.
2626
The clock source might not be traceable to a category 1 frequency source.
2727

28-
|`gm.ClockClass 140`
29-
|T-GM clock is in `HOLDOVER` mode, is out of holdover specification, but it is still traceable to the category 1 frequency source.
30-
3128
|`gm.ClockClass 248`
3229
|T-GM clock is in `FREERUN` mode.
3330
|====

‎modules/nw-ptp-three-nic-hardware-config-reference.adoc

+41
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/ptp/configuring-ptp.adoc
4+
5+
:_mod-docs-content-type: REFERENCE
6+
[id="nw-ptp-three-nic-hardware-config-reference_{context}"]
7+
= 3-card E810 NIC configuration reference
8+
9+
Use this information to understand how to configure 3 E810 NICs as PTP grandmaster clock (T-GM).
10+
11+
Before you configure the 3-card cluster host, you must connect the 3 NICs by using the 1PPS faceplate connections.
12+
The primary NIC `1PPS_out` outputs feed the other 2 NICs.
13+
14+
When you configure a 3-card T-GM, you need to compensate for the 1PPS signal delay that occurs when you connect the NICs by using the SMA1 connection ports.
15+
Various factors such as cable length, ambient temperature, and component and manufacturing tolerances can affect the signal delay.
16+
To compensate for the delay, you must calculate the specific value that you use to offset the signal delay.
17+
18+
.3-card E810 T-GM PtpConfig CR reference
19+
[cols="1,2" width="90%", options="header"]
20+
|====
21+
|PtpConfig field
22+
|Description
23+
24+
|`spec.profile.plugins.e810.pins`
25+
a|Configure the E810 hardware pins with the PTP Operator E810 hardware plugin.
26+
27+
* `$iface_timeTx1.SMA1` enables the `1PPS OUT` connection for `SMA1` on NIC 1.
28+
* `$iface_timeTx1.SMA2` enables the `1PPS OUT` connection for `SMA2` on NIC 1.
29+
* `$iface_timeTx2.SMA1` and `$iface_timeTx3.SMA1` enables the `1PPS IN` connection for `SMA1` on NIC 2 and NIC 3.
30+
* `$iface_timeTx2.SMA2` and `$iface_timeTx3.SMA2` disables the `SMA2` connection on NIC 2 and NIC 3.
31+
32+
|`spec.profile.ts2phcConf`
33+
|Use the `ts2phcConf` field to configure parameters for the NICs.
34+
Set `ts2phc.master 0` for NIC 2 and NIC 3.
35+
This configures the timing source for NIC 2 and NIC 3 from the 1PPS input, not GNSS.
36+
Configure the `ts2phc.extts_correction` value for NIC 2 and NIC 3 to compensate for the delay that is incurred for the specific SMA cable and cable length that you use.
37+
The value that you configure depends on your specific measurements and SMA1 cable length.
38+
39+
|`spec.profile.ptp4lConf`
40+
|Set the value of `boundary_clock_jbod` to 1 to enable support for multiple NICs.
41+
|====

‎modules/ptp-dual-nics.adoc

+21-12
Original file line numberDiff line numberDiff line change
@@ -4,40 +4,49 @@
44

55
:_mod-docs-content-type: CONCEPT
66
[id="ptp-dual-nics_{context}"]
7-
= Using dual-NIC Intel E810 hardware with PTP
7+
= 2-card E810 NIC configuration reference
88

9-
{product-title} supports single and dual-NIC Intel E810 hardware for precision PTP timing in grandmaster clocks (T-GM) and boundary clocks (T-BC).
9+
{product-title} supports single and dual-NIC Intel E810 hardware for PTP timing in grandmaster clocks (T-GM) and boundary clocks (T-BC).
1010

1111
Dual NIC grandmaster clock::
12+
+
13+
--
1214
You can use a cluster host that has dual-NIC hardware as PTP grandmaster clock.
1315
One NIC receives timing information from the global navigation satellite system (GNSS).
1416
The second NIC receives the timing information from the first using the SMA1 Tx/Rx connections on the E810 NIC faceplate.
1517
The system clock on the cluster host is synchronized from the NIC that is connected to the GNSS satellite.
16-
+
18+
1719
Dual NIC grandmaster clocks are a feature of distributed RAN (D-RAN) configurations where the Remote Radio Unit (RRU) and Baseband Unit (BBU) are located at the same radio cell site.
1820
D-RAN distributes radio functions across multiple sites, with backhaul connections linking them to the core network.
19-
+
21+
2022
.Dual NIC grandmaster clock
21-
image::561_OpenShift_Using_PTP_network_0124.png[Dual NIC PTP grandmaster clock connected to GNSS timing source and downstream PTP boundary and ordinary clocks]
22-
+
23+
image::openshift-ptp-using-dual-nic-ptp.png[Dual NIC PTP grandmaster clock connected to GNSS timing source and downstream PTP boundary and ordinary clocks]
24+
2325
[NOTE]
2426
====
25-
In a dual-NIC T-GM configuration, a single `ts2phc` process reports as two `ts2phc` instances in the system.
27+
In a dual-NIC T-GM configuration, a single `ts2phc` program operate on two PTP hardware clocks (PHCs), one for each NIC.
2628
====
29+
--
2730

2831
Dual NIC boundary clock::
29-
For 5G telco networks that deliver mid-band spectrum coverage, each virtual distributed unit (vDU) requires connections to 6 radio units (RUs). To make these connections, each vDU host requires 2 NICs configured as boundary clocks.
3032
+
33+
--
34+
For 5G telco networks that deliver mid-band spectrum coverage, each virtual distributed unit (vDU) requires connections to 6 radio units (RUs). To make these connections, each vDU host requires 2 NICs configured as boundary clocks.
35+
3136
Dual NIC hardware allows you to connect each NIC to the same upstream leader clock with separate `ptp4l` instances for each NIC feeding the downstream clocks.
37+
--
3238

3339
Highly available system clock with dual-NIC boundary clocks::
34-
You can configure Intel E810-XXVDA4 Salem channel dual-NIC hardware as dual PTP boundary clocks that provide timing for a highly available system clock.
35-
This is useful when you have multiple time sources on different NICs.
36-
High availability ensures that the node does not lose timing synchronisation if one of the two timing sources is lost or disconnected.
3740
+
41+
--
42+
You can configure Intel E810-XXVDA4 Salem channel dual-NIC hardware as dual PTP boundary clocks that provide timing for a highly available system clock.
43+
This configuration is useful when you have multiple time sources on different NICs.
44+
High availability ensures that the node does not lose timing synchronization if one of the two timing sources is lost or disconnected.
45+
3846
Each NIC is connected to the same upstream leader clock.
3947
Highly available boundary clocks use multiple PTP domains to synchronize with the target system clock.
4048
When a T-BC is highly available, the host system clock can maintain the correct offset even if one or more `ptp4l` instances syncing the NIC PHC clock fails.
4149
If any single SFP port or cable failure occurs, the boundary clock stays in sync with the leader clock.
42-
+
50+
4351
Boundary clock leader source selection is done using the A-BMCA algorithm. For more information, see link:https://www.itu.int/rec/T-REC-G.8275.1/en[ITU-T recommendation G.8275.1].
52+
--

‎modules/ptp-three-card-grandmaster.adoc

+29
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * networking/ptp/about-ptp.adoc
4+
5+
:_mod-docs-content-type: CONCEPT
6+
[id="ptp-three-card-grandmaster_{context}"]
7+
= 3-card Intel E810 PTP grandmaster clock
8+
9+
{product-title} supports cluster hosts with 3 Intel E810 NICs as PTP grandmaster clocks (T-GM).
10+
11+
3-card grandmaster clock::
12+
+
13+
--
14+
You can use a cluster host that has 3 NICs as PTP grandmaster clock.
15+
One NIC receives timing information from the global navigation satellite system (GNSS).
16+
The second and third NICs receive the timing information from the first by using the SMA1 Tx/Rx connections on the E810 NIC faceplate.
17+
The system clock on the cluster host is synchronized from the NIC that is connected to the GNSS satellite.
18+
19+
3-card NIC grandmaster clocks can be used for distributed RAN (D-RAN) configurations where the Radio Unit (RU) is connected directly to the distributed unit (DU) without a front haul switch, for example, if the RU and DU are located in the same radio cell site.
20+
D-RAN distributes radio functions across multiple sites, with backhaul connections linking them to the core network.
21+
22+
.3-card Intel E810 PTP grandmaster clock
23+
image::openshift-ptp-3-card-grandmaster.png[3-card PTP grandmaster clock connected to GNSS timing source and downstream PTP boundary and ordinary clocks]
24+
25+
[NOTE]
26+
====
27+
In a 3-card T-GM configuration, a single `ts2phc` process reports as 3 `ts2phc` instances in the system.
28+
====
29+
--

‎networking/ptp/about-ptp.adoc

+4-2
Original file line numberDiff line numberDiff line change
@@ -30,10 +30,12 @@ include::modules/nw-ptp-introduction.adoc[leveloffset=+1]
3030
Before enabling PTP, ensure that NTP is disabled for the required nodes. You can disable the chrony time service (`chronyd`) using a `MachineConfig` custom resource. For more information, see xref:../../machine_configuration/machine-configs-configure.adoc#cnf-disable-chronyd_machine-configs-configure[Disabling chrony time service].
3131
====
3232

33-
include::modules/ptp-dual-nics.adoc[leveloffset=+1]
34-
3533
include::modules/ptp-linuxptp-introduction.adoc[leveloffset=+1]
3634

3735
include::modules/ptp-overview-of-gnss-grandmaster-clock.adoc[leveloffset=+1]
3836

3937
include::modules/cnf-about-ptp-and-clock-synchronization.adoc[leveloffset=+1]
38+
39+
include::modules/ptp-dual-nics.adoc[leveloffset=+1]
40+
41+
include::modules/ptp-three-card-grandmaster.adoc[leveloffset=+1]

‎networking/ptp/configuring-ptp.adoc

+6-2
Original file line numberDiff line numberDiff line change
@@ -25,21 +25,25 @@ include::modules/nw-ptp-device-discovery.adoc[leveloffset=+1]
2525

2626
include::modules/nw-ptp-configuring-linuxptp-services-as-grandmaster-clock.adoc[leveloffset=+1]
2727

28-
include::modules/nw-ptp-configuring-linuxptp-services-as-grandmaster-clock-dual-nic.adoc[leveloffset=+1]
28+
include::modules/nw-ptp-configuring-linuxptp-services-as-grandmaster-clock-dual-nic.adoc[leveloffset=+2]
29+
30+
include::modules/nw-ptp-configuring-linuxptp-services-as-grandmaster-clock-three-nic.adoc[leveloffset=+2]
2931

3032
[role="_additional-resources"]
3133
.Additional resources
3234

3335
* xref:../../networking/ptp/ptp-cloud-events-consumer-dev-reference.adoc#cnf-configuring-the-ptp-fast-event-publisher-v2_ptp-consumer[Configuring the PTP fast event notifications publisher]
3436
35-
include::modules/nw-ptp-grandmaster-clock-configuration-reference.adoc[leveloffset=+2]
37+
include::modules/nw-ptp-grandmaster-clock-configuration-reference.adoc[leveloffset=+1]
3638

3739
include::modules/nw-ptp-grandmaster-clock-class-reference.adoc[leveloffset=+2]
3840

3941
include::modules/nw-ptp-e810-hardware-configuration-reference.adoc[leveloffset=+2]
4042

4143
include::modules/nw-ptp-dual-wpc-hardware-config-reference.adoc[leveloffset=+2]
4244

45+
include::modules/nw-ptp-three-nic-hardware-config-reference.adoc[leveloffset=+2]
46+
4347
include::modules/ptp-configuring-dynamic-leap-seconds-handling-for-tgm.adoc[leveloffset=+1]
4448

4549
include::modules/nw-ptp-configuring-linuxptp-services-as-boundary-clock.adoc[leveloffset=+1]

0 commit comments

Comments
 (0)