What:		/sys/class/rtrs-client
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	When a user of RTRS API creates a new session, a directory entry with
		the name of that session is created under /sys/class/rtrs-client/<session-name>/

What:		/sys/class/rtrs-client/<session-name>/add_path
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	RW, adds a new path (connection) to an existing session. Expected format is the
		following::

		    <[source addr,]destination addr>
		    *addr ::= [ ip:<ipv4|ipv6> | gid:<gid> ]

What:		/sys/class/rtrs-client/<session-name>/max_reconnect_attempts
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	Maximum number reconnect attempts the client should make before giving up
		after connection breaks unexpectedly.

What:		/sys/class/rtrs-client/<session-name>/mp_policy
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	Multipath policy specifies which path should be selected on each IO:

		round-robin (0):
		    select path in per CPU round-robin manner.

		min-inflight (1):
		    select path with minimum inflights.

		min-latency (2):
		    select path with minimum latency.

What:		/sys/class/rtrs-client/<session-name>/paths/
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	Each path belonging to a given session is listed here by its source and
		destination address. When a new path is added to a session by writing to
		the "add_path" entry, a directory <src@dst> is created.

What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/state
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	RO, Contains "connected" if the session is connected to the peer and fully
		functional.  Otherwise the file contains "disconnected"

What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/reconnect
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	Write "1" to the file in order to reconnect the path.
		Operation is blocking and returns 0 if reconnect was successful.

What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/disconnect
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	Write "1" to the file in order to disconnect the path.
		Operation blocks until RTRS path is disconnected.

What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/remove_path
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	Write "1" to the file in order to disconnected and remove the path
		from the session.  Operation blocks until the path is disconnected
		and removed from the session.

What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_name
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	RO, Contains the name of HCA the connection established on.

What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_port
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	RO, Contains the port number of active port traffic is going through.

What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/src_addr
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	RO, Contains the source address of the path

What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/dst_addr
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	RO, Contains the destination address of the path

What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/cur_latency
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	RO, Contains the latency time calculated by the heart-beat messages.
		Whenever the client sends heart-beat message, it checks the time gap
		between sending the heart-beat message and receiving the ACK.
		This value can be changed regularly.

What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reset_all
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	RW, Read will return usage help, write 0 will clear all the statistics.

What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/cpu_migration
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	RTRS expects that each HCA IRQ is pinned to a separate CPU. If it's
		not the case, the processing of an I/O response could be processed on a
		different CPU than where it was originally submitted.  This file shows
		how many interrupts where generated on a non expected CPU.

		"from:"
		    is the CPU on which the IRQ was expected, but not generated.
		"to:"
		    is the CPU on which the IRQ was generated, but not expected.

What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reconnects
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	Contains 2 unsigned int values, the first one records number of successful
		reconnects in the path lifetime, the second one records number of failed
		reconnects in the path lifetime.

What:		/sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/rdma
Date:		Feb 2020
KernelVersion:	5.7
Contact:	Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description:	Contains statistics regarding rdma operations and inflight operations.
		The output consists of 6 values::

		    <read-count> <read-total-size> <write-count> \
		    <write-total-size> <inflights> <failovered>