Msr command cell facility
Download
1 / 30

MSR Command Cell Facility - PowerPoint PPT Presentation


  • 65 Views
  • Uploaded on

MSR Command Cell Facility. Fred Kuhns Applied Research Laboratory. Highlights. Overview High level description - Application Layer MSR Command Interface Overview Cell format and field definitions Example. Definitions.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' MSR Command Cell Facility' - xue


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Msr command cell facility

MSR Command Cell Facility

Fred Kuhns

Applied Research Laboratory


Highlights
Highlights

  • Overview

  • High level description - Application Layer

  • MSR Command Interface Overview

  • Cell format and field definitions

  • Example


Definitions
Definitions

  • Session: Open connection between the CP and a specific SPC. Intended to represent open connections and command state

  • Transaction: Represent a complete command. A transaction terminates with either an EOF is received by the CP or and error occurs.

  • EOF: End of File is returned to CP with the last bit of command data is returned or in response to a Cancel message (or an error occurs)


Overview cmd interface on cp
Overview - Cmd Interface on CP

  • Synchronous Request/Response protocol

  • Timeout can be specified as well as the number of retries - Per session option

    • Essentially provides a reliable service

    • Issue: if no reply, cmd/reply msg lost in port, channel or CP. Retries may be a bad thing.

  • Address - MSR Port and Command

    • <MSR_Port, MSR_Command>

  • Message destination - Callback function within the Port’s kernel (implements command)


Command interface on cp
Command Interface on CP

  • Types of messages:

    • New Command,Get Nextset of reply data Command,CancelCommand

    • Error Reply, EOF Reply, Continued Reply

  • Message Identifiers - Only requires a sequence number initialized to 0 for each New Command:

    • One sending entity on CP,

    • One outstanding command for each port,

    • Ports send exactly one reply msg per command msg,

    • Command must fit within one cell,

    • Replies may span multiple cells.


Command interface on port
Command Interface on Port

  • Callback function registered with MSR kernel and called under 3 cases:

    • New Command

      • Flags = Command; Sequence = 0; Length = valid bytes in buffer; Buffer = application data

    • Next Command

      • Flags = Command | Next; Sequence = previous+1; Length = valid bytes in buffer; Buffer = application data

    • Cancel Command

      • Flags = Command | Cancel; Sequence = previous+1; Length = 0; Buffer contains no valid data


Command interface on port1
Command Interface on Port

  • Callback function must:

    • Read from/Write to supplied buffer

    • Set length = Bytes written to buffer (in/out param)

    • Indicate if an error occurred (return -1)

    • Whether more data exists (return 0 => EOF, return > 0 => Not EOF, return < 0 => ERROR | EOF)

  • Framework:

    • generates reply message using same Command value and Sequence number.

    • sets flags indicating status (EOF, Error etc)


Failure modes
Failure Modes

  • Library support for lost messages:

    • if (timeout > 0, Replies > 0), then CP API library will re-send with RETRY flag set.

    • if (timeout > 0, Replies = 0 or all replies failed), then API library returns error to application

    • If (timeout = 0 - No Timeout), then send operation blocks indefinitely.

  • Lost Command message -

    • if (timeout > 0 and retries > 0), CP resends command; same sequence number but RETRY flag set. Command buffer and flags passed to callback fn.


Failure modes1
Failure Modes

  • Lost Reply message,

    • if no retries, Any issues?

    • if retries then CP resends

      • New Command - Port knows this is a duplicate command (RETRY flag). Application responsible for handling retries. If an issue can use unique message ids. Extreme case use a history (last reply message).

      • Next Command - Port receives Command w/Sequence > 0, w/RETRY flag. Passed to application which chooses the correct course of action. The intent is to ensure there are no holes in the reply data received by the CP.

      • Cancel message - same as Next command.


Possible enhancements
Possible Enhancements

  • Support asynchronous messaging:

    • Multiple outstanding commands per port

    • Asynchronous I/O on CP

    • Speed up boot process and dynamic configuration

    • Facilitates implementing port monitoring (ping or heartbeat) for fault detection and recovery.

    • two methods for reporting results:

      • upcall - function registered by application is called when results arrive

      • poll - application periodically polls library for results.

  • Support Broadcast and/or Multicast


Msr command layer
MSR Command Layer

  • Simple messaging facility optimized for MSR.

  • Command message (CP sends):

    • Sent by CP to a specific MSR port (unicast)

    • Must fit within one AAL0 cell.

    • Message header, includes:

      • protocol version

      • Command

      • Sequence number

      • flags

    • Application data follows header

    • Library implements Request/Reply protocol.


Msr command layer1
MSR Command Layer

  • Reply Message (Port sends):

    • Port must send reply message in response to a Command message.

    • Reply message Header:

      • version and sequence number: same as command msg.

      • Includes application data and flags indicating if command was successful and if more data exists (EOF).

    • Application registers command specific callback function at port.

    • Callback function must conform to specified interface.


Msr command overvew
MSR Command Overvew

  • Command Protocol description

    • Control Processor sends command messages to a specific port and expects to receive a reply message indicating either Success or Failure. This is termed a Command Cycle.

    • There is the notion of a Command Transaction which may include one or more command cycles. A command transaction is terminated when the target (port) responds with a reply msg containing an EOF


MSR Command Overvew

  • Command Protocol description, continued

    • CP processing of Reply msg, depends on EOF flag:

      • If EOF is set then no further reply data is available and the command transaction is closed.

      • If EOF is not set then there is remaining data and the command transaction is still open.

    • If remaining data (Not EOF), then CP must follow with a either a Next or Cancel command message.

      • Sequence number indicates the “chunk” of data to be returned.

      • Command indicates the message’s destination

      • sequence number = previous + 1


Apic cell format

24

16

8

0

x x x x x x x x x x x x x

pin

x

pout

xx

cc

ld

cid

gfc

vpi

vci

pti

cl

ver

length

command/status

sequence number

flags

MSR Command

Message

APIC Cell Format

  • Cell payload contains the MSR Command

  • Command header is 4 Bytes, leaving 44 Bytes for sub-commands and data.


Atm apic header

x x x x x x x x x x x x x

pin

x

pout

xx

cc

ld

cid

gfc

vpi

vci

pti

cl

ATM/APIC Header

  • pin (Ports-In) - Port cell arrived

    • Tx not used (set to 000b)

    • Rx: port cell arrived on (is the below correct?)

      • 001 Port 0, 010 Port 1, 100 Port 2, etc.

  • pout (Ports-Out) - Set of output ports.

    • Tx: Command library sets:

      • 001 Fiber/Link, 010 Ribbon/Switch, 011 Both

      • 101 Loopback MV0, 110 Loopback MV1

    • Rx: Set by VCXT, see pin above.


Atm apic cell format

x x x x x x x x x x x x x

pin

x

pout

xx

cc

ld

cid

gfc

vpi

vci

pti

cl

ATM/APIC Cell Format

  • cc (Control Cell Indicator) - Not used, set to 0b

  • ld (Low Delay) - Not used, set to 0b.

    • Should we use low delay?

  • cid (Connection Identifier) - set to vci value.

  • gfc (Generic Flow Control) - set to 0000b.


x x x x x x x x x x x x x

pin

x

pout

xx

cc

ld

cid

gfc

vpi

vci

pti

cl

ATM/APIC Cell Format

  • vpi (Virtual Path Identifier) - Set to 0x0.

  • vci (Virtual Circuit Identifier) - Equal to cid.

    • See presentation on MSR configurations for a complete list of VCI assignments.

  • pti (Payload Type) - Set to 000b (data cell)

  • cl (Cell Loss Priority) - Set to 0b (High Priority)


Msr command header
MSR Command Header

  • Version (2 bits) - Protocol version. Allows for at most 4 versions. Current version set to 0.

    • field width was a trade off with the length field.

  • Length (6 bits) - Number of valid data bytes.

    • 0 <= Length <= 44, so 6 bits sufficient.

    • This field is indirectly set by the application or command implementation. The CP library and kernel interfaces allow for applications to pass a buffer pointer and indicate the number of valid data bytes.

ver

length

command/status

sequence number

flags


MSR Command Header

ver

length

command/status

sequence number

flags

  • Command/Status (8 Bits) CP inserts command value, SPC/port inserts status information.

    • Valid Commands are listed in $SYS/msr/msr_ctl.h, also see $MSR/utils/command/*.{c,h}

    • Library API on CP accepts Command as argument. implementation in kernel - array of function pointers, uses Command as index

    • Reply msg Status indicating success or an error code (Upcall, ATM, Cmd Invalid, Cmd Not Implemented, or Other Cmd Error).


Current commands
Current Commands

  • Current set of commands:

    • set_debug/get_debug - set debug flags/masks.

    • port_init - set local port number.

    • policy - Manage MSR policy object. Currently sets the Policy control flags (Gate 0 enable)

    • dq - Preshanth?

    • fipl -Ed?

    • rp_pcu - extensive set of sub-commands.

    • rp_inst - send message to plugin instance

    • rp_class - send message to plugin base class


Msr debug command
MSR DEBUG Command

  • Dynamic setting of debug mask - affects what messages are sent to syslogd

  • set_debug or get_debug command

    • Valid debug categories/modules

      • apic, ipfwd, iprx, iptx, mem, dq, stats, ctl, conf, kern, natm, pcu, plugin, gate

    • Valid debug levels - 0 - 255

      • predefined: verbose, warning, error, critical

  • Interface in MSR kernel

  • MSR_DEBUG((MSR_DEBUG_<category>|MSR_DEBUG_LEVEL_<level>, “format”, args));


Msr rp pcu command
MSR RP_PCU Command

  • rp_pcu Command sub-commands

    • addfltr/remfltr - add/remove pkt filter at gate x

    • flist - port prints current gate x filter list

    • bind/unbind - bind/unbind instance to fltr/gate combo

    • create - create plugin instance

    • free - remove plugin instance

    • clist - port prints current plugin class list

    • ilist - port prints current instance list

    • load - load plugin - not implemented

    • null - no-op. Can be used as a ping operation


MSR Command Header

ver

length

command/status

sequence number

flags

  • Sequence Number (8 Bits) - Is of primary use by the applications.

    • When command message first sent, sequence = 0.

    • If the reply does not include an EOF flag, then CP increments sequence by one for each subsequent command message.

    • When EOF is received the Command Transaction is complete and the sequence number is reset to 0.


MSR Command Header

ver

length

command/status

sequence number

flags

  • Flags (8 bits) - Bit field, valid flags are:

    • Invalid - flag = 0, should not occur

    • CMD - cell contains a valid command from CP

    • REPLY - cell contains reply from Port

    • ERROR - Reply only, error processing on Port

    • EOF - No reply data remains, end of cmd transaction

    • NEXT - get next set of reply data

    • CANCEL - cancel current cmd transaction

    • RETRY - set if cp resend a command after it was lost


Cp library api
CP Library API

  • Library API for application on CP,

    • int sendcmd(int sid, int cmd, char *data, int flags, int *dlen)

      • sid = session id,

      • cmd - Command to execute on port

      • data = buffer pointer,

      • flags =

        • RETRY (reply timeout),

        • CANCEL (cancel current command),

        • Next (get next set of reply data)


Msr kernel api
MSR Kernel API

  • MSR kernel interface $SYS/msr/msr_ctl.{h,c}

  • Callback function signature:

    • msr_ctl_<cmd> (void *buf, u_int8_t flags, u_int8_t seq, u_int8_t *dlen)

    • buf = command buffer w/application data,

    • flags =

      • CMD,

      • NEXT,

      • RETRY or

      • CANCEL,

    • seq = sequence number indicating reply data set, and

    • dlen is input/output parameter, data length in bytes.


Kernel state diagram
Kernel State Diagram

Next

Retry

Command

Command

Command

Cancel

EOF

Proto Error

Closed

Idle

Proto Error

Retry


Cp library state diagram
CP Library State Diagram

Retry

Next

Result of a timeout

Wait

(for reply)

Protocol Error

Command

EOF

Open Session

Closed

Idle

Protocol Error


Example sending cmd to port

cell hdr

cmd

data

Example Sending Cmd to Port

Lookup sub-command

perform function call

then report results

wugs

192.168.200.X

192.168.204.X

P4

P0

SPC/FPX

SPC/FPX

Next/Prev

Hop

Next/Prev

Hop

msr_ctl

DQ

DQ

192.168.205.X

192.168.201.X

P5

P1

SPC/FPX

SPC/FPX

reply();

plugin instance created:

Status,

Instance ID

Next/Prev

Hop

Next/Prev

Hop

DQ

DQ

sendcmd();

create plugin instance:

port id = 0,

PluginID = 200

192.168.202.X

192.168.206.X

P6

P2

Report command

completion status

to application.

SPC/FPX

SPC/FPX

Next/Prev

Hop

Next/Prev

Hop

DQ

DQ

192.168.202.2

192.168.203.X

192.168.207.X

P7

P3

SPC/FPX

SPC/FPX

Next/Prev

Hop

CP

DQ

DQ

192.168.203.2


ad