1 / 32

Slc-aware Ioc

Slc-aware Ioc. SLC Executive Message Service. SLC Executive. Slc-aware IOC Initialization The SLC Executive Thread Startup Process Shutdown Process Some General Thread Issues Utilities. Overall Architecture. slcIoc Initialization. The saIocMain.cpp file

tom
Download Presentation

Slc-aware Ioc

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Slc-aware Ioc SLC Executive Message Service Diane Fairley 11/11/2004

  2. SLC Executive • Slc-aware IOC Initialization • The SLC Executive Thread • Startup Process • Shutdown Process • Some General Thread Issues • Utilities

  3. Overall Architecture

  4. slcIocInitialization • The saIocMain.cpp file • Epics initialization via st.cmd file • Read environment variables • slcIoc name (like the microname: XL01, XL02, … ) • Proxy IP address (ip address of PX01 for now) • Load epics databases • Currently a Restart/Stop subroutine record • start cmlog client • iocInit() • slcStart() • iocsh(NULL)

  5. slcStart() • Initializes slc globals • Starts the slcExec thread • Executes the Restart function

  6. SLC Globals • slcThreads[ thread enum] • Array of thread information and global resources • discovered_name • Name string, from environment variable • proxy_ip_address • from environment variable • The ip address string in xxx.xxx.xxx.xxx format • Db downloadEvent semaphore • saIocJobs[jobcode] = thread enum • converts micro job code to an slcIoc thread enumeration • slcSockets[ socket enum] • Array of socket descriptors • dbExists

  7. The SLC ExecutiveThread • Manages the slc startup, creating all other slc threads in order • Supervises threads and globals • Manages a clean shutdown of slc threads • Responds to slcRestart (IPL a micro) • Responds to slcStop (RESET a micro) • SlcExec thread initialization • slcCmlogOpen • Create a message queue • Set slcExec global active=True • Go into loop waiting for messages

  8. slcRestart • Commanded via IPL on VMS control system • TST_SLC_NOTIFY message “BOOT” • Received by msgRecv, forwarded to msgHdlr • Commanded from ioc shell • >slcRestart • Commanded from channel access via subroutine record • Sets CSTR:<micro>:1:REST.A=0, • EDM display can set this • Commanded from slcStart() initialization

  9. SLC Restart

  10. slcRestart • Send TEST_START message to slcExec • From msgHdlr, or • from slcRestart iocsh command, or • from subRecord • slcExec process: Phase I • If threads are running, call slcStop • Wait for all threads to stop • Create downloadEvent semaphore for threads to wait on • Create all threads except msgRecv, dbRecv threads • Wait for these threads to set their global active=True

  11. Restart: Phase I

  12. slcRestart • Restart Phase II • Create the msgRecv and dbRecv threads • Wait for the Recv threads to set their global active=True • Send a db_download_req message to dbSend message queue. • Goes back to its loop waiting for messages and checking on thread active flags

  13. Restart: Phase II

  14. slcRestart • Restart Phase III • Database Service downloads the database • Database Service sets the dbExists flag to true • Database Service signals the downloadEvent semaphore to allow threads to proceed.

  15. Ready to go

  16. General Thread Initialization • slcCmlogOpen • Create resources • Some will create a message queue • Recv threads create a socket • Async threads may create a timer • Create any other thread specific resources • Set active=True • Perform dblists (optional) • Wait on database event • Go into loop waiting for messages or wake-up

  17. slcStop • Commanded via RESET on VMS control system • TST_SLC_NOTIFY message “RSET” • Received by msgRecv • Commanded from the ioc shell • >slcStop • Commanded from channel access via subroutine record • Sets CSTR:<micro>:1:REST.A = 1 • EDM display can set this

  18. slcStop • Send TEST_STOP message to slcExec • From msgHdlr, or • from iocsh command, or • from subRecord • slcExec process • Set all the thread global stop=True; • Send TEST_STOP message to all threads with message queues • Wait for all thread global “active” flags to be set false • Wait for all threads to stop • Delete the global db downloadEvent • Set dbExists to false; • Set all thread global “stop” flags back to False; • Could re-initialize all globals, do final global resource cleanup…. • go back to loop waiting for messages

  19. General Thread Shutdown • Release resources • msg queues, • sockets, • heap memory, • timers, • semaphores • slcCmlogClose() • set active=False • Set tid=NO_THREAD • Return

  20. General slcIoc thread assumptions • There are three basic thread types • ‘socket’ threads that wait at sockets • ‘message queue’ threads that wait at queues • Async threads that wake up periodically • Non-message queue threads must poll the stop flag! • No ‘kill a thread’ function in epics thread library • Allows for orderly clean-up in all threads at all times • ‘waits’ should have a timeout • Possibly, message queue threads may wait indefinitely at the queue because they can receive a TEST_STOP message. • Sockets should be non-blocking or interruptable

  21. Some Utilities • Thread Utilities • slcThreadNameEnum getThreadEnum(epicsThreadId tid); • slcThreadNameEnum getThreadEnumByName(const char* name); • char* getMicroname(void); eg: ‘XL02’ • char* getProxyIPString(void); eg: ‘142.67.94.25’ • some new wait(…) utilities • To be used by threads when waiting (with a timeout) on a flag or socket or …. • slcCmlog Utilities • int slcCmlogOpen(void); • int slcCmlogClose(void); • int slcCmlogLogMsg (…,.,..,…., ...); • Message queue utilities • epicsMessageQueueId createMsgQ( slcThreadNameEnum thrd ); • vmsstat_t sendToMsgQ ( slcThreadNameEnum thrd, void* msg, int size );

  22. slcIoc Message Service • Threads overview • Message structure • Some Assumptions • Message Flow • Utilities

  23. Message Service

  24. Slc Message Service • The msgRecv thread • “owns” the msg service socket; creates and destroys it • Receives all incoming messages at the msg service socket • Sends the incoming message to the proper slc Hdlr thread • Uses the slcIocJobs[job code] global • The msgSend thread • Sends all outgoing message at the message service socket • “uses” the message service socket • When the connection fails, waits for the msgRecv thread to re-connect. • Checks for a valid socket in slcSockets[sdMsg] • The msgHdlr thread • Handles messages with the TEST function code • Handles the new TST_SLC_NOTIFY messages

  25. Message Structure NOTE: this structure is shared with VMS and micros

  26. Message Service Assumptions • Sockets are non-blocking or interruptable • They must be able to see the global thread stop flag • Large messages will need heap memory allocations • Message queues do not (efficiently) allow variable size msgs • Use a memory pool to avoid memory leaks, fragmentation • Messages from the VMS control system are little-endian, VMS format, and packed. • The message service threads are responsible for converting the header portions of the message • The other ‘job’ threads are responsible for converting the function specific portion of the message (reqdata)

  27. Message Service message flow • The msgRecv thread • Receives the message into a local inMsg variable using a getBuffer utility. • interprets the fwdheader • Allocate memory to store a multi-packet message. • converts/unpacks the msgheader into native format • replaces timestamp in msgheader • Translates the incoming job code to an slcIoc thread id • sends the entire message on to the correct queue. • Message queue send actually copies the message • Large messages: reqdata portion would include a pointer

  28. Message Service message flow • The job thread • Reads the message into an inMsg variable. • Ignores the fwdheader portion • Reads the message data length from the msgheader • converts/unpack the incoming function specific data • Releases large message memory when it can, if necessary • uses the data – unidentified requirements here. • converts/packs the function specific reply into an outMsg variable. • Allocates memory for large response message • fills in the new msgheader. (leaves it in native format) • The outMsg reply is sent to the msgSend queue.

  29. Message service message flow • The msgSend thread • Reads the message into a msg variable • converts/packs the msgheader • fills in the fwdheader • sends reply to VMS control system • Creates and sends multiple buffers for large messages • De-allocates memory used by large messages

  30. Converting and packing/unpacking • Forward header and Msgheader • The forward header follows networking standard in VMS and micros – uses hton and ntoh • Jobs don’t need to see the forward header • Conversion and unpacking handled in by Message Service • Function Specific Data (requests and replies) • Typically a structure is defined for the data and shared by the VMS, micros and slcIocs • See nfs_ref/inc_c for xxx_reqreply_struc.* • To unpack, must memcpy data field-by-field from incoming msg.reqdata[] to a variable of this structure, then convert. • To pack, do the reverse, convert then memcpy field-by-field from structure to outgoing msg.reqdata[] • Utilities will be written for each shared structure.

  31. Conversion Utilities • void sswap(const unsigned short* s); • void lswap(const unsigned long* l); • void swap_int2u(void* pOut2u, const void* pIn2u); • void swap_int4u(void* pOut4u, const void* pIn4u); • void cvt_vms_to_ieee(void* pOutR4, const void* pInR4); • void cvt_ieee_to_vms(void* pOutR4, const void* pInR4); • int getVMSTime(void* pDest); • int epicsTimeToVMSTime(void* pDest, const epicsTimeStamp* eTime); • long int diffVMSTimes_10usec(void* pRef, void* pLater); • long int diffVMSTimes_10msec(void* pRef, void* pLater);

  32. TCP Utilities • SOCKET getConnection(unsigned short port, unsigned short socket); • Gets a connection with the proxy • Registers with the proxy • Reports error conditions • vmsstat_t getBuffer(SOCKET sd, fwd_hdr_ts *pFwd, void *pBuf, int bytesReqd, int *pBytesRcvd); • Reads incoming message • Allocates memory and re-creates full message from multi-packet • Reports error conditions

More Related