WW TSS-05InBatch Advanced Troubleshooting Matt Mummert Shri Mariyala October, 2012
Project Start.Best Practices. Batch systems are often considered mission critical in the production process; therefore, batch systems need to be designed so that the process integrity is ensured even during unexpected external events. A batch management solution, facilitating the batch execution on physical process equipment, expands across multiple layers of the system, including (but not limited to): Control system infrastructure Network infrastructure Computer system infrastructure Batch Management software Userinteraction Each one of these layers must be thoroughly analyzed and considered as a critical part of the batch management solution. It is vital that, after the critical variables surrounding the batch execution are identified, the system is designed to address their needs at the right level. (From InBatchR&D WHITE PAPER - Best Practices and Considerations for Designing Reliable InBatch Solutions)
Project Start.Best Practices. One of the most important things you can do on a project is to make sure that the system functional specification completely describes the desired operation of the system and no more. (From InBatch deployment guide)
Best Pratices – What Do We Take Away? • InBatchDeployment Guide • InBatchR&D WHITE PAPER - Best Practices and Considerations for Designing Reliable InBatch Solutions
Advanced Troubleshooting - Topics • InBatch Architecture • Batch Equipment • Batch Phase Logic • IBCli/DAServer • Batch IBMX • Batch Warm Restart Files • BatchMngr Parameters
Advanced Troubleshooting - Topics • BatchGuiConfig Ctrl • Batch Redundancy • Batch History • Reports • System Monitoring • Log Flags
InBatch Architecture 1 Batch Server + DAServer Access Reports Remotely PLCs InBatch Dev Client InBatch RuntimeClients/Touch Information Server Batch History
InBatch Integrate with System Platform GR AOS Batch Server + DAServer Access Reports Remotely • AOr PLCs InBatch Dev Client InBatch RuntimeClients/Touch Information Server Batch History
Extremely flexible and powerful batch management capabilities Extremely flexible and powerful platform and plant model MX communication Phases modeled within Control System PHASE 4 PHASE 5 PHASE 6 PHASE 1 PHASE 2 PHASE 3 Phases modeled within System Platform System Platform Integration – How ? InBatch Server Device Device Integration SystemPlatform
Batch Equipment Analysis • Batch started but no activity, why? • What is equipment status? • How to track and identify which equipment in use? • What is Unit, Connection & Segment Status? • How to update equipment status (TagView)
Equipment Status – What Do We Take Away? • Unit Status values must be consistent - Process Model Vs USTAT tags • USTAT tags are linked to the correct addresses in PLC • Blank or incorrect values for USTAT leads to “waiting for available equipment” state • Allocating a connection allocates all segments of that connection • All connections that contain segment allocated by another connection are ‘busy’ • Technote 25 – Using Segments to define Model Limitations in InBatch.
Monitor Phase Status – WWClient Note: Phase status values using WWClient
Phase Monitor – What Do We Take Away? • Phase Control/Status bits linked to the control system must be unique and valid addresses in PLC. Duplicate items cause issues • Phase must always be in ready state prior to the beginning of the phase execution • At any point in time, only status is present. • Technotes as follows:
Phase Monitor – What Do We Take Away? • TN 555 – Active batch behavior when IBCli looses communications to IOServer • TN 504 - Phase control bit behavior • TN 312 – InControl phase logic for InBatch • TN 23 – Developing Allen Bradley Phase logic for InBatch
InBatch IBCli Parameters • Response Time - Time after which an IBCli request to the IOServer will timeout. Default 15 seconds • Ping Time - Interval at which IBCli will ping the IOServer to detect a connection loss. Default 2 seconds. To enable debug for IBCli, set the Ping Time to “-999” • Advise All - Performs an Advise All on initialization • Disable Timestamp – Disable the end of phase timestamp feature for this topic.
IBCli debug – Log Messages with PingTime = -999Reading a value from DAServer/IADAS:2011/07/20 16:20:13.399/-03B4:14A4/IBCli /...(ProcessUpdate) Update for [G2_V9072-V9074-CS-DONE] of  2011/07/20 16:20:13.399/-03B4:14A4/IBCli /...(ProcessUpdate) Time Stamp for [G2_V9072-V9074-CS-DONE] of [2011/07/20 20:20:11] Quality [1c0]
IBCli debug – Log Messages with PingTime = -999InBatch writing value to DAServer/IADAS2011/07/20 16:20:13.899/-03B4:14A4/IBCli /(tt) wwCliPoke - request:  tag: <G2_V9072-V9074-CS-RESET> Value:2011/07/20 16:20:13.899/-03B4:14A4/IBCli /...(tt) Adding tag to request - request:  tag: <G2_V9072-V9074-CS-RESET> req type:  2011/07/20 16:20:13.899/-03B4:14A4/IBCli /...(ProcessUpdate) Update for [G2_V9072-V9074-CS-RESET] of  2011/07/20 16:20:13.899/-
InBatch - Environment Editor • Any updates to Environment Editor requires Update Environment to affect changes, which means, stop runtime services, which means, it costs production down time.
InBatch IBMX – What Do We Take Away? • Platform must be deployed to enable IBMX • IDE must be installed in order to browse Galaxy browser inside of TagLinker • Requires IDE and Runtime license for Platform • Use TagView, Phase Logic, Batch Display and ObjectViewer to track Phase Control/Status tags • InBatch and ArchestrA AppServer Communication Recommendations using IBMX – White Paper (Tech Article)
Batch Warm Restart • Warm Restart files - If there is no warm restart folder, batchmngr doesn’t startup • .batchwrdebug – Creating a .batchwrdebug folder from command line enables the debug for warm restart information. This folder created under x:\>program files\wonderware\InBatch\Cfg\Config_a\.batchwrdebug
BatchMngr Parameters • Param Timeout - Time in seconds, of all attempts to read a parameter tag value before timeout failure. Default value set to 30 seconds. If you see any parameter read timeouts in the logger tune this setting • Phase Batch Status – Puts all phases on hold when batch is on hold – Add this setting if all phases needs to be on hold when batch is on hold • Disconnect Clients – Disconnect all deadlock (misbehaving) clients – Add this setting if any clients misbehaving, otherwise it will cause negative impact to the BatchMngr
BatchGuiConfig – Properties • Note: MessageBoxes has to be disabled from popup, otherwise it will cause the view to not respond until message is acked.
BatchGuiConfig – Events • Terminate/Initialize the control using Startup/Shutdown events of Batch Server accordingly.
SFCCtrl and OcxBatchCtrl • Terminate/Initialize these controls inside of view using Shutdown/Startup events of BatchServer
InBatch ActiveX Controls – What Do We Take Away? • View locks up due to InBatch ActiveX Controls not terminated gracefully when connection lost to BatchServer (Server shutdown/loss network connection) • Technote 364 – How to configure BatchGuiConfig ActiveX Ctrl for Redundancy Client Switch-Over. This technote is good for Standalone Server also. • Starting with InBatch 9.5, use Technote 797 “Configuring InBatch Runtime Client Communication Timeouts.” These timeouts are configured in registry and technote covers details on configuration.