The evolution of the aps beamline personnel safety system pss
This presentation is the property of its rightful owner.
Sponsored Links
1 / 41

The Evolution of the APS Beamline Personnel Safety System (PSS) PowerPoint PPT Presentation


  • 91 Views
  • Uploaded on
  • Presentation posted in: General

The Evolution of the APS Beamline Personnel Safety System (PSS). Phil McNamara-Presenter Kenneth Belcher Greg Markovich Advanced Photon Source Safety Interlocks Group.

Download Presentation

The Evolution of the APS Beamline Personnel Safety System (PSS)

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The evolution of the aps beamline personnel safety system pss

The Evolution of theAPS Beamline Personnel Safety System (PSS)

Phil McNamara-Presenter

Kenneth Belcher

Greg Markovich

Advanced Photon Source

Safety Interlocks Group

Work supported by U.S. Department of Energy, Office of Science, Office of Basic Energy Sciences, under Contract No. DE-AC02-06CH11357


Pss history overview

PSS History Overview


Gen1 pss

GEN1 PSS

  • Processors:

    • Dissimilar:

      • Chain A - Allen Bradley (AB) PLC5 – ESD, C&C intermixed

      • Chain B - General Electric (GE) 90-70 – ESD only – Faults were not latched

  • Programming Languages

    • Dissimilar :

      • Chain A – Ladder Logic

      • Chain B – State Logic, Ladder Logic and MegaBasic

  • HMI:

    • LEDs and mechanical PBs

  • I/O Interface:

    • Hardwired to terminal blocks

    • Inconsistent addressing

  • DIW Monitoring

    • LOVE process controllers, binary output, 1 channel per station

  • Validations

    • Invasive, 2-3 days

      • Hardwired Front-End Simulator

      • Laptops running DOS (Windows 98)

  • Communication

    • Relay isolation for signals between chains (status, permits, and heart beat)

    • One-way communication to EPICS

      • Chain A – DH+ through a Data Communications Module (PLC writes only, no read functions)

      • Chain B – RS232 through a Processor Control Module (PLC writes only, no read functions)


Gen1 pss1

GEN1 PSS

Open status

Open status

Closed status

Open status

Chain A

Emergency Shutdown and Command and Control PLC

Safety

Shutter 1

Safety

Shutter 2

Photon

Shutter 2

Closed status

SR Trip/Fault

Beam Line

Door closed

Estop

WDT

Integral

Shutters

Closed status

Station A

ACIS

Station A Beam Ready

Station A Searched

Global OnlineSolenoid

WDT

Air Supply

Door closed

Estop

Closed status

SR Trip/Fault

Photon

Shutter 1

Open status

Chain B

Emergency Shutdown PLC

Closed status

Open status

Open status

Note: No closed switch status for PS1 to ACIS


Gen1 pss2

GEN1 PSS

Beamline PSS interface diagram

Front

-

End

Storage Ring

components

MPS

DIW flow sensors

Beamline

Beamline

Beamline

components

vacuum

valves

FE shutter & valve status

DIW flow

Front

-

End vacuum

sensors

position

sensors

sensors

FE shutter & vacuum permit

FEEPS

FEEPS

(

Mezzanine Rack)

PSS

EPICS

EPICS

Front

-

End vacuum

IOC

Beamline component

IOC

valves position

DIW flow sensors

Beamline EPS

(

SV

,

FV

,

FEV

,

BIV

)

PS

1

open request

Remote

FE shutter permit

Shutter

Interface

BL Shutter

Relay contact from Love Controllers

BL Shutter

status

permits

FES Open status

PSS PLC

Front

-

End Control System

(

Mezzanine Rack)

User Panels

Shutter air supply

Beamline Shutters

(

Pushbuttons, key

Pressure sensors

(

1

,

2

or

3

)

switches and LED

Indicators)

Closed switches

Solenoid

FE shutter permit

PSS SR trips

control

GOL key status

PSS

Beamline Control System

Remote IO link

(

Station Enclosure)

Open switches

Solenoid control

Closed switches

(except PS1)

Hutch doors

ACIS Sector Interface

Front

-

End Shutters

Enclosure

Air supply shutoff

(except PS1)

Search

stations

Remote I/O link

E

-

stop

buttons

FES Closed status

ACIS

ACIS PLC

ACIS

Controlled

Up to 6 stations

IOC

(

Main Control

Equipment

Room)

(

RF

&

Dipole)


The evolution of the aps beamline personnel safety system pss

GEN1 PSS

Testing Methodology

  • 1. The Front-End Shutter (FES) Simulator is connected.

    • The Front-End Shutters are critical devices that can’t be operated during the validation. Therefore their operations and statuses must be simulated.

    • The connectors on the Front-End Relay Distribution Panel (FERDP) for the Front-End Shutters (FES), Access Control Interlock System (ACIS), Front-End Equipment Protection System (FEEPS), De-Ionized Water (DIW) and Beamline Equipment Protection System (BLEPS) systems are unplugged and the simulator connectors are plugged in their place.

    • The simulator allows the operator to manipulate and monitor the signals normally supplied by these devices and/or systems.

  • 2. An Input/Output Validation is performed.

    • Verify that each device is properly functioning and wired to the assigned input or output of the Emergency Shut Down (ESD) system’s PLC.

    • Verify chain independence.

  • 3. A Functional Test (Validation) is performed.

    • The system is monitored for proper response to inputs from the field devices and/or the simulator for each test case.

    • In order to facilitate specific test cases for critical devices, the Station User Panel (15U) connectors are unplugged, and a test box is plugged in series with the existing devices.

    • This test box is used to interrupt the signals from the field devices to simulate individual tests cases.

    • After these particular tests are done, the test box is unplugged and the connectors are plugged back into their original receptacles.

    • This process is repeated at each 15U several times during the validation.

  • 4. At the end of the Functional Validation, the FERDP simulator is unplugged and the connectors are plugged back in, and a functional test of the front end components is performed.


Gen1 pss front end relay distribution panel ferdp

GEN1 PSS Front-End Relay Distribution Panel (FERDP)

GEN1 PSS


Gen1 pss3

GEN1 PSS

Validation FES Simulator Cables

Validation FES Simulator Rear Panel


Validation fes simulator

Validation FES Simulator

GEN1 PSS


Gen1 pss4

GEN1 PSS

DOS-based GE validation laptop


The evolution of the aps beamline personnel safety system pss

GEN1 PSS

Testing Methodology Problems

  • Disconnecting the real I/O for validation is invasive.

  • Problems in reconnecting the real I/O.

    • Not reconnected in the proper location

    • Not seated fully

    • Connectors or wires broken during the process

  • Maintaining laptops that are compatible with Windows 98 & DOS.


Gen2 pss

GEN2 PSS

  • Processors:

    • Dissimilar

      • Chain A - AB PLC5, ESD

      • Chain B - GE 90-70, ESD

      • Chain C - Industrial computer utilizing Siemens I/O, Command & Control

  • Programming Languages

    • Dissimilar :

      • Chain A – Ladder Logic

      • Chain B – State Logic, Ladder Logic and MegaBasic

      • Chain C – Cimplicity soft PLC (Wonderware)

  • HMI:

    • Soft Panel displays & controls

  • I/O Interface:

    • Hardwired to terminal blocks

  • DIW Monitoring

    • LOVE process controllers, binary output, 1 channel per station

  • Validations

    • Invasive, 2-3 days

      • Front-End Simulator

      • Laptops running DOS (Windows 98)

  • Communication

    • Relay isolation for signals between chains (status, permits, and heart beat)

    • One-way communication from ESD systems to Chain C using Profibus (ESD systems write only, no read functions)

    • Profibus between Chain C and EPICS


Gen3 pss

GEN3 PSS

  • Processors:

    • Similar - AB ControlLogix L61

      • Chain A – ESD

      • Chain B – ESD

      • Chain C – Command & Control

  • Programming Languages

    • Ladder Logic

  • HMI:

    • Soft Panel displays and controls

  • I/O Interface:

    • Hardwired to Circuit Boards

    • Consistent locations

  • DIW Monitoring

    • Initially LOVE process controllers, binary output,

      1 channel per station

    • Migrated quickly to Analog modules in the ESD PLCs

  • Validations

    • 2-3 days

      • All I/O simulated (Diode injection)

      • Touch Panel HMI

  • Communication

    • Electronic isolation for signals between chains (status, permits, and heart beat)

    • One-way communication from ESD systems to Chain C using Profibus (ESD systems write only, no read functions)

    • Profibus between Chain C and EPICS


Gen3 pss1

GEN3 PSS

Station B Door Control

E

P

I

C

S

HMI &

Non Critical I/O

Chain A

Emergency Shutdown (ESD) PLC

Photon

Shutter 2,

Safety

Shutter 1,

Safety

Shutter 2

data

WDT, Cross Trip, Search Status, etc

Chain-C

Command & Control (C&C) PLC

Station A

Safety I/O

Station B

Safety I/O

ACIS –PSS

Interface

HMI

HMI

data

Beam Line

Station A Door Control

Chain B

Emergency Shutdown (ESD) PLC

Integral

Shutters

Global On/Off Line

Air Supply

FES limit switches are not shown but are interfaced the same as GEN1 PSS with ACIS connected directly to FES closed limit switches. ACIS does not monitor PS1 closed limit switches.


Gen3 pss2

GEN3 PSS

Beamline PSS interface diagram

Front

-

End

Storage Ring

components

MPS

DIW flow sensors

Beamline

Beamline

Beamline

components

vacuum

valves

FE shutter & valve status

DIW flow

Front

-

End vacuum

sensors

position

sensors

sensors

FE shutter & vacuum permit

FEEPS

FEEPS

PSS EPICS

IOC - RSI &

Clock Sync

(

Mezzanine Rack)

EPICS

Front

-

End vacuum

IOC

Beamline component

valves position

DIW flow sensors

Beamline EPS

(

SV

,

FV

,

FEV

,

BIV

)

PS1 Open request

FE shutter permit

Analog PLC modules

direct to transducers (8)

BL Shutter

BL Shutter

status

permits

FES Open status

PSS PLC

Front

-

End Control System

(

Mezzanine Rack)

Shutter air supply

Pressure sensors

Chain A only

User Panels

Beamline Shutters

(1,2 or 3)

(

HMI’s and key

Switches)

Closed switches

FE shutter permit

PSS SR trips

PSS Trip

Test

Solenoid

Control

GOL key status

PSS

MEZZIE

Beamline Control System

Remote IO link

(

Station Enclosure)

Open switches

Solenoid control

Closed switches

(except PS1)

Hutch doors

ACIS Sector Interface

Front

-

End Shutters

Enclosure

Air supply shutoff

(except PS1)

Search

stations

Remote I/O link

E

-

stop

buttons

FES closed status

ACIS

ACIS PLC

ACIS

Controlled

Up to 8 stations

IOC

(

Main Control

Equipment

Room)

(

RF

&

Dipole)


The evolution of the aps beamline personnel safety system pss

GEN3 PSS

Testing Methodology

  • 1. A testing system is connected in parallel.

    • The Front-End Shutters are critical devices that can‘t be operated during the validation.

    • A HMI & PLC are connected to all 3 PLCs via Ethernet.

    • When the Test Cover is lifted the Front-End Shutters solenoid outputs and ACIS SR Permit are disconnected through relays.

    • The HMI system will allow the operator to monitor signals coming from the PSS.

  • 2. An I/O Validation is performed.

    • This will still be done to verify that each device is properly functioning and wired to the assigned input or output of the Emergency Shut Down (ESD) system’s PLC.

    • Verify chain independence.

  • 3. A Functional Test (Validation) is performed.

    • The Validation injection hardware is connected with no field device disconnections.

    • The power supplies are disconnected from the field devices via electro-mechanical relays via the Validation system.

    • The Validation system will allow the operator to manipulate all inputs going to the PSS.

    • The ESD system can be monitored for proper response to inputs.

    • All test case scenarios can be simulated.

  • 4. At the end of the Functional Test, the Validation system is disconnected.

  • 5. The field devices, inputs and outputs are re-enabled with none of the PSS wiring being disturbed.

  • 6. The method of disabling and re-enabling the PSS inputs and outputs is failsafe.


The evolution of the aps beamline personnel safety system pss

GEN3 PSS

Testing Methodology – Key Changes

  • Once the Validation system is connected, there is no unplugging and plugging in of connectors during the functional test.

  • The I/O Validation uses a touch panel HMI instead of laptop computers.

  • All inputs are disabled via relays instead of being unplugged.

  • All functional testing is done via injection of test signals. This allows for a full functional test of the system to be performed.

  • Upon completion of the functional test, the critical devices are re-enabled instead of reconnected.


The evolution of the aps beamline personnel safety system pss

GEN3 PSS

Testing Methodology

Normal Operation-Validation System not connected

Door Switch

PSS

D1

Shutter Closed

Limit Switch

D2

2

Shutter Open

Limit Switch

D3

1

+24V

CR1

Connections for the Validation System

  • Power is supplied to the PSS field device inputs via CR1

  • Signals from the field devices pass through the series diodes


The evolution of the aps beamline personnel safety system pss

GEN3 PSS

Testing Methodology -

Validation System connected

Door Switch

PSS

Shutter Closed

Limit Switch

Shutter Open

Limit Switch

1

+24V

2

CR1

Bus Monitor

Disable power to field devices

Test Mode Enabled

Injected

Test Signals

Removable Test Connector

1 Power is still supplied to the PSS field device inputs via CR1

2 Chain C verifies Validation System is connected to turn OFF its outputs


The evolution of the aps beamline personnel safety system pss

GEN3 PSS

Testing Methodology

Validation System connected

Door Switch

PSS

Shutter Closed

Limit Switch

Shutter Open

Limit Switch

1

+24V

CR1

Bus Monitor

Inject

Test Signals

Disable power to field devices

Test Mode Enabled

2

3

Removable Test Connector

1 Power is removed from the PSS field device inputs via CR1

2 The Validation System verifies field device power is disabled

3 Test signals can now be injected for functional testing


The evolution of the aps beamline personnel safety system pss

GEN3 PSS

Testing Methodology - Disabling of FES

Validation System connected

3

2

1

  • Front-End Shutter (FES) operations and Storage Ring Permit enabled via 4-pole relays

  • All systems can verify FES is disabled

  • 3 Outputs are monitored for proper operation


Gen3 pss3

GEN3 PSS


The evolution of the aps beamline personnel safety system pss

GEN3 PSS

Validation System Main menu allows for configuration of beamline


Gen3 pss4

GEN3 PSS

Front-End Shutter Functional Validation screen


Gen3 pss5

GEN3 PSS

Station Functional Validation screen


The evolution of the aps beamline personnel safety system pss

GEN3 PSS

Testing Methodology Problems

1. The connectors used in the Validation system are Multi Pin (~120) and are very fragile.

2. Real equipment is only exercised during the I/O validation.


The gen1 mini upgrade

The GEN1 Mini-Upgrade

GEN1 PSS with the following important but quick and cheap fixes

  • Hardware Changes

    • Changed the DIW monitoring to analog modules in the ESD PLCs (default set points are loaded at power up and allow adjust via HMI thereafter).

    • Added Chain B Fault Lights to the Panel.

  • Software Changes

    • Modified Chain B code to require a reset before re-granting Storage Ring Permit .

    • Modified Chain B code to indicate a flow fault condition by blinking the new Chain B Minor Fault LED.

    • Modified Chain B code so that any remote block to PLC communication loss will latch a fault, remove all shutter permits, and indicate a fault via blinking Chain B Fault LED.

    • Modified Chain A code so that the Storage Ring Permit was not a latched signal.

    • Modified Chain A code to lose station "Searched" status when it sees a Crash button pressed.

    • Modified Chain A code to generate a “Chain B Shutter Permit Lost" minor fault when a shutter is open and the corresponding Chain B shutter permit goes low.

    • Modified Chain A code to add a “quick search” when a signal, only available when the FES simulator is connected to the FERDP during validations, is asserted.

    • Modified Chain A code to lose search status for all stations any time a remote I/O communications fault exists.

    • Modified both Chains A & B to add a DIW bypass signal which is only available when the FES simulator is connected to the FERDP during validations, is asserted.

    • Modified both Chains A & B to remove all Shutter permits when the Storage Ring Permit is removed.


The gen1 upgrade

The GEN1 Upgrade

  • Processors:

    • Redundant and Dissimilar:

      • Chain A - AB PLC5 ESD and C&C in Separate routines

      • Chain B - GE RX7i, ESD

  • Programming Languages

    • Similar :

      • Chain A – Ladder Logic

      • Chain B – Ladder Logic and C++

  • HMI:

    • LEDs and mechanical PBs

  • I/O Interface:

    • Hardwired to terminal blocks

    • Software mapping of all I/O for standardization

  • DIW Monitoring

    • Analog modules in the ESD PLCs

  • Acceptance testing

    • Done in Lab (when changes are made or every 5 years) using a Wonderware-based software simulator

  • Validations

    • Annual, Noninvasive, 1/2 day

      • Touch Panel HMI Simulating the Front-End Shutters

  • Communication

    • Relay isolation for signals between chains (status, permits, and heart beat)

    • One-way communication to EPICS

      • Chain A – DH+ through a Data Communications Module (PLC writes only, no read functions)

      • Chain B – RS232 through a Processor Control Module (PLC writes only, no read functions)


The gen1 upgrade1

The GEN1 Upgrade

  • Software Changes

  • A thorough and concise specification document

  • Each PLC chain separates logic routines - functionally

    • Input and Output map(s)

    • ESD – immediate radiation hazard - logic

      • Storage Ring Permit removing faults and trips

    • ESD – potential radiation hazard - logic

      • Front-End Shutter Permit removing faults and trips

      • Beamline Shutter Permit removing faults and trips

    • Command and Control logic (Chain A only)

    • Status logic

    • Warning logic

  • Major, Serious, and Minor Faults were changed to Faults, Trips, and Warnings with much clearer diagnostics for troubleshooting

  • ESD Permits driven by real equipment status rather than resulting fault codes

  • Standardized station addressing (100s = Station A, 200s = Station B)

  • Chain B reports the faults and trips

  • Chain B looks at Search status from Chain A at the leading edge of the searched signal

  • Doors locked when the search is complete, eliminating many Lock and Unlock buttons.

  • Each station now has Beam Ready, Fault/Trip, and No Access Allowed indicators.


Gen 1 upgrade pss

GEN 1 Upgrade PSS

Open status

  • Hardware Changes

Open status

Closed status

Open status

Chain A

Emergency Shutdown and Command and Control PLC

Safety

Shutter 1

Safety

Shutter 2

Photon

Shutter 2

Closed status

SR Trip/Fault

Beam Line

Door closed

Estop

WDT

Integral

Shutters

Closed status

Station A

ACIS

Station A Searched

Station A Beam Ready

Crosstrip

Crosstrip

Global OnlineSolenoid

WDT

Air Supply

Door closed

Estop

Closed status

SR Trip/Fault

Closed status

Photon

Shutter 1

Open status

Chain B

Emergency Shutdown PLC

Closed status

Open status

ACIS now monitors PS1 as a backup for PS2

Chain B PS1 Permit added

Search Status from A to B is now leading edge trigger

Cross-trips were added

Open status


Gen1 upgrade pss

GEN1 Upgrade PSS

Beamline PSS interface diagram

Front

-

End

Storage Ring

components

MPS

DIW flow sensors

Beamline

Beamline

Beamline

components

vacuum

valves

FE shutter & valve status

DIW flow

Front

-

End vacuum

sensors

position

sensors

sensors

FE shutter & vacuum permit

FEEPS

FEEPS

(

Mezzanine Rack)

PSS

EPICS

EPICS

Front

-

End vacuum

IOC

Beamline component

IOC

valves position

DIW flow sensors

Beamline EPS

(

SV

,

FV

,

FEV

,

BIV

)

PS1 Open request

FE shutter permit

EPICS clock sync

Remote

Shutter

Analog PLC modules

direct to transducers (8)

Interface

BL Shutter

BL Shutter

status

permits

FES Open status

PSS PLC

Front

-

End Control System

(

Mezzanine Rack)

Shutter air supply

Pressure sensors

Chain A only

User Panels

Beamline Shutters

(1,2 or 3)

(

Pushbuttons, key

switches and LED

Indicators)

Closed switches

Solenoid

FE shutter permit

PSS SR trips

control

GOL key status

PSS

Beamline Control System

Remote IO link

(

Station Enclosure)

Open switches

Solenoid control

Hutch doors

ACIS Sector Interface

ALL Closed switches

Front

-

End Shutters

Enclosure

Air supply shutoff

(except PS1)

Search

stations

Remote I/O link

E

-

stop

buttons

FES closed status

ACIS

ACIS PLC

ACIS

Controlled

Up to 6 stations

IOC

(

Main Control

Equipment

Room)

(

RF

&

Dipole)


Gen1 upgrade pss1

GEN1 Upgrade PSS

Testing Methodology

  • Internal memory tables were created to map all beamline I/O to a common addressing scheme (I/O wiring is not consistent across all beamlines).

    • This allows the Acceptance and Validation Systems to have a single program for all beamlines.

    • This allows the Acceptance and Validation Systems to control where the inputs to the ESD logic originate (i.e., from either real world devices or software signals).

    • This allows the Acceptance and Validation Systems to simulate the Front-End Shutters and other critical signals.


Gen1 upgrade pss2

GEN1 Upgrade PSS

Testing Methodology – The ATS

  • Before PLC code is downloaded for testing at a beamline, testing is performed in a lab simulator called the Acceptance Test System (ATS)

  • The ATS procedures tests ALL of the PLC logic

    • ESD – immediate radiation hazard - logic (each fault and trip tested n+1 times)

      • Storage Ring Permit removing faults and trips

    • ESD – potential radiation hazard - logic (each fault and trip tested n+1 times)

      • Front-End Shutter Permit removing faults and trips

      • Beamline Shutter Permit removing faults and trips

    • Command and Control logic

    • Warning logic


Gen1 upgrade pss3

GEN1 Upgrade PSS

Acceptance Test System


Gen1 upgrade pss4

GEN1 Upgrade PSS

Testing Methodology – The VTS

  • I/O validation is performed first to verify that all beamline input and output devices operate correctly and are mapped properly to internal memory tables

  • The VTS procedures perform testing for ONLY the following PLC logic

    • ESD – immediate and potential radiation hazard - logic

      • Storage Ring Permit removing faults and trips

      • Shutter Permit removing faults and trips

  • After a Validation is complete, the Validation System is disconnected from the PSS and the PSS returns to normal with no residual data (created by the Validation System) remaining in the data tables of the PLCs.

  • It is of utmost importance to ensure the real inputs are mapped to the internal files/bits and no faults or trips are blocked after the Validation System is removed. This is accomplished by the following method:

    • A pulse, originating from the Validation System, must be present to activate any of these Validation System functions in the PLCs.

    • Perform End-to-End Test (PSS input through ACIS output to dump beam).


Gen1 upgrade pss5

GEN1 Upgrade PSS

Validation Test System


Gen1 upgrade pss6

GEN1 Upgrade PSS

Validation System Main menu allows for configuration of beamline


Gen1 upgrade pss7

GEN1 Upgrade PSS

Functional Validation screen manipulates FES switches to create trips and faults


Gen1 upgrade pss8

GEN1 Upgrade PSS

Main Functional Validation screen contains mezzanine based signals and allows Validator to monitor SR Permit status


Gen3 upgrade pss

GEN3 Upgrade PSS

  • Processors:

    • Similar - AB ControlLogix L61

      • Chain A – ESD

      • Chain B – ESD

      • Chain C – Command & Control

  • Programming Languages:

    • Ladder Logic

  • HMI:

    • Soft Panel displays and controls

  • I/O Interface:

    • Hardwired to Circuit Boards

    • Software mapping of all I/O

    • Diagnostic Modules on Front-End Shutter I/O

  • DIW Monitoring

    • Analog modules in the ESD PLCs

  • Acceptance Testing

    • Done in Lab (when changes are made or every 5 years) using a Wonderware-based software simulator

  • Validations

    • Annual, Noninvasive, 1/2 day

      • Touch Panel HMI simulating the Front-End Shutters

  • Communication

    • Electronic isolation for signals between chains (status, permits, and heart beat)

    • One-way communication from ESD systems to Chain C using Produced and Consumed Tags over Ethernet

    • Ethernet between EPICS and Chain C using Produced and Consumed Tags


Aps pss comparison

APS PSS Comparison


  • Login