Oxford university particle physics site report
This presentation is the property of its rightful owner.
Sponsored Links
1 / 23

Oxford University Particle Physics Site Report PowerPoint PPT Presentation


  • 81 Views
  • Uploaded on
  • Presentation posted in: General

Oxford University Particle Physics Site Report. Pete Gronbech Systems Manager. Central Physics Computing Services. E-Mail hubs In last year 2.7M messages were relayed (78GB), 0.8M from Physics systems. A further, 2.8M rejected as spam. Last month 345K rejected, 208K delivered.

Download Presentation

Oxford University Particle Physics Site Report

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Oxford university particle physics site report

Oxford University Particle Physics Site Report

Pete Gronbech

Systems Manager

Hepix Vancouver - Oxford Site Report


Oxford university particle physics site report

Hepix Vancouver - Oxford Site Report


Central physics computing services

Central Physics Computing Services

  • E-Mail hubs

    • In last year 2.7M messages were relayed (78GB), 0.8M from Physics systems. A further, 2.8M rejected as spam. Last month 345K rejected, 208K delivered.

    • Anti-virus and anti-spam measures increasingly important in email hubs. Some spam inevitably leaks through and clients need to deal with this in a more intelligent way.

  • Windows Terminal Servers

    • Use is still increasing.

    • Retired NT4 based service, Now Win2k and 2003.

    • Introduced an 8 CPU server (TermservMP) . Much more powerful system but still awaiting updated versions of some applications which will run properly on OS.

  • Web / Database

    • New web server (Windows 2003) in service. Some initial problems with migrating the permissions from the old service.

    • New web applications for lecture lists, Computer inventory

    • Other databases for admissions and finals

  • Exchange Servers

    • Running two new servers using Exchange 2003 running on Windows server 2003. Default for new accounts. Much better Web interface, support for mobile devices and for tunnelling through firewalls. Existing mailboxes will be migrated soon.

Hepix Vancouver - Oxford Site Report


Oxford university particle physics site report

Hepix Vancouver - Oxford Site Report


Particle physics strategy the server desktop divide

Servers

Group DAQ Systems

Windows FileServer

General Purpose Unix Server

Mail Server

Web Server

Win 2K PC

Win 2K PC

Win XP PC

Linux System

Win 2K PC

Desktops

Particle Physics Strategy The Server / Desktop Divide

Approx 200 Windows 2000 Desktop PC’s with Exceed used to access central Linux systems

Hepix Vancouver - Oxford Site Report


Windows status

Windows Status

  • Migration to Windows 2000 domain nearly complete for PP users and their computers.

  • Windows XP pro is default OS for new desktops and laptops.

  • We now have to expect routine reboots of desktops to apply security patches. Give notice whenever possible.

Hepix Vancouver - Oxford Site Report


Migration to linux

Migration to Linux

  • Central Unix systems are Linux based

    • Red Hat Linux 7.3 is becoming the standard

    • Treat Linux as just another Unix and hence a server OS to be managed centrally.

    • Wish to avoid badly managed desktop PC’s running Linux.

  • Linux based file server (April 2002)

  • Digital Unix and VMS services were closed in August 2002

  • General purpose Linux server installed August 2002

  • Small batch farm installed Feb 2003

Hepix Vancouver - Oxford Site Report


Oxford university particle physics site report

CDF

7.3.1

7.3.1

7.3.1

7.3.1

7.3.1

7.3.1

matrix

Autumn 2002

RH7.3

RH7.3

RH7.3

RH7.3

RH7.3

RH7.3

RH7.3

RH7.3

4*Dual 2.4GHz systems

4*Dual 2.4GHz systems

Autumn 2003

PBS Batch Farm

General Purpose Systems

Fermi7.3.1

Fermi7.3.1

RH7.3

RH7.3

RH7.3

1Gb/s

pplx2

pplx1

morpheus

pplxfs1

pplxgen

minos DAQ

RH7.3

RH7.3

RH6.2

RH7.1

ppminos1

ppminos2

pplx3

(SNO)

ppnt117

(HARP)

cresst DAQ

RH7.3

RH7.3

Grid Development

ppcresst1

ppcresst2

RH7.3

RH7.3

RH7.3

RH6.2

RH7.3

RH6.2

RH7.3

Atlas DAQ

RH7.3

RH7.3

pptb01

tblcfg

se

ce

grid

tbwn01

pptb02

Hepix Vancouver - Oxford Site Report

ppatlas1

atlassbc


Oxford university particle physics site report

Early Linux Systems

pplx1P4 Xeon 2.4GHz2GB RAM(1998)

pplx2Dual 450MHz Pentium II1024MB RAM(1999)

pplx3Dual 800MHz Pentium II512MB RAM(2000)

CDF group systemruns Fermi 7.3.1

General Purpose Linux

SNO group systemruns Red Hat 6.2


Oxford university particle physics site report

The new (April 2002) Linux File Server: pplxfs1

8*146GB SCSI disks

Dual 1GHz PIII, 1GB RAM

Hepix Vancouver - Oxford Site Report


Oxford university particle physics site report

General Purpose Linux Server : pplxgen

pplxgen is a Dual 2.2GHz Pentium 4 Xeon based system with 2GB ram. It is running Red Hat 7.3

It was brought on line at the end of August 2002 to share the load with pplx2 as users migrated off al1 (the Digital Unix Server)

Hepix Vancouver - Oxford Site Report


Oxford university particle physics site report

PP batch farm running Red Hat 7.3 with Open PBS can be seen below pplxgen

This service became fully operational in Feb 2003. Additional 4 worker nodes to be installed this month. (October 2003)

Hepix Vancouver - Oxford Site Report


Oxford university particle physics site report

http://www-pnp.physics.ox.ac.uk/ganglia-webfrontend-2.5.4/

Power Cut

Hepix Vancouver - Oxford Site Report


Oxford university particle physics site report

CDF Linux Systems

Morpheus

is an IBM x370

8 way SMP 700MHz Xeon

with 8GB RAM and

1TB Fibre Channel disks

Installed August 2001

Purchased as part of a JIF grant

for the CDF group

Runs Fermi Red Hat 7.3.1

Will use CDF software developed at

Fermilab and here to process data

from the CDF experiment.

Hepix Vancouver - Oxford Site Report


Oxford university particle physics site report

Second round of CDF JIF tender: Dell Cluster - MATRIX

10 Dual 2.4GHz P4 Xeon servers running Fermi Linux 7.3.1 and SCALI cluster software. Installed December 2002


Oxford university particle physics site report

CDF Linux Systems - MATRIX

Approx 7.5 TB for SCSI RAID 5 disks

are attached to the master node.

Each shelf holds 14 * 146GB disks.

These are shared via NFS with the worker nodes.

OpenPBS batch queuing software is used.

Hepix Vancouver - Oxford Site Report


Oxford university particle physics site report

Plenty of space in the second rack for expansion of the cluster.

Additional Disk Shelf with 14*146GB plus two extra nodes will shortly be ordered. (Autumn 2003)

Hepix Vancouver - Oxford Site Report


Oxford university particle physics site report

Grid development systems. EDG Test bed setup, currently 2.0.3

Hepix Vancouver - Oxford Site Report


Oxford university particle physics site report

Tape Backup is provided by

a Qualstar TLS4480

tape robot with 80 slots and Dual Sony AIT3 drives.

Each tape can hold 100GB

of data.

Installed Jan 2002.

Netvault Software from BakBoneis used, running on morpheus, for

backup of both cdf and particle

physics systems.

Main userdisks backed up every

weekday night data disks not generally

backed up BUT weekly backups to

OUCS HFS service provide

some security.

Hepix Vancouver - Oxford Site Report


Network access

Network Access

Super Janet 4

2.4Gb/s with Super Janet 4

Physics

Backbone

Router

100Mb/s

Physics

Firewall

OUCS

Firewall

100Mb/s

1Gb/s

Backbone

Edge

Router

1Gb/s

100Mb/s

Campus

Backbone

Router

100Mb/s

1Gb/s

depts

Backbone

Edge

Router

depts

100Mb/s

depts

100Mb/s

depts


Physics backbone upgrade to gigabit autumn 2002

Physics Backbone Upgrade to Gigabit Autumn 2002

Linux

Server

1Gb/s

Physics

Firewall

Server

switch

1Gb/s

Win 2k

Server

1Gb/s

100Mb/s

Particle

Physics

1Gb/s

100Mb/s

Physics

Backbone

Router

100Mb/s

1Gb/s

desktop

Clarendon

Lab

100Mb/s

1Gb/s

desktop

1Gb/s

1Gb/s

100Mb/s

Astro

Atmos

Theory


Network security

Network Security

  • Constantly under threat from worms and viruses. Boundary Firewall’s don’t solve the problem entirely as people bring infections in on laptops.

  • New firewall based on stateful inspection. Policy is now `default closed`. Some teething problems as we learnt what protocols were required but there has been a very significant improvement in security.

  • Main firewall passes average 5.8GB/hour (link saturates at peak). Rejects 26,000 connection per hour (7 per second). Mischievous connects rejected 1500/hour, one every 2.5 secs. During blaster worm this reached 80/sec.

  • Additional firewalls installed to protect the Atlas construction area and to protect us from attacks via dialup or VPN.

  • Need better control over how laptops access our network. Migrating to a new Network Address Translation system so all portables connect through a managed `gateway`.

  • Have made it easier to keep Anti-Virus software uptodate via simply connecting to a web page. Important that everyone managing their own machines takes advantage of this. Very useful for both laptops and home systems

    (see http://www.physics.ox.ac.uk/sophos)

  • Keeping OS’s patched is a major challenge. Easier when machines are all inside one management domain but is still very time consuming. Must compare to perhaps 1-few man months of IT support staff effort to clean out a successful worm from the network.

Hepix Vancouver - Oxford Site Report


Goals for 2003 4 computing

Goals for 2003/4 (Computing)

  • Continue to improve Network security

    • Need better tools for OS patch management

    • Need users to help with their private laptops

      • Use automatic updates (e.g. Windows Update)

      • Update Antivirus software regularly

    • Segment the network by levels of trust

    • All the above without adding an enormous management overhead !

  • Reduce number of OS’s

    • Remove last NT4 machines and exchange 5.5

    • Digital Unix and VMS very nearly gone.

    • Getting closer to standardising on RH 7.3 especially as the EDG software is now heading that way.

  • Still finding it very hard to support laptops but now have a standard clone and recommend IBM laptops.

  • What version of Linux to run ? Currently almost all 7.3 but Red Hat’s proposal to have limited support & hiving off free releases to fedora project will become a problem.

  • Looking into Single Sign On for PP systems

Hepix Vancouver - Oxford Site Report


  • Login