TACCT 330 – UNIX OVERVIEW OF A UNIX SYSTEM - PowerPoint PPT Presentation

Tacct 330 unix overview of a unix system l.jpg
Download
1 / 95

TACCT 330 – UNIX OVERVIEW OF A UNIX SYSTEM. UNIX Operating System - core concepts. Unit 1 Introduction to UNIX. What Is UNIX?.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

TACCT 330 – UNIX OVERVIEW OF A UNIX SYSTEM

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Tacct 330 unix overview of a unix system l.jpg

TACCT 330 – UNIXOVERVIEW OF A UNIX SYSTEM

UNIX Operating System - core concepts


Unit 1 introduction to unix l.jpg

Unit 1Introduction to UNIX


What is unix l.jpg

What Is UNIX?

  • Textbook Definition: “A time sharing operating systems kernel: a program that controls the resources of a computer and allocates them among its users…[and] also [its] compilers, editors, command languages, programs… and so on” (Kernighan & Pike).

  • “An Operating System designed by Programmers for Programming” (Ken Thompson).

  • UNIX Motto: “Live Free or Die!!”

    • Open systems concept

    • Customizable environments

  • UNIX Philosophy: “Leave me alone… I know what I’m doing!!”

    • Minimal user interface

    • Minimal dialog with system

    • Cryptic and short commands

    • Less than friendly to beginning users

  • UNIX Caveat: “You are given enough rope to hang yourself.”

Stanberry

U1-1


Why unix l.jpg

Why UNIX?

  • Portability: Open Systems Concept

  • Scalability: Mainframe to PC

  • Flexibility: Customizing, source code available

  • Compatibility: Standards

  • Security: Built-in safeguards

  • Performance: Robustness

  • Availability: Fault-tolerance

  • Multiplicity: Multi-user and multi-threaded

  • Efficiency: Minimal overhead

  • Modularity: Small programs accomplish complex tasks

  • Cost: Minimal, vendor independence

Stanberry

U1-2


Comparison of current operating systems network operating systems l.jpg

Comparison of Current Operating Systems/Network Operating Systems

Stanberry

U1-3


Unix history l.jpg

UNIX History

  • Bell Laboratories in the late 1960’s and early 1970’s

    • Ken Thompson and Dennis Ritchie worked on the MULTICS project that was canceled. They ported pieces of the operating system to a smaller PDP-7 machine. In a play on words, they called the scaled down version UNICS which later became UNIX.

    • In 1970, UNIX was ported to a PDP-11 to support the Bell Labs patent office.

    • In 1973, the language was re-written in C language (developed by Dennis Ritchie) to facilitate porting in the future.

  • UNIX in the mid 1970’s to early 1980’s

    • UNIX becomes very popular “for educational purposes” in the academic environment.

    • In 1977, the University of California at Berkeley releases the early versions of the Berkeley Software Distribution (BSD) UNIX which eventually becomes an independent form of the operating system.

    • In 1983, Bell Labs release System V which becomes the modern incarnation of the original UNIX system.

  • UNIX Today

    • UNIX systems today are a combination of the System V and BSD versions of UNIX that attempt to be cohesive through the adoption of standards.

Stanberry

U1-4


Flavors of unix l.jpg

Flavors of UNIX

  • SVR4: SunOS (which is largely based on BSD UNIX) combines with Bell Labs (now AT&T) System V to provide the first hybrid system

  • SunOS/Solaris: Sun Microsystems

  • AIX: IBM

  • HP-UX: Hewlett-Packard

  • FreeBSD: Free, Internet

  • Linux: Free, PC platform

  • ULTRIX: Digital

  • XENIX: Microsoft

  • OS/390 Open Edition: IBM, Mainframe

Stanberry

U1-5


Unit 2 unix concepts l.jpg

Unit 2UNIX Concepts


Open systems concept l.jpg

Open Systems Concept

Open Systems: Through the use of standards, applications

can interoperate and port data across a wide variety of

hardware and software platforms.

  • Operational Goals

    • Portability across hardware

    • Device-independent file system

    • Tasks performed on distributed machines (client/server).

    • Efficient balancing of multiple users and tasks

  • Business Objectives

    • Maximize investments (scalability and portability)

    • Vendor independence

    • Standardization (re-work costs, training)

    • Support long-term life of applications

    • Maintain flexibility to respond to business needs

Stanberry

U2-1


Major components of the unix operating system l.jpg

Major Components of the UNIX Operating System

  • Kernel

  • Applications

  • File System

  • Communications/Networking

  • Shell

  • Programming Environment

  • Utilities

  • Text Processing

  • Documentation

  • System Management

Stanberry

U2-2


The kernel l.jpg

The Kernel

  • Definition: The central core of the operating system.

    • It is loaded into memory at boot time, although may be dynamically linkable.

    • It manages users, processes and security.

    • It controls I/O access to peripheral devices.

    • It is mostly written in the C language to facilitate portability.

Applications

Shell

Kernel

Hardware

Stanberry

U2-3


The unix file system l.jpg

The UNIX File System

  • Definition: The system of organizing files within the operating system.

    • In UNIX, every object is represented to the operating system as a file. All I/O on the UNIX system is file I/O.

    • There are only three types of files at the operating system level:

      • Ordinary - containing text.

      • Directory - containing pointers to other files.

      • Special - containing pointers to devices.

  • Communications/Networking is facilitated through the file system. Other machines and network objects are represented as files to the operating system.

Stanberry

U2-4


The shell l.jpg

The Shell

  • Definition: The interface between the user and the kernel of the operating system. It runs as a UNIX program that interprets commands and initiates user processes. Some shells facilitate programming.

  • Standard UNIX Shells

    • sh - Bourne Shell, written by Steve Bourne, default shell of original Bell Labs UNIX.

    • ksh - Korn Shell, written by David Korn, an extension of the Bourne Shell in System V UNIX that added scripting capabilities. Now it is generally the default shell in the System V descendants such as IBM’s AIX.

    • csh - C Shell, written by Bill Joy of UC Berkeley, generally the default shell of BSD descendants such as Sun Microsystems. Based largely on the C language, it most robustly supports operating system modifications.

    • bash - Bourne Again Shell, written by FSF/GNU (Free Software Foundation/GNU’s Not UNIX) founded by Richard Stallman of MIT. Used in extensively Linux and other free versions of UNIX.

  • Other UNIX shells include menu-driven and graphical interfaces that generate commands.

Stanberry

U2-5


The unix programming environment l.jpg

The UNIX Programming Environment

  • Components of the UNIX Programming Environment:

    • Shell Programming: Programs written in the shell language to process UNIX commands.

    • Programming Languages: Programs written in languages such as C, C++, Java, assembler, etc. and compiled to run within the UNIX operating system.

    • Programming Utilities: Tools in the UNIX environment that facilitate programming. Categories include:

      • Text Editors - create and modify source files in ASCII text. Standard editors are ed and vi. Other editors include INed, ISPF and GNU emacs.

      • Source Code Control - manages versions of software as programs are being developed. Also keeps programming teams from stepping on each others changes.

      • Debuggers and Code Checkers - check syntax and compilation errors and facilitate the correction of these errors.

      • On-line Documentation - manual pages document commands, system calls and other programming environment details. Some UNIX systems have graphical or menu-driven on-line help facilities such as InfoExplorer in IBM’s AIX.

Stanberry

U2-6


System management l.jpg

System Management

  • Definition: The tasks and complex procedures involved in managing and maintaining UNIX systems. Included are the following functions:

    • Software installation

    • Defining user accounts

    • Configuring peripheral devices

    • Data storage management

    • Back up and archival of data

    • Performance and system monitoring

    • Problem determination and resolution

  • On many UNIX systems, software packages and tools are added on top of the shell to support these functions. These can be operating system specific such as SMIT and Logical Volume Manager (LVM) that are used with IBM’s AIX, or multi-vendor products such as Veritas Volume Manager, Control-M and ReelBackup.

Stanberry

U2-7


Unix programming just for the true techies l.jpg

UNIX PROGRAMMING(JUST FOR THE TRUE TECHIES)


Logging in and out l.jpg

Logging In and Out

  • Login: All users must provide a user name and a password to log into the system. Below is a dialogue:

    login: ds59478

    ds59478’s Password: (password does not echo)

    $

  • In this case, the login has been demonstrated with my user name of ds59478. The dollar sign prompt represents the shell (default of ksh in AIX).

  • Logout: The logout command can be invoked with any of the following commands:

    $ <ctrl-d> (press ctrl key and letter d together)

    $ exit

    $ logout

    login:

  • The logout commands will remove the user from their current shell. If the current shell is the only session, the result will be a new login prompt as above.

Stanberry

L1-1


Invoking the shell and the passwd command l.jpg

Invoking the Shell and the passwd command

  • Type the name of the shell to invoke a new shell. The C shell appears as a % at the login prompt by default. Below is a dialogue:

    $ ksh

    $ csh

    % <ctrl-d>

    $ exit

    $ logout

    login:

  • To create or change the password for the account, use the passwd command. Below is an example:

    $ passwd

    Changing password for ds59478

    ds59478’s Old password: (no echo to terminal)

    ds59478’s New password: (no echo to terminal)

    Enter the new password again: (no echo to terminal)

    $

Stanberry

L1-2


Who else is on the system l.jpg

Who Else is on the System?

  • With the who command, you can see who is on the system and some information about each user:

    $ who

    ds59478 pts/1 Dec 26 14:39 a209045 pts/9 Nov 17 10:27

    $

  • The output of the who command is the username, terminal information, their login date/time and the network address of the login point.

  • The pipe operator (c1 | c2) takes the output of one command (c1) and uses it as the input of another command (c2). The command wc counts the number of lines, words and bytes in from the input. The -l option on wc gives just the number of lines. Therefore, we can combine the who and wc (with -l option) programs with a pipe to get the number of users on the system:

    $ who | wc -l

    2

    $

  • We will cover the pipe as well as other operators in more detail in later sections.

Stanberry

L1-3


Date and time functions l.jpg

Date and Time Functions

  • With the date command, the shell returns the current date and time as below:

    $ date

    Sun Dec 26 15:05:40 EST 1999

    $

  • Note that there is a time command, but it performs a totally different function. The time command returns the amount of CPU time that it takes to execute a command. As you can see below, the CPU time to execute the command is negligible:

    $ time

    real 0m0.00s

    user 0m0.00s

    sys 0m0.00s

    $

Stanberry

L1-4


Month and year functions l.jpg

Month and Year Functions

  • With the cal command, the shell returns the specified month or year:

    $ cal 1 2000

    January 2000

    Sun Mon Tue Wed Thu Fri Sat

    1

    2 3 4 5 6 7 8

    9 10 11 12 13 14 15

    16 17 18 19 20 21 22

    23 24 25 26 27 28 29

    30 31

    $

  • To get the output for an entire year, leave out the month parameter:

    $ cal 2000

    <Try it to get the output>

  • Note: Try the cal command for September 1752.

  • Can you explain the output?

Stanberry

L1-5


Other simple unix commands l.jpg

Other Simple UNIX Commands

  • With the clear command, the user clears the screen:

    $ clear

  • The echo command writes to the screen:

    $ echo Out to Lunch

    Out to Lunch

    $

  • The banner command writes to the screen in large letters:

    $ banner Out to Lunch

    #######

    # # # # #####

    # # # # #

    # # # # #

    # # # # #

    # # # # #

    ####### #### #

    ##### ####

    # # #

    # # #

    # # #

    # # #

    # ####

    #

    # # # # # #### # #

    # # # ## # # # # #

    # # # # # # # ######

    # # # # # # # # #

    # # # # ## # # # #

    ####### #### # # #### # #

    $

Stanberry

L1-6


Editing ordinary files the vi editor l.jpg

Editing Ordinary Files – The vi editor

  • Although there are a variety of editors available on UNIX systems, it is important to learn vi for the following reasons:

    • It is a standard editor available on all UNIX systems

    • It is the default editor for many programs

    • It is the only editor available in some situations

  • The vi editor is an extremely robust program that is rich in features, options and customizability. In this course, we will cover only the most basic features of vi to build a working knowledge from which we can edit files.

  • The vi editor works in two modes: command and edit. Thecommand mode is the default when vi is initiated as follows:

    $ vi file1

  • Once vi is initiated, the following keystrokes will transfer to edit mode:

    • i – inserts text at the cursor

    • I – inserts text at the start of the line

    • a – appends text after the cursor

    • A – appends text after the end of the line

Stanberry

L1-7


Command mode the vi editor l.jpg

Command Mode – The vi editor

  • To transfer from editing mode to the command mode, use the escape (<esc>) key. Once in command mode, keys control navigation as follows:

    • h,b,0  left, beginning of word, beginning of line

    • j,G  down, end of document

    • k,1G  up, beginning of document

    • l,w,$  right, end of word, end of line

  • Text editing can be done with the following commands:

    • r – replace letter with a new letter

    • cw – change word (replace word with new word)

    • dw – delete word

    • x – delete letter

    • . – repeat the last command (u will undo the last command)

  • Commands for the file are preceded with a colon:

    • :q! – quit out of the file without saving

    • :x,:wq, ZZ – exit and write (or save) the file

  • Searches for can be done with the /<text pattern> command:

Stanberry

L1-8


Summary the vi editor l.jpg

G

1G

w,$

b,0

h

l

j

k

Summary – The vi editor

  • Directional keys

  • Edit mode

    • i – inserts text at the cursor

    • I – inserts text at the start of the line

    • a – appends text after the cursor

    • A – appends text after the end of the line

  • Command mode

    • <esc> key places you in command mode

    • :q! – exit without saving

    • :x, :wq, ZZ – exit and save

  • To correct mistakes

    • x – delete letter

    • dw – delete word

Stanberry

L1-9


Customizing the environment editing profile l.jpg

Customizing the Environment – editing ~/.profile

  • By using the vi editor to edit UNIX files, we can learn how to customize the user environment by editing up the ~/.profile file. Follow these instructions carefully:

    • Bring up the .profile file in vi:

      $ vi ~/.profile

    • Browse the file using the arrow keys. Do not make any changes to the file at this time.

    • Press the <esc> key to initiate command mode and enter the command to exit without writing to the file:

      :q!

Stanberry

L1-10


Lab 2 the unix file system l.jpg

Lab 2The UNIX File System


Unix files l.jpg

UNIX Files

  • Definition: All data on UNIX systems is structured as files. Ordinary files are simply a collection of bytes with no structure imposed by the operating system.

    • Ordinary files are determined to be in structures such as flat text, arguments, source code or binary executable by the UNIX programs (including the shell) that read them rather than by the operating system itself. This is different from other operating systems such as MVS or Windows.

    • All files are considered ordinary except for two types:

      • Directories are special files that contain pointers to the filenames of the files contained in the directory. Directory files are represented with a special character (/).

      • Special files are used to represent peripheral devices such as storage devices, printers and terminals.

    • All files contain a special descriptor called an inode. Each inode contains the following information about the file:

      • The userid of the owner of the file

      • The groupid of the group of the file

      • The size of the file (in blocks)

      • The permissions of the file

      • The date and time of the last modification to the file

      • The link count

    • All directories contain pointers to the filename and the inode number of each file.

Stanberry

L2-1


Directory navigation cd and pwd commands l.jpg

Directory Navigation - cd and pwd commands

  • To change the current directory, use the cd command:

    $ cd /home/ds59478

    $

  • To print the current directory, use the pwd command:

    $ pwd

    /home/ds59478

    $

  • There are different ways to use cd:

    $ cd ..(takes you to parent directory)

    $ cd -(takes you to last directory you were in)

    $ cd ~(takes you to home directory)

    $ cd /home/bob(full pathname)

    $ cd ../bob(relative pathname)

    $ cd ~bob(bob’s home directory)

Stanberry

L2-2


File system hierarchy the df command l.jpg

File System Hierarchy - the df command

  • The file system for each UNIX machine is laid out in a hierarchy that begins with the root (/) directory. To display all of the mounted filesystems (and the logical volumes to which they are mapped), use the df command:

    $ df -k(The -k option displays disk space in kb)

  • Each UNIX machine has a different file system hierarchy, but below is an example of a typical AIX structure:

/

/usr

/dev

/home

/sbin

/etc

/var

/tmp

/nfs

...

/home/user1

/home/user2

...

.profile

file1

...

/home/user1/bin

file2

prog1

...

Stanberry

L2-3


Creating and removing directories mkdir and rmdir l.jpg

Creating and Removing Directories - mkdir and rmdir

  • To create a new directory, use the mkdir command:

    $ pwd

    /u/ds59478

    $ mkdir stuff

    $ cd stuff

    $ pwd

    /u/ds59478/stuff

    $

  • To remove a directory, use the rmdir command:

    $ cd ..

    $ pwd

    /u/ds59478

    $ rmdir stuff

    $ cd stuff

    _cd[2]: stuff: not found.

    $

  • For the rmdir command to work, these must be true:

    • The directory must be empty

    • The directory must not be the current directory.

Stanberry

L2-4


Creating and removing files echo and rm l.jpg

Creating and Removing Files - echo and rm

  • There are many ways to create new files. For a simple example, the echo command can be used:

    $ mkdir stuff

    $ cd stuff

    $ echo This is a test > test

  • In this case, the output of the echo command is redirected to a new file that is called test. The (>) symbol facilitates this redirection. Using this technique, any UNIX program that produces output can create a file.

  • To remove a file, use the rm command:

    $ echo This is a test > test

    $ rm test

    $

  • Note: These are the first destructive commands that we have covered to this point. Unlike other operating systems, UNIX does not ask for confirmation before it removes the file. Also, when you create a file via redirection (>), it will overwrite a file that already exists without asking for confirmation.

Stanberry

L2-5


A word on filenames l.jpg

A Word on Filenames

  • In the spirit of UNIX, filenames have very few restrictions on them. There is a limit as to the size of the filename that is usually ridiculously large (AIX is 255 characters). In general, all characters are fair game, but it is important to follow certain guidelines to prevent troubles down the road. Generally, filenames should:

    • not contain imbedded blanks

    • generally be restricted to alphanumeric characters

    • should not include metacharacters (*?></:$![]{}| \`”)

    • be case sensitive (usually lowercase unless special)

    • only start with . if meant to be hidden

    • never begin with a + or -

    • generally be descriptive of the content

  • The reasons for these restrictions will become clearer as we learn more about UNIX and using the shell. For now, it is a good idea to remember these guidelines when creating files.

Stanberry

L2-6


Listing files and their contents ls cat and touch l.jpg

Listing Files and Their Contents - ls, cat and touch

  • The cat command prints the contents of the file to the screen. Below is an example:

    $ echo this is a test > test

    $ cat test

    this is a test

    $

  • The ls command lists the files and sub-directories in the current working directory:

    $ ls

    newdir test

    $

  • To further illustrate this, we can use the touch command which creates files of binary length zero:

    $ touch file1 file2 file3 file4

    $ ls

    file1 file2 file3 file4 newdir test

    $

Stanberry

L2-7


Long listing of files ls l l.jpg

Long Listing of Files - ls -l

  • With no other options, the ls command gives you no information about the files it is listing other than the file name. With the -l option, there is much more information provided about each file:

    $ ls -l

    total 16

    -rw-rw-r-- 1 ds59478 dasd 0 Dec 29 00:05 file1

    -rw-rw-r-- 1 ds59478 dasd 0 Dec 29 00:05 file2

    -rw-rw-r-- 1 ds59478 dasd 0 Dec 29 00:05 file3

    -rw-rw-r-- 1 ds59478 dasd 0 Dec 29 00:05 file4

    drwxrwxr-x 2 ds59478 dasd 512 Dec 27 13:12 newdir

    -rw-rw-r-- 1 ds59478 dasd 15 Dec 28 23:59 test

    $

  • The long listing provides in fields all of the information that is stored in the inode of the file. Below is a summary of each field:

    (1) The file type and the permission bits

    (2) The link count

    (3) The name of the owner of the file

    (4) The name of the group of the file

    (5) The size of the file in bytes

    (6) The date the file was last modified

    (7) The filename

Stanberry

L2-8


File types and permission bits l.jpg

File Types and Permission Bits

  • The first bit of the first column on a long listing of a file indicates the file type. It is as follows:

    • blank (-) indicates an ordinary file

    • d indicates a directory file

    • other letters (such as b,c or l) indicate a special file

  • The next 9 bits indicate the permissions of the file. They consist of a read, write and execute bit for the owner of the file, the group of the file and all others. Each bit is either represented by the r,w or x (if on) or the blank (-) if off. Below is a summary of their meanings:

    • For an ordinary file:

      • r = permission to look at the contents of the file

      • w = permission to change the contents of the file

      • x = permission to execute file as a command

    • For a directory file:

      • r = permission to list the files in the directory

      • w = permission to create and remove files in directory

      • x = permission to make it the current directory

Stanberry

L2-9


Octal notation and permission bits l.jpg

Octal Notation and Permission Bits

  • The permission bits are often summarized by looking at each group of three bits as an octal number. In octal notation, each bit holds a place as follows:

    • r = 2**2 = 4 +

    • w = 2**1 = 2 +

    • x = 2**0 = 1 = octal permission

  • Each bit is either on or off, which represents a number for the group of three. There are a total of three numbers with one representing owner, group and other.

  • Therefore, we can read the permissions as 664 for each file and 775 for the directory for the example below:

    $ ls -l

    total 16

    -rw-rw-r-- 1 ds59478 dasd 0 Dec 29 00:05 file1

    -rw-rw-r-- 1 ds59478 dasd 0 Dec 29 00:05 file2

    -rw-rw-r-- 1 ds59478 dasd 0 Dec 29 00:05 file3

    -rw-rw-r-- 1 ds59478 dasd 0 Dec 29 00:05 file4

    drwxrwxr-x 2 ds59478 dasd 512 Dec 27 13:12 newdir

    -rw-rw-r-- 1 ds59478 dasd 15 Dec 28 23:59 test

    $

Stanberry

L2-10


Changing permission bits chmod l.jpg

Changing Permission Bits - chmod

  • To change the permission bits, you can use the chmod command with the octal notation below is an example:

    $ ls -l

    total 16

    drwxrwxr-x 2 ds59478 dasd 512 Dec 27 13:12 newdir

    -rw-rw-r-- 1 ds59478 dasd 15 Dec 28 23:59 test

    $ chmod 775 test

    $ ls -l

    total 16

    drwxrwxr-x 2 ds59478 dasd 512 Dec 27 13:12 newdir

    -rwxrwxr-x 1 ds59478 dasd 15 Dec 28 23:59 test

    $

  • By changing the permission bits to 775, the execute bit has been turned on for the owner, group and others.

  • Permission bits can also be changed with symbolic notation. By using u for owner, g for group, o for other and a or blank for all, any of the read, write or execute bits can be added using a (+) sign, reduced using a (-) sign or set explicitly using the (=) sign. Below are examples:

    $ chmod o+w (permits write for others)

    $ chmod go-r(restricts read from group and others)

    $ chmod +x(adds execute for all)

Stanberry

L2-11


Linking files ln command l.jpg

Linking Files - ln command

  • The second column of the long listing is the count of links to a file. Use the ln command to add a link. This command creates a file that has the same inode number as an existing file. The link file is the same physical file as the original file that can be accessed by a different name. Below is a dialogue to demonstrate:

    $ echo this is a test > test

    $ ls -l

    total 8

    -rw-rw-r-- 1 ds59478 dasd 15 Dec 29 16:52 test

    $ ln test link_to_test

    $ ls -l

    total 16

    -rw-rw-r-- 2 ds59478 dasd 15 Dec 29 16:52 link_to_test

    -rw-rw-r-- 2 ds59478 dasd 15 Dec 29 16:52 test

    $ rm test

    $ ls -l

    total 8

    -rw-rw-r-- 1 ds59478 dasd 15 Dec 29 16:52 link_to_test

    $

Stanberry

L2-12


Changing the owner and group chown and chgrp l.jpg

Changing the Owner and Group - chown and chgrp

  • The third and fourth columns of the long listing are the owner and the group with permissions to the file. The owner can be changed with the chown command:

    # ls -l

    total 16

    drwxrwxr-x 2 ds59478 dasd 512 Dec 29 17:05 newdir

    -rwxrwxr-x 1 dasd dasd 15 Dec 28 23:59 test

    # chown ds59478 test

    # ls -l

    total 16

    drwxrwxr-x 2 ds59478 dasd 512 Dec 29 17:05 newdir

    -rwxrwxr-x 1 ds59478 dasd 15 Dec 28 23:59 test

    #

  • The group can be changed with the chgrp command:

    # ls -l

    total 16

    drwxrwxr-x 2 ds59478 dasd 512 Dec 29 17:05 newdir

    -rwxrwxr-x 1 ds59478 dasd 15 Dec 28 23:59 test

    # chgrp sys test

    # ls -l

    total 16

    drwxrwxr-x 2 ds59478 dasd 512 Dec 29 17:05 newdir

    -rwxrwxr-x 1 ds59478 sys 15 Dec 28 23:59 test

    #

Stanberry

L2-13


Default permissions umask l.jpg

Default Permissions - umask

  • The umask specifies what permissions will be set by default on new files and directories. It is represented as an octal number from which the default file (666) and directory (777) permissions are subtracted. For example:

    • umask 000 = 666 for files and 777 for directories

    • umask 002 = 664 for files and 775 for directories

    • umask 022 = 644 for files and 755 for directories

  • In AIX, a default umask of 022 is automatically set up. This can be changed by the root user of the system.

  • The default umask can be set for each user in their user profile. User profiles will override the default.

  • On many UNIX systems, the umask that users have specified in their profile has become the subject of security audits.

Stanberry

L2-14


Hidden files ls al command l.jpg

Hidden Files – ls –al command

  • On UNIX file systems, filenames that begin with a dot (.) are hidden. In order to see the hidden files on a system, the –a option must be used with the ls command. When combined with the long listing, the command is ls –al as seen below:

    $ cd ~

    $ ls -al

    total 88

    drwxr-xr-x 6 ds59478 dasd 512 Dec 29 16:52 .

    drwxr-sr-x 128 root tla 2560 Nov 10 11:56 ..

    -rw------- 1 ds59478 dasd 400 Aug 5 17:06 .Xauthority

    -rw-r----- 1 ds59478 dasd 185 Nov 11 1998 .Xdefaults

    -rw-r----- 1 ds59478 dasd 74 May 21 1998 .exrc

    -rw-r----- 1 ds59478 dasd 1144 Nov 11 1998 .profile

    -rw------- 1 ds59478 dasd 2880 Dec 31 03:20 .sh_history

    drwxr-xr-x 2 ds59478 dasd 512 Mar 8 1999 .ssh

    drwxr-sr-x 2 ds59478 dasd 512 May 21 1998 bin

    drwxr-sr-x 2 ds59478 dasd 512 May 21 1998 etc

    drwxrwxr-x 3 ds59478 dasd 512 Dec 29 04:32 stuff

    $

  • The . file represents the current directory and the .. file represents the parent directory in the file system.

  • The ~/.profile file contains the user profile. Statements executed at login time set the user’s preferences.

Stanberry

L2-15


Lab 3 unix commands l.jpg

Lab 3UNIX Commands


General format of unix commands man entries l.jpg

General Format of UNIX Commands – man entries

  • Because of the way in which UNIX evolved, there are very few standards in the command syntax. In general, command formats follow a convention:

    $ command –options argument1 argument2 ... etc

  • However, this is not always true. Also, the same option may mean totally different things to different commands.

  • As a result, most commands have a manual entry. The manual entry is a file that is stored in a particular directory (that is customizable on every system) that contains documentation for the command. To view this documentation for a particular command, you can use the man command. The following example looks at the manual entry for the ls command:

    $ man ls

  • Manual entries usually include the proper syntax for the command, and explain all of the options and arguments that the command expects. It may also include special notes and/or about warnings about certain uses of the command.

Stanberry

L3-1


Copying files cp command l.jpg

Copying Files – cp command

  • To create a copy of a file, you can use the cp command. The format is as follows:

    $ cp source_file target_file

  • The above command creates a copy of the file in the current working directory. By using pathnames, cp can create a copy of a file in a different directory as follows:

    $ cp source_file ~/bin/target_file

  • The above example creates a copy of the file called target_file in my home directory. If you want to create a file of the same name, but in a different directory, you can use the dot (.) instead of a target file name:

    $ cp same_filename ~/bin/.

  • Note: The cp command leaves the source file alone, but it still can potentially be destructive because it will overwrite any target file that already exists. To prevent possible data loss, you can use the –i option of cp that warns the user if the target file exists. Below is a sample:

    $ cp –i source_file target_file

Stanberry

L3-2


Moving or renaming files mv command l.jpg

Moving (or Renaming) Files – mv command

  • The cp command creates a new file that is an independent copy of a file without changing the source file. By contrast, the mv command logically makes a copy of the file and deletes the original source file. Internally, it does this by simply changing the filename (or renaming the file) in the inode. The format is as follows:

    $ mv old_file new_file

  • By using pathnames, mv can place a file in a different directory as follows:

    $ mv old_file ~/bin/new_file

  • The mv command can also move the file to a different directory using the same name by using the dot (.) instead of a target file name:

    $ mv same_filename ~/bin/.

  • Note: The mv command also will overwrite any target file that already exists. Therefore, you can also use the –i option of mv that warns the user if the target file exists. Below is a sample:

    $ mv –i old_file new_file

Stanberry

L3-3


Displaying files pg and more commands l.jpg

Displaying Files - pg and more commands

  • Thus far, we have seen how to view the contents of an ordinary file using the cat command and the vi editor. There are some other commands that better facilitate looking at files (particularly large files).

  • The pg command displays a file a page at a time. The file can be scrolled using the enter key or searched using the /<search pattern> text. On AIX systems, there is a special file called /etc/filesystems that stores information on all of the filesystems. This is generally a large file, so we can use it as an example:

    $ pg /etc/filesystems (type <ctrl-c> to cancel)

  • The more command is similar to the pg command. The main difference is the fact that the file can be scrolled either a line at a time with the enter key, or a page at a time with the space bar. Below is an example:

    $ more /etc/filesystems (type <ctrl-c> to cancel)

  • Often, the more command is combined with another command using the pipe to control the output:

    $ cat /etc/filesystems | more

Stanberry

L3-4


Displaying files head and tail commands l.jpg

Displaying Files – head and tail commands

  • There are a few other commands that can be useful when displaying large text files. The head command, by default, displays the first ten lines of a file. For an example, we can use the /etc/filesystems file again:

    $ head /etc/filesystems

  • The head command is useful for glancing at the format of a large file without having to go into it.

  • The tail command displays the last ten lines of a file by default:

    $ tail /etc/filesystems

  • There are not too many cases where the tail command by itself is useful. However, some files are continuously updated. The –f option of the tail command will update the screen as output is added to the file. An example of this can be found with the system log (in AIX it is under /var/sysadm/messages). The system log is constantly updated, and the command below allows the log to be followed:

    $ tail –f /var/adm/messages (type <ctrl-c> to cancel)

Stanberry

L3-5


Searching for text patterns grep command l.jpg

Searching for Text Patterns – grep command

  • The grep command is very common utility used for searching out patterns in text files. There are many options and uses of the grep command that we will expand upon throughout the class. For now, we can look at some simple examples.

  • Let’s say that I have a file called birthdays in which I store all of the names and dates of the birthdays that are important to me. If I want to find a person’s birthday without looking at the entire file, I can use the grep command as follows:

    $ grep India birthdays

    India 12/10

    $

  • The –i option of the grep command says to ignore case:

    $ grep –i india birthdays

    India 12/10

    $

  • To look at all of the January birthdays:

    $ grep 01 birthdays

    Matt 01/28

    Rob 01/27

Stanberry

L3-6


More searching for text patterns egrep command l.jpg

More Searching for Text Patterns – egrep command

  • The egrep command allows for multiple patterns to be searched. These patterns are separated by an or (|) symbol in the text string that is represented in single quotes (‘’). Below is an example with the –i option:

    $ egrep –i ‘rob|india’ birthdays

    India 12/10

    Rob 01/27

    $

  • One of the more useful ways in which to use grep and egrep is to redirect the input of another command using the pipe (|) symbol. Therefore, commands which generate a great deal of output can be filtered for the relevant data. Below is an example with the df command. The df command produces output for every filesystem (with the –k option specifying the size in kilobytes). If you are only interested in the home filesystems, then the command below will filter the output (in this case only 1 home filesystem):

    $ df –k | grep home

    /dev/Less 1228800 1137740 8% 809 1% /home/ess

    $

Stanberry

L3-7


Sorting text files sort command l.jpg

Sorting Text Files – sort command

  • The sort command takes the lines of input to a specified file and sorts them according to the options. With the sort command, the format is important:

    $ sort [-t delimiter] [+field[.column]] [-options] file

  • Some of the main options are as follows:

    • d – sorts in dictionary order (alphabetical)

    • n – sorts in numeric order

    • r – sorts in reverse order

  • Sort has some common sense defaults, such as the space being the delimiter and the first field and column. Therefore, to sort the names in the birthday file in alphabetical order, you can use the following:

    $ sort –d birthdays

  • However, if you wanted to sort the file by the birth month in reverse order, you could use the following:

    $ sort +1 –rn birthdays

Stanberry

L3-8


Command i o redirection l.jpg

Command I/O Redirection

  • By default, the sort command generates output to what is referred to as standard output. In most cases, standard output means the screen. As we have seen, by using the redirection (>), the output of a command can be redirected to a file. Therefore, to create a file of the birthdays in alphabetical order, you can use the following:

    $ sort –d birthdays > alphabetical_birthdays

  • Standard output is known as a file descriptor that is represented by (>). There are two other standard file descriptors that are associated with every command. They are standard input, which is represented by (<), and standard error that is represented by (2>).

  • An example of redirecting standard input is using a file for the input of the mail command. Below, we can see how this technique sends a file to someone via mail:

    $ mail user1 < letter

  • All commands may potentially generate error messages. These messages are sent to standard error (once again, usually the screen). These can be redirected to a file as well. For example, if we try to cat a file that does not exist, we will get an error. In the example below, this error is redirected to an error log:

    $ cat non_existent_file 2> error_log

Stanberry

L3-9


Combined command i o redirection l.jpg

Combined Command I/O Redirection

  • As mentioned before, using (>) to redirect standard output is a potentially dangerous technique. If a file by the same name exists, the (>) will overwrite the file. To solve this, redirection can be instructed to append to rather than overwrite a file using (<<) for standard input, (>>) for standard output and (2>>) for standard error. Below is an example:

    $ cat file >> append_file

  • Note: If the file specified in the >> or 2>> does not exist, one will be created automatically.

  • By combining these I/O redirection commands, commands can utilize several files. In the example below, a file will be used for input, a new file will be created for output and an error log will be appended:

    $ custom_command < infile > outfile 2>> error_log

  • The pipe (|) operator that we have discussed previously has a built-in I/O redirection combination mechanism. It takes the standard output of one command and uses it as (or “pipes” it to) the standard input of another command. Another example of a combination takes the output of the df command, pipes it to a search for home filesystems, sends the standard output to a new file and appends the standard error to an error log:

    $ df –k | grep home > home_filesystems 2>>error_log

Stanberry

L3-10


Metacharacters and wildcards l.jpg

Metacharacters and Wildcards

  • Metacharacters are characters that have a special meaning to the shell. We mentioned earlier that the metacharacters (*?></:$![]{}| \`”) should not be used in filenames. This will become abundantly clear as we begin to use them with shell commands and programs.

  • The metacharacters make the foundation of what is known in UNIX as regular expressions. The regular expressions form a language of pattern matching in text that is generally understood by most UNIX programs. These are used to greatly enhance shell commands and programs.

  • Wildcards are a type of metacharacter that are used to substitute filenames that are of certain criteria. They are used with many UNIX commands. The wildcards are as follows:

    * – substitutes all characters

    ? – substitutes a single character

    ! – substitutes all characters except this character

    [ ] – substitutes a range of characters

Stanberry

L3-11


Examples of wildcards find and grep l.jpg

Examples of Wildcards – find and grep

  • Many of the commands that we have learned up to this point can and often do make use of wildcards. Below are examples:

    $ ls test? (lists files with test + 1 char such as test1 & test2)

    $ cp * ~/. (copy everything in the current directory to my home directory)

    $ rm –r * (recursively remove everything in current directory ***VERY DANGEROUS)

    $ ls *[1-5] (lists any files ending in 1-5 such as matt1 and test5)

    $ mv [!n]* ~/. (move all files except beginning with an n to my home directory)

  • The find command is a powerful tool with many options. It searches a directory hierarchy for files and can perform a number of different functions on the files it finds. In the simple example below, find locates all of the files in the current directory structure that begin with the letter m and prints them to the screen:

    $ find . –name m* -print

  • The grep command is also interesting when combined with wildcards. For an example, let’s say that I have a sub-directory under my home directory called personal where I keep information about people. If I wanted to know everything in there about my wife India, grep would print a line for each occurrence of the name and also print which files in which they reside. Below is the example:

    $ grep India ~/personal/*

    /u/ds59478/personal/birthdays:India 12/10

    /u/ds59478/personal/email:India IndiaRS@aol.com

    /u/ds59478/personal/phone:India (972) 506-0843

    $

Stanberry

L3-12


Lab 4 shell scripts l.jpg

Lab 4Shell Scripts


Introduction to shell scripts l.jpg

Introduction to Shell Scripts

  • On the most basic level, a shell script is simply an ordinary text file that contains a set of shell commands. You can use vi to create a basic shell script:

    $ vi cmd_test(use vi to add the following lines)

    pwd

    date

    echo hello world

  • Although we will learn several ways to execute shell scripts, the most basic way to execute it is with the ksh command as below:

    $ ksh cmd_test

    /u/ds59478/stuff

    Sun Jan 2 04:00:55 EST 2000

    hello world

  • Essentially what we have created is a custom new UNIX command that executes three other commands. Although this example is not a particularly useful command, it provides the basis for shell scripts.

Stanberry

L4-1


Executable shell scripts l.jpg

Executable Shell Scripts

  • On the previous page, we created a shell script called cmd_test that is a custom UNIX command. Like other UNIX commands, we should be able to execute it without the ksh command:

    $ cmd_test

    ksh: cmd_test: 0403-006 Execute permission denied.

    $

  • However, by default, ordinary files are not created with the execute permission bit turned on:

    $ ls -l cmd_test

    -rw-r--r-- 1 ds59478 dasd 26 Jan 2 04:00 cmd_test

    $

  • To make the cmd_test file executable, use the chmod command:

    $ chmod +x cmd_test

    $ ls -l cmd_test

    -rwxr-xr-x 1 ds59478 dasd 26 Jan 2 04:00 cmd_test

    $ cmd_test

    /u/ds59478/stuff

    Sun Jan 2 05:40:43 EST 2000

    hello world

    $

  • Shell scripts that are to be used again should have the permissions set to executable, and should be moved to the ~/bin directory.

Stanberry

L4-2


Shell script variables l.jpg

Shell Script Variables

  • To demonstrate the use of variables in shell scripts, we shall modify the contents of our test script:

    $ vi cmd_test(use vi to change file to lines below)

    pwd

    date

    x='hello world'

    echo $x

  • In this script, we assign the value of hello world to a variable called x. In the syntax of shell scripting, the equal sign (=) represents assignment and the string is represented by the single quotes (‘’). The echo command uses the ($) to represent the value of a variable as its string. Below is the result:

    $ cmd_test

    /u/ds59478/stuff

    Sun Jan 2 06:00:48 EST 2000

    hello world

    $

  • In this script, the $x was replaced with the hello world string.

Stanberry

L4-3


Shell processes l.jpg

Shell Processes

  • Now, notice what happens when we look at the $x variable:

    $ echo $x

    $

  • The script on the previous slide set the value of $x to hello world, but it is now empty. To see why, add the following statement (by using vi) to the cmd_test script:

    echo $$

  • The $$ parameter of echo displays the process id number:

    $ cmd_test

    /u/ds59478/stuff

    Sun Jan 2 06:13:57 EST 2000

    hello world

    294170

    $ echo $$

    337662

    $

  • Notice that the process id numbers are different. This is because the command executes its own process called a child process. This is different from the original process which is called the parent process.

Stanberry

L4-4


Sourcing shell processes l.jpg

Sourcing Shell Processes

  • In general, UNIX commands create their own child processes for the life of the command. We see this when we execute the cmd_test script via ksh or directly. However, there is a way to execute the command in our current shell process. This is done by using the source (.) command as follows:

    $ . cmd_test

    /u/ds59478/stuff

    Sun Jan 2 06:25:25 EST 2000

    hello world

    337662

    $ echo $x

    hello world

    $ echo $$

    337662

    $

  • As you can see the process ids are the same for the cmd_test as they are for the shell session. Also, the shell now has the $x variable set to hello world. This is known as “sourcing” a command.

Stanberry

L4-5


More shell processes ps command l.jpg

More Shell Processes – ps command

  • In general, at any given time, UNIX systems have many processes running on them at once (multi-user, multi-threaded). To view all of these processes, we can use the ps command with the –ef flag:

    $ ps –ef

  • To view your processes in this format, you can pipe the output of the ps command to a grep for your id:

    $ ps -ef | grep ds59478

    ds59478 102222 337662 0 06:41:56 pts/2 0:00 grep ds59478

    ds59478 140262 337662 24 06:41:56 pts/2 0:00 ps -ef

    ds59478 337662 379778 0 05:33:50 pts/2 0:00 -ksh

    $

  • At the time this command was running, I had only my native (or parent) shell process going. The other two entries are for the ps and grep commands themselves.

  • There are many commands that revolve around the management of processes that are outside the scope of this course. However, learning the ps command and using echo $$ provide an introductory look at processes.

Stanberry

L4-6


Shell environment variables set command l.jpg

Shell Environment Variables – set command

  • In previous labs, we have seen a special file called ~/.profile which contains user defined defaults for the shell environment. There are actually several different files that serve to customize the user environment. We covered the ~/.profile because it is a common convention used in all UNIX.

  • Specifically on AIX systems, there are several files such as /etc/environment, /etc/passwd, /etc/groups and /etc/profile that help define the default user environment. Generally, users can not edit these files without the help of a system administrator. However, these files contain the default definitions of several environment variables.

  • The set command displays all of the variables that are defined to the shell. When you first log into a system, this includes all of the environment variables that get initialized. If you define a new variable to the shell, then it also can be displayed by using the set command. Below is an example:

    $ x=matt

    $ set | grep x

    x=matt

    $

Stanberry

L4-7


Shell environment variables path l.jpg

Shell Environment Variables – PATH

  • One of the most basic and common uses of shell programming is creating and/or modifying the shell environment variables via the ~/.profile file. For example, all variables set in the /etc/profile file can be overridden by changing them in the ~/.profile of the user.

  • An example of this is the PATH variable. This variable determines the directory paths that the shell will search when attempting to execute a command. The following command allows you to view your PATH variable:

    $ echo $PATH

  • Binary executable programs are normally stored in a bin directory. When users write a script that will be used again, they will often move the script to the ~/bin directory. If it is a program that could be used by other users, then it can be moved to a shared bin directory such as /usr/bin.

  • In the example below, we create a new script in the ~/bin directory and source it. The system knows where to find the script because of the PATH variable:

    $ echo ‘clear;ls –al’ > ~/bin/clr

    $ . clr

Stanberry

L4-8


Shell environment variables ps1 and export l.jpg

Shell Environment Variables – PS1 and export

  • Another shell variable is PS1, which controls the shell prompt. Some users (especially those coming from a DOS background) would like to see the current directory as part of the shell prompt. This is done with the PS1 variable as below:

    PS1=‘$PWD $ ’

  • As we learned before, shell variables only exist for the current (or parent) shell process. If subsequent (or child) processes are started, then the shell variables need to be re-assigned. However, this can be avoided with the export command:

    export PATH PS1

  • The export command passes the shell variables to the child process so that the shell variables do not need to be reassigned. This process is aptly called inheritance, because parent shells pass on their values to the child shells.

  • At this point, we can see that the ~/.profile is not just a special file, but a working shell script that assigns variables, passes parameters and executes UNIX commands.

Stanberry

L4-9


Lab 5 the unix programming environment l.jpg

Lab 5The UNIX Programming Environment


Introduction to programming in unix l.jpg

Introduction to Programming in UNIX

  • One of the great strengths of the UNIX operating system is its ability to support programming. The UNIX environment supports the following types of programming:

    • Shell Programming

      • Programming shell scripts with the syntax and commands that are supported by a native UNIX shell. Examples include sh, ksh, csh, bash and many others.

    • Interpreted Programming Languages

      • Programming in a language syntax that is fed into a larger UNIX program (known as an interpreter) and executed by that program. Examples include sed, awk and perl.

    • Compiled Programming Languages

      • Programming in a language syntax that is compiled into object code and run as a binary executable program. Examples include C, C++, FORTRAN, PASCAL, assembler, etc.

  • In this section we will expand on the shell programming concepts introduced in Lab 4. We will also introduce the interpreted languages of sed, awk and perl.

Stanberry

L5-1


Shell programming using arguments l.jpg

Shell Programming – using arguments

  • In the last section, we introduced the concept of shell scripts. We can also use the shell as a programming environment by taking advantage of the constructs that are built into the shell scripting language. In this section, we will learn some of these basic constructs to expand on our shell programming abilities.

  • Also, in the last section, we learned about shell variables. These are variables that are defined within the shell script. Arguments are special shell variables that get passed to the program at the execution of the command. In many of the UNIX commands we have learned, we have been passing arguments to programs. If you recall, the standard format of a command is:

    $ command –options argument1 argument2 ... etc

  • In order to use arguments, the format is simply ${n} with n being the position of the argument in the command. Consider the following simple program below:

    $ cat arg_script

    echo I have a dog named $1

    echo I have a spouse named $2

    echo I have a dad named $3

    $ ksh arg_script daisy india jack

    I have a dog named daisy

    I have a spouse named india

    I have a dad named jack

    $

Stanberry

L5-2


Shell programming read statement l.jpg

Shell Programming – read statement

  • Arguments are very useful for programming in the context of UNIX-style commands. However, like any programming language, shell scripts can also read input interactively from users. In the example below, the same program that read arguments now presents an interactive dialogue with the user:

    $ cat read_script

    echo 'What is the name of your dog?'

    read dog

    echo 'You better take' $dog 'for a walk!'

    $ ksh read_script

    What is the name of your dog?

    daisy

    You better take daisy for a walk!

    $

  • The read command utilizes standard input. By default, as in this example, this means input from the screen. However, this can be redirected to use input from other sources such as a file.

Stanberry

L5-3


Shell programming integer arithmetic operations using expr l.jpg

Shell Programming – Integer Arithmetic Operations using expr

  • By using the expr statement, simple integer arithmetic can be performed inside a shell program. For an example, we can write a relatively simple and useful shell script that will convert blocks of disk to megabytes. In AIX, a block of disk is 512 bytes. Two blocks are equal to 1024 bytes, which is also known as a kilobyte. Therefore, if we divide the number of blocks by 2048, then we get the number of megabytes. The script below does this:

    $ cat conv

    expr $1 / 2048

    $ conv 10000

    4

    $

  • A few things of note about the script above. First, I used the chmod +x command to make the file conv executable so that I did not need to use the ksh command to execute it. Also, the expr returns an integer number, so the resulting number in megabytes is rounded.

Stanberry

L5-4


Shell programming conditional operations using if l.jpg

Shell Programming – Conditional Operations using if

  • The conv command from the last exercise is somewhat useful in aiding the calculation of disk space on AIX systems. However, let’s say that this program could work better if it could convert either from blocks to megabytes or vice-versa.

  • In order to do this, we will need to calculate the formula depending upon the type of input. This requires what is known as conditional processing. The most simple form of conditional processing is the if statement.

  • The format of the if statement is as follows:

    if test condition

    then

    statements (executed only on true condition)

    else

    statements (executed only on false condition)

    fi

  • Note: If statements can be nested to check for multiple conditions. This will be done in the updated conv program on the next page.

Stanberry

L5-5


Shell programming updated conv program using if statements l.jpg

Shell Programming – Updated conv program using if statements

  • The updated conv program uses nested if statements to check for blocks, mb, or invalid data:

    $ cat conv

    if test "$1" = "blocks"

    then

    expr $2 / 2048

    else

    if test "$1" = "mb"

    then

    expr $2 \* 2048

    else

    echo You have entered invalid data:

    echo 'The format is: conv [blocks|mb] number'

    fi

    fi

    $ conv blocks 10000

    4

    $ conv mb 4

    8192

    $ conv foo

    You have entered invalid data:

    The format is: conv [blocks|mb] number

    $

Stanberry

L5-6


Shell programming a look at the profie script l.jpg

Shell Programming – A look at the ~/.profie script

  • We can also use the if statement in the ~/.profile script. In the statement below, the script checks for the existence of new mail and executes a mail session if there is some:

    $ cat ~/.profile

    PATH=/bin:/usr/local/bin:/usr/bin:/sbin:_

    /usr/local/etc/httpd:/etc:/usr/ucb:$HOME/bin:/usr/bin/X11:.

    export PATH

    TERM=ibm3151

    if [ `tty` = /dev/tty0 ]

    then

    export TERM=ibm3151

    else

    export TERM=vt100

    fi

    set -o vi

    if [ -s "$MAIL" ]

    then echo "$MAILMSG"

    fi

    EDITOR=/usr/bin/vi

    export EDITOR

    PS1='$ '

Stanberry

L5-7


Interpreted programming languages sed awk and perl l.jpg

Interpreted Programming Languages – sed, awk and perl

  • As mentioned earlier, the UNIX programming environment supports interpreted programming language scripts. Three very popular languages in the UNIX environment are sed, awk and perl. Below is a brief explanation of each:

    • sed – stands for stream editor. It is mostly used for repetitive changes in text patterns as a “find and replace”. It can be issued interactively from the vi editor or via the command line.

    • awk – named for its authors Aho, Weinberg and Kernighan of Bell Labs. Specializes in formatting text from multiple input sources. Serves as a great tool for report generation. Very similar to C language in structure (developed by the same people).

    • perl – stands for practical extraction and reporting language, developed by Larry Wall. Full blown programming language that combines the features and functions of C, sed, awk and shell programming. There are many different ways to perform the same function using perl. Available on multiple platforms as public domain software.

  • There are entire books and courses dedicated to each one these languages. However, as an introduction, in this class we will write a very simple script in each language.

Stanberry

L5-8


An example of sed text substitution l.jpg

An example of sed – text substitution

  • People most commonly use sed for global substitutions in text files. Below is a simple example:

    $ cat syllabus

    This file contains the spring syllabus for MGT6346

    which will be taught in the spring semester.

    This spring, I will spring into action as

    I teach this class in the spring.

    $ sed s/spring/summer/ syllabus > summer_syllabus

    $ cat summer_syllabus

    This file contains the summer syllabus for MGT6346

    which will be taught in the summer semester.

    This summer, I will spring into action as

    I teach this class in the summer.

    $

  • Notice that only one occurrence of “spring” was substituted on each line. To substitute all occurrences, specify the letter g in the criteria for a global substitution:

    $ sed s/spring/summer/g syllabus > summer_syllabus

Stanberry

L5-9


An example of awk formatting the output of ls l.jpg

An example of awk – formatting the output of ls

  • In the UNIX environment, awk (using the –f option) is often used to take the output from a command and format it:

    $ cat awk_ls

    BEGIN {print "Bytes" "\t" "Filename"} (sets up header)

    {sum += $5;print $5 "\t" $9} (Loops through input)

    END {print "Total Bytes are "sum} (sets up footer)

    $ ls -l | awk -f awk_ls

    Bytes Filename

    101 awk_ls

    172 summer_syllabus

    172 syllabus

    15 test

    Total Bytes are 460

    $

  • Although awk is still commonly used, awk programs that interact with UNIX commands are generally being replaced by the more robust perl language. The availability of awk to perl conversion utilities has helped to facilitate this migration.

Stanberry

L5-10


An example of perl interactively listing home directory files l.jpg

An example of perl –interactively listing home directory files

  • An example of how perl can interact with commands (as we have seen with awk) and with UNIX commands (as we have seen with shell scripting) can be seen in the program below:

    $ cat perl_dir

    #!/usr/bin/perl

    print "Enter the username of the home directory you would like to view: ";

    chop($hdir = <STDIN>);

    chdir(~$hdir) || die "Invalid username"

    foreach(<*>) {

    print "$_\n";

    }

    $

  • A topic within the world of perl that is outside the scope of this course (but worth mentioning) is the use of macros. One of the things that makes perl so powerful is that perl macros can be written on one platform and ported to others. The use of perl macros is one of the reasons that it is becoming a popular scripting language for web-based applications.

Stanberry

L5-11


Unit 3 unix security l.jpg

Unit 3UNIX Security


Unix security concepts l.jpg

UNIX Security Concepts

  • The UNIX operating system is designed by default to be secure:

    • Only the root user can perform any functions by default.

    • The file system is restricted by the umask.

    • Since all functions are accessed via files which are secured by permissions, all functions are restricted by default.

  • On the other hand, the UNIX operating system is easily compromised once the root password has been obtained. The root user can:

    • Access all files in the file system.

    • Access all directories in the file system.

    • Perform system management functions.

    • Boot the system and operate it in maintenance mode.


Unix users l.jpg

UNIX Users

  • User Accounts:

    • Each user has a unique name, numeric ID and password.

    • Every file has the numeric user ID of its owner stored in its inode. This can only be changed by the owner of the file or the root user.

    • The root user is typically the only default user on the system. Some systems may have a generic guest user, but its permissions will be restricted.

  • All user accounts are defined to a file that is usually stored at /etc/passwd. This file is readable to all users. In the example below, I have used grep to find my password in the /etc/password file:

    $ grep ds59478 /etc/passwd

    ds59478:!:59478:152:Matt_Stanberry_45199:/u/ds59478:/usr/bin/ksh

    $

  • Below is an explanation of each field:

    • username

    • pointer to /etc/security/passwd where encrypted password is stored

    • user id number

    • primary group id number

    • description of the user account

    • login directory (initial value of $HOME)

    • login shell (initial value of $SHELL)


Unix groups l.jpg

UNIX Groups

  • Groups:

    • Each group has a unique name and numeric ID.

    • A group is established as a set of users that need access to the same files. All users are a member of at least one group. By using the groups command, a user can list the groups to which they have been defined.

    • Every file has the numeric group ID of its group stored in its inode. This can only be changed by the owner of the file, a group member or the root user.

    • There are 3 groups defined on all AIX systems by default. They are system, security and staff.

  • All groups are defined to a file that is usually stored at /etc/group. This file is also readable to all users. In the example below, I have used grep to find my group in the /etc/group file:

    $ grep dasd /etc/group

    dasd:!:152:dasd,ds59478,a200067,ds56305,ds56228,ds56221,ds56336

    $

  • Below is an explanation of each field:

    • group name

    • pointer to /etc/security/group where encrypted password is stored

    • comma separated list of the usernames of the group members


Controlling access to root l.jpg

Controlling Access to root

  • The root user id can be accessed by two ways:

    • Logging on to the system with the username of root and the root password .

    • Using the su command and providing the root password.

  • When the su command is used, the original username is stored in a log (in AIX this is stored at /var/adm/sulog). This provides accountability for the actions of the root user. Therefore, system administrators should only login with the root password in special circumstances such as brining up the system in maintenance mode.

  • However, this is not really enforceable since anyone with the root password can login as root. This has prompted the use of special security software. Examples include:

    • ANS – Sun Microsystems

    • Citadel – BSD

    • Lucent – System V

    • CyberGuard - Linux


Ways to protect the system l.jpg

Ways to Protect the System

  • System security is compromised in a variety of ways. Below are some things to be aware of:

    • Social Engineering: The most common ways that intruders gain access to systems is by obtaining passwords. This is often done by gaining information about individuals with access to systems. Passwords should not be family members, interests, etc. that are easily guessed by someone that knows things about you.

    • Proximity: Servers need to be stored in a physically secure area that is away from boot media, panel keys, etc. Also, technicians need to reside in a secure area where password clues can not be obtained. All users should lock their terminals when away from their desks.

    • Login Policy: Passwords need to be changed regularly and placed under restrictions (minimum length, similarity to old passwords, key words, etc.) Unsuccessful login attempts should be tracked and reported. Users should beware of Trojan horses.

    • Auditing: As much as possible, system activity should be tracked and logged. Users must be held accountable for activity that is performed under their user name. The root password should always be accounted for and logged. Old userids should be deleted when individuals change jobs.


Summary unix security l.jpg

Summary: UNIX Security

  • Advantages:

    • Unlike some operating systems, UNIX has security functions built into the operating system.

    • The file system of UNIX simplifies the understanding of the security functions.

    • The root user is capable of circumventing the security functions if necessary.

  • Disadvantages

    • System is easily compromised once root password is obtained.

    • Many functions are restricted to the root password, which increases the need for its accessibility.

    • There is no accountability for actions performed by the root user.

    • All of the above factors have led to the use of additional security software in the UNIX environment.


Unit 4 compatibility and standards l.jpg

Unit 4Compatibility and Standards


Why are standards important l.jpg

Why are Standards Important?

  • The use of standards make the Open Systems Concept work:

    • Applications can interoperate and port data across a wide variety of hardware and software platforms.

  • This makes the following UNIX attributes possible:

    • Portability

    • Scalability

    • Compatibility

  • There are two types of standards:

    • De Facto:

      • Standards that are established by broad industry usage of a technology.

    • De Jure:

      • Standards that are established by recognized standards bodies through a formal voting process. This process is facilitated by standards organizations.


Standards organizations l.jpg

Standards Organizations

  • ISO – International Standards Organization

    • Combination of national standards organizations in over 90 countries that covers all manufacturing processes (ex. ISO 9000).

  • IEEE – Institute of Electrical and Electronic Engineers

    • Developed Portable Operating Systems Interface (Posix) standard for UNIX.

  • OSF – Open Software Foundation

    • Non-profit research and development organization whose goal is to provide a software solution for an open systems computing environment.

  • X/Open

    • An association of vendors devoted to promoting the practical implementation of open systems.


Posix standards l.jpg

POSIX Standards

  • POSIX is a set of standards that define common interfaces across operating systems.

  • Most of the interfaces are defined in terms of the C language.

  • Below are a list of some POSIX standards:

    • 1003.1 - System Application Program Interface (API) for the C language

    • 1003.1e - Security Extensions

    • 1003.1f - Transparent File Access

    • 1003.1g - Protocol Independent Networking

    • 1003.2 - Shell and Utilities

    • 1003.15 - Batch System Administration


Osf standards l.jpg

OSF Standards

  • OSF offers a comprehensive, integrated set of services and technologies known as Distributed Computing Environment (DCE).

  • DCE supports the development, use and maintenance of distributed applications.

  • OSF/DCE offerings are organized as follows:

    • Fundamental Distributed Services

      • Remote Procedure Call (RPC)

      • Directory Services

      • Time Services

      • Security Services

      • Thread Services

    • Data Sharing Services

      • Distributed File Services

      • Diskless Support Services


X open standards l.jpg

X/Open Standards

  • X/Open is dedicated to developing an open multi-vendor common applications environment.

  • Publishes X/Open Portability Guide (XPG).

  • Latest Version: XPG 4.2

    • X/Open specifications

      • Commands and Utilities

      • System Interfaces and Headers

      • SOCKETS Networking API

  • Open systems environment specification based upon both formal and de facto standards.

  • Intended to provide application portability and interoperability while allowing movement of users between systems without re-training.

  • A joint venture between OSF and X/Open created the Open Group in 1996. These organizations work together closely in developing the two standards.


Unit 5 unix performance l.jpg

Unit 5UNIX Performance


Monitoring unix performance l.jpg

Monitoring UNIX Performance

  • Monitoring Tools

    • Native operating system tools

    • Home grown scripts/tools

    • Operating system specific tools

    • Third party products

      • LandMark

      • Candle Command Center/ECO

      • Magic

      • SAS/MXG

  • AIX Performance Tools

    • sar – system activity report

    • vmstat – virtual memory statistics

    • iostat – reports input/output activity

    • tprof – CPU utilization and application profiling

    • svmon – memory utilization

    • filemon – disk and file input/output statistics

    • PDT – Performance Diagnostic Tool

    • PerfPMR – Performance Problem Management Report


Managing processes l.jpg

Managing Processes

  • Scheduling

    • cron daemon and crontab

    • Batch processing

    • Backups

      • Incremental

      • Tools

    • skulker

    • Scheduling products

      • Control-M

      • CA-7

      • InterMix

  • Signals

    • Priority

      • nice and renice commands

    • Kill

      • Orphaned processes

      • Zombie processes


Managing resources l.jpg

Managing Resources

  • Resource Limits

    • Pre-defined limits to the amount of resources that individual users can occupy on a system.

    • Pro-actively prevents performance issues.

    • AIX examples

      • fsize – size of an individual file

      • core – size of a core dump file

      • cpu – max CPU seconds before process is killed

      • data – max data segments in memory

      • stack – max stack size in memory

      • rss – real memory size

    • In the AIX operating system, the above (and others) can be set either using the ulimit command or in the file /etc/security/limits.

    • Limits can be placed in other areas such as at the filesystem or logical volume level.


Performance management decisions l.jpg

Performance Management Decisions

  • Technical Issues

    • CPU – Queue statistics

    • Memory – Paging statistics

    • SCSI/Controller – I/O statistics

    • Disk/Peripherals – I/O statistics

  • Managerial Issues

    • Impact of performance issues

    • Importance of system

    • Importance of application

    • Costs

      • Opportunity

      • Impact

      • Tools

      • Man Hours


  • Login