guide to unix using linux third edition l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Guide To UNIX Using Linux Third Edition PowerPoint Presentation
Download Presentation
Guide To UNIX Using Linux Third Edition

Loading in 2 Seconds...

play fullscreen
1 / 33

Guide To UNIX Using Linux Third Edition - PowerPoint PPT Presentation


  • 181 Views
  • Uploaded on

Guide To UNIX Using Linux Third Edition. Chapter 5: Advanced File Processing. Objectives. Use the pipe operator to redirect the output of one command to another command Use the grep command to search for a specified pattern in a file Use the uniq command to remove duplicate lines from a file

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Guide To UNIX Using Linux Third Edition' - aerona


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
guide to unix using linux third edition

Guide To UNIX Using LinuxThird Edition

Chapter 5:

Advanced File Processing

objectives
Objectives
  • Use the pipe operator to redirect the output of one command to another command
  • Use the grep command to search for a specified pattern in a file
  • Use the uniq command to remove duplicate lines from a file
  • Use the comm and diff commands to compare two files
  • Use the wc command to count words, characters and lines in a file

Guide to UNIX Using Linux, Third Edition

objectives continued
Objectives (continued)
  • Use manipulation and transformation commands, which include sed, tr, and pr
  • Design a new file-processing application by creating, testing, and running shell scripts

Guide to UNIX Using Linux, Third Edition

advancing your file processing skills
Advancing YourFile-Processing Skills
  • Selection commands focus on extracting specific information from files

Guide to UNIX Using Linux, Third Edition

advancing your file processing skills continued
Advancing YourFile Processing Skills (continued)
  • Manipulation and transformation commands alter and transform extracted information into useful and appealing formats

Guide to UNIX Using Linux, Third Edition

advancing your file processing skills continued6
Advancing YourFile Processing Skills (continued)

Guide to UNIX Using Linux, Third Edition

using the selection commands
Using the Selection Commands
  • Using the Pipe Operator

– The pipe operator (|) redirects the output of one command to the input of another

    • An example would be to redirect the output of the ls command to the more command
    • The pipe operator can connect several commands on the same command line

Guide to UNIX Using Linux, Third Edition

using the pipe operator
Using the Pipe Operator

Using pipe operators and connecting commands is useful when viewing directory information

Guide to UNIX Using Linux, Third Edition

using the grep command
Using the grep Command
  • Used to search for a specific pattern in a file, such as a word or phrase
  • grep’s options and wildcard support allow for powerful search operations
  • You can increase grep’s usefulness by combining with other commands, such as head or tail

Guide to UNIX Using Linux, Third Edition

using the uniq command
Using the uniq Command
  • Removes duplicate lines from a file
  • Compares only consecutive lines, therefore uniq requires sorted input
  • uniq has an option that allows you to generate output that contains a copy of each line that has a duplicate

Guide to UNIX Using Linux, Third Edition

using the uniq command continued
Using the uniq Command (continued)

Guide to UNIX Using Linux, Third Edition

using the uniq command continued12
Using the uniq Command (continued)

Guide to UNIX Using Linux, Third Edition

using the comm command
Using the comm Command
  • Used to identify duplicate lines in sorted files
  • Unlike uniq, it does not remove duplicates, and it works with two files rather than one
  • It compares lines common to file1 and file2, and produces three column output
    • Column one contains lines found only in file1
    • Column two contains lines found only in file2
    • Column three contains lines found in both files

Guide to UNIX Using Linux, Third Edition

using the diff command
Using the diff Command
  • Attempts to determine the minimal changes needed to convert file1 to file2
  • The output displays the line(s) that differ
  • Codes in the output indicate that in order for the files to match, specific lines must be added or deleted

Guide to UNIX Using Linux, Third Edition

using the wc command
Using the wc Command
  • Used to count the number of lines, words, and bytes or characters in text files
  • You may specify all three options in one issuance of the command
  • If you don’t specify any options, you see counts of lines, words, and characters (in that order)

Guide to UNIX Using Linux, Third Edition

using the wc command continued
Using the wc Command (continued)

The options for the wc command:

–l for lines

–w for words

–c for characters

Guide to UNIX Using Linux, Third Edition

using manipulation and transformation commands
Using Manipulation and Transformation Commands
  • These commands are: sed, tr, pr
  • Used to edit and transform the appearance of data before it is displayed or printed

Guide to UNIX Using Linux, Third Edition

introducing the sed command
Introducing the sed Command
  • sed is a UNIX/Linux editor that allows you to make global changes to large files
  • Minimum requirements are an input file and a command that lets sed know what actions to apply to the file
  • sed commands have two general forms
    • Specify an editing command on the command line
    • Specify a script file containing sed commands

Guide to UNIX Using Linux, Third Edition

translating characters using the tr command
Translating CharactersUsing the tr Command
  • tr copies data from the standard input to the standard output, substituting or deleting characters specified by options and patterns
  • The patterns are strings and the strings are sets of characters
  • A popular use of tr is converting lowercase characters to uppercase

Guide to UNIX Using Linux, Third Edition

using the pr command to format your output
Using the pr Command toFormat Your Output
  • pr prints specified files on the standard output in paginated form
  • By default, pr formats the specified files into single-column pages of 66 lines
  • Each page has a five-line header containing the file name, its latest modification date, and current page, and a five-line trailer consisting of blank lines

Guide to UNIX Using Linux, Third Edition

designing a new file processing application
Designing a New File-Processing Application
  • The most important phase in developing a new application is the design
  • The design defines the information an application needs to produce
  • The design also defines how to organize this information into files, records, and fields, which are called logical structures

Guide to UNIX Using Linux, Third Edition

designing records
Designing Records
  • The first task is to define the fields in the records and produce a record layout
  • A record layout identifies each field by name and data type (numeric or nonnumeric)
  • Design the file record to store only those fields relevant to the record’s primary purpose

Guide to UNIX Using Linux, Third Edition

linking files with keys
Linking Files with Keys
  • Multiple files are joined by a key: a common field that each of the linked files share
  • Another important task in the design phase is to plan a way to join the files
  • The flexibility to gather information from multiple files comprised of simple, short records is the essence of a relational database system

Guide to UNIX Using Linux, Third Edition

creating the programmer and project files
Creating the Programmerand Project Files
  • With the basic design complete, you now implement your application design
  • UNIX/Linux file processing predominantly uses flat files
  • Working with these files is easy, because you can create and manipulate them with text editors like vi and Emacs

Guide to UNIX Using Linux, Third Edition

creating the programmer and project files continued
Creating the Programmerand Project Files (continued)

Guide to UNIX Using Linux, Third Edition

formatting output
Formatting Output
  • The awk command is used to prepare formatted output
  • For the purposes of developing a new file-processing application, we will focus primarily on the printf action of the awk command

Awk provides a shortcut to other UNIX/Linux commands

Guide to UNIX Using Linux, Third Edition

using a shell script to implement the application
Using a Shell Script toImplement the Application
  • Shell scripts should contain:
    • The commands to execute
    • Comments to identify and explain the script so that users or programmers other than the author can understand how it works
  • Use the pound (#) character to mark comments in a script file

Guide to UNIX Using Linux, Third Edition

running a shell script
Running a Shell Script
  • You can run a shell script in virtually any shell that you have on your system
  • The Bash shell accepts more variations in command structures that other shells
  • Run the script by typing sh followed by the name of the script, or make the script executable and type ./ prior to the script name

Guide to UNIX Using Linux, Third Edition

putting it all together to produce the report
Putting it All Together toProduce the Report
  • An effective way to develop applications is to combine many small scripts in a larger script file
  • Have the last script added to the larger script print a report indicating script functions and results

Guide to UNIX Using Linux, Third Edition

chapter summary
Chapter Summary
  • UNIX/Linux file-processing commands are (1) selection and (2) manipulation and transformation commands
  • uniq removes duplicate lines from a sorted file
  • comm compares lines common to file1 and file2
  • diff tries to determine the minimal set of changes needed to convert file1 into file2

Guide to UNIX Using Linux, Third Edition

chapter summary continued
Chapter Summary (continued)
  • tr copies data read from the standard input to the standard output, substituting or deleting characters specified
  • sed is a file editor designed to make global changes to large files
  • pr prints the standard output in pages

Guide to UNIX Using Linux, Third Edition

chapter summary continued33
Chapter Summary (continued)
  • The design of a file-processing application reflects what the application needs to produce
  • Use record layout to identify each field by name and data type
  • Shell scripts should contain commands to execute programs and comments to identify and explain the programs

Guide to UNIX Using Linux, Third Edition