Principles of Database Systems With Internet and Java Applications

Principles of Database SystemsWith Internet and Java Applications Today’s TopicChapter 11 Managing Information in Files Instructor’s name and information goes here Please see the notes pages for more information.

Chap. 11 Managing Information in Files • Hardware Characteristics of File Systems • Hierarchy of storage • Disk drives • Hardware architecture of storage systems • Software architecture of storage systems • Manipulating Streams and Files in Java • Java classes for input and output • Readers and writers • File classes • Methods for Representing Values in Streams • Length-based fields • Fixed-length fields

Fundamental File Structure Concepts • Information is stored in application memory • Information is made persistent in files • The format of information in memory is different from its format in files • Examples of changes in format • Pointers (references in Java) have no information content • Access to files is in large chunks (pages) • cannot read or write a byte or word • The performance of applications is determined by the file performance

Disk performance, 1998 and 2000 • Capacity Seagate 8 GB (Seagate 73) • 512 bytes per sector • 170 sectors per track (423) • 16 tracks per cylinder (24) • 6526 Cylinders (14,100) • Speed • Latency • seek min 0.78 (.6), max 19, avg 8 msec (6) • rotational delay? see spindle speed (3 msec) • Transfer rate • 10,000 rpm spindle speed • Maximum transfer rate? see above • 6 msec/track, 14 KB/msec (24 KB/msec) • IBM Travelstar 6, 4200 rpm, laptop • 268 sectors/track, 2 platters, 11,648 cylinders

Physical organization of hardware • Disk drive • Disk controller, attached to drive • buffer of data to be moved to/from • controls head position • transfer of data when sector is under head • I/O controller, attached to memory bus • DMA, direct memory access • moves data from memory to/from disk controller buffer • Memory • Contains application data and I/O buffers • CPU • moves data to/from I/O buffers

Journey of a Byte, software actions • Program asks to write contents of variable to file Text • Application code asks OS to write • memory address and byte count • Operating system passes job to the file manager • File manager identifies file in a file table • File manager finds physical address of sector • File manager makes sure the sector is in buffer • Data is moved from application memory to I/O buffer • OS asks I/O controller to write to specific disk sector • I/O controller schedules the multiple accesses • I/O controller moves sector data to disk buffer • Disk controller moves disk heads • When the sector passes under the head, the I/O controller writes to the disk surface

Java Classes for Input and Output • See JDK (Java Development Kit) API (application programming interface) • http://java.sun.com/products/jdk/1.1/docs/api/packages.html • class InputStream { public abstract int read() throws IOException; public int read (byte[]) throws IOException; public long skip(long n) throws IOException; public int available() throws IOException; • class PrintStream extends OutputStream { public void print(String); public void println(String); public void print(int); public void print(byte); • Notice • overloading, arrays, strings, exceptions

Samples from Chapter 11 • Look at examples in package dbjava.files • class Video in Video.java • Video has main method • notice use of ‘static’, argument list • Video supports reading and writing in a variety of styles • print, printLabeled: simple text • read, write: length-based • readObject, writeObject: serializable

Simple Text Output and Input • Labeled text output, one label and field per line • public void printLabeled (PrintStream out) { out.println ("name\t"+name); out.println ("id\t"+id); out.println ("genre\t"+genre); • Look at result using “od -xc -A x video.txt • Reader and Writer classes • take advantage of Unicode characters (2 byte) • Readers support readLine • Unlabeled text input, one field per line • public void read (BufferedReader in) throws IOException { // read a Video object into the calling object name = in.readLine(); id = Integer.parseInt(in.readLine()); genre = in.readLine();

Java Exceptions and Exception Handling • Java supports exceptions with • exception class definitions class IOException extends Exception {… }; • throw statement, signals exceptional situation • try/catch blocks, contain and handle exceptions • Some methods may throw exceptions: • FileInputStream(String name) throws IOException • If there is no such file, or it cannot be opened • Sample from java.io try {file = new FileInputStream(“video.txt”);} catch (IOException e) { e.printStackTrace (System.err); }

Storing Objects in Files • Class Video {String name; int id; String genre;} • Representation in memory uses memory addresses (picture on board) • Representation in file must be different • Memory addresses have no meaning after program quits execution • File representation must preserve meaning • Length-based Fields • Store String as length plus value • 12Men in Black0312313action comedy

Data Input and Output Streams class DataOutputStream {public final void writeBoolean (boolean v) throws IOException; public final void writeByte (int v) throws IOException; public final void writeShort (int v) throws IOException; public final void writeInt (int v) throws IOException; public final void writeLong (long v) throws IOException; public final void writeFloat (float v) throws IOException; • Features • Writes binary values of Java values • Platform independent • See methods read and write of class Video

Write Length-based Fields in Binary public class Video { … public void write(DataOutputStream out) throws IOException { // write object as length-based and // fixed-length fields out.writeShort(name.length()); out.writeBytes(name); out.writeInt(id); out.writeShort(genre.length()); out.writeBytes(genre); } }; • Look at output with od (octal dump)

Class FieldOps class FieldOps { // a class that supports static operations on length-based fields public static String readLength(DataInput in) throws IOException {...} public static void writeLength(DataOutput out, String str) throws IOException{...} • Class FieldOps encapsulates the representation of a String value as length-based field • uses static methods • Consider • class FieldInputStream extends DataInputStream • this causes big problems

Styles of Representing Fields • Delimited text • text followed by delimiter • transform value into text representation • eliminate delimiter ambiguity • Length-based binary • binary length field • block of bytes, uninterpreted • determine length of block, write length first • Fixed-length • block of bytes of known length • must know length to read field • Identified field • name, delimiter, value, delimiter

Using Files in Java • File classes • class FileInputStream extends InputStream • class FileReader extends Reader • new FileInputStream (“video.txt”); • opens a file named “video.txt” in input mode • creates a connection with OS file • Method close breaks connection between stream and OS file • To create a data stream from a file • new DataInputStream(new FileInputStream(…));

Java Serializable • Java supports fully self-describing files of objects • Two data structures make this possible • interface Serializable {}; • no methods, so no effort for programmer • class Video implements Serializable { ... • classes ObjectInputStream, and ObjectOutputStream • Video v; ObjectInputStream in; • in.read (v); // gets an object from object stream • Sample in class Video • java dbjava.app_a.Video owrite video.obj • analyze program execution and output

Octal dump of Video.obj 0000000 aced 0005 7372 0005 5669 6465 6fbc cd0b ¬ í \0 005 s r \0 005 V i d e o ¼ Í \v 0000010 077f 1bda ba02 0003 4900 0269 644c 0005 \a 177 033 Ú º 002 \0 003 I \0 002 i d L \0 005 0000020 6765 6e72 6574 0012 4c6a 6176 612f 6c61 g e n r e t \0 022 L j a v a / l a 0000030 6e67 2f53 7472 696e 673b 4c00 046e 616d n g / S t r i n g ; L \0 004 n a m 0000040 6574 0012 4c6a 6176 612f 6c61 6e67 2f53 e t \0 022 L j a v a / l a n g / S 0000050 7472 696e 673b 7870 0000 007b 74000d61 t r i n g ; x p \0 \0 \0 { t \0 \r a 0000060 6374 696f 6e20 636f 6d65 6479 7400 0c4d c t i o n c o m e d y t \0 \f M 0000070 656e 2069 6e20 426c 6163 6b00 e n i n B l a c k Note self-describing with type defs, fixed length and length-based fields

Files of Records (Section 11.6) • Unit of access for applications is the record • logical record: • storage for a single object (or collection) • physical record: unit of physical access to file • Usually enforced by OS, not application • Possible organizations • Fixed-length records • each record the same size • binary search available if sorted • Variable-length records • size depends on information content • sequential search only

Evaluation of Fixed-Length Record Organization • Consider fixed-length sorted records • binary search, cost log2n • 20 reads for million records • Is this good? • Insertion • new first record, move every record • This is terrible • Hence, no advantage to fixed length

Variable-Sized Records • How to know which bytes are part of a record? • Use length-based representation • 2-byte length of record • bytes of record • Read of length-based record, load into object • Read length • Read bytes into buffer • Unpack buffer into object • Write of length-based • pack object into buffer • write length, write bytes from buffer

Java for Variable-Length Records • Explained in class using Java code • Major problem: how to pack and unpack buffers? • Java class ByteArrayOutputStream • write to byte array using stream operations • class Video supports write to DataOutput • bufferStr = new DataOutputStream (new ByteArrayOutputStream()) • v.write(bufferStr); // pack Video object into bytes • byte[] buffer = bufferStr.toByteArray(); • out.writeShort(buffer.length); • out.writeBytes(buffer);

Random Access Files • Class RandomAccessFile • includes read, write, and seeking • implements DataInput, DataOutput • hence acts like a data input stream and a data output stream • class RecordFile • part of package dbjava.app_a • Supports read and write of any record that supports read and write to DataInput and DataOutput • See interface InputOutputRecord

Database Information in Files • Each row of a relational table is a record • Sample in class of class definition for table • Each relational table is a collection of records • A file is a collection of records • Need a way to translate from record in memory to record in file

Principles of Database Systems With Internet and Java Applications