Into Java - Part XIV

By Simon Grönlund

Into Java, Part 14
Streams are important data structures to know of. Not long after the first computers were powered on, directly controlled by switches, terminals cabled to the machines were introduced. Over time, keyboards of many different kinds surfaced and needed to be plugged into the computer, and devices like video screens also needed connections. Today we use files on disks or other media, networks, radio links and quite a few more techniques to receive or send information.

Every time we use some kind of data stream, it is simply a sequence of bytes or characters. Basically any kind of stream is handled the same way, there is no big difference between streaming audio data and a text file stream, except that the text file is locally stored and the audio may spring from a live concert somewhere.

The four basic stream classes in Java effectively shield you from the exact mechanism of how a stream is read or written, mainly because the actual functions differ for source and target types. The most important thing to us is that we can read or write to a stream.

That hiding of the low level implementation is not only convenient to us, it also gives the low level guys and girls the freedom to change their implementation, as long as the interface, the API, is untouched. Also, new technology can be offered without any need to change our application.

We will however look at what a basic stream is and what we can do with it. Then we will add the convenience classes of Java to the stream and make use of a few such classes. Both input and output streams in Java have a veritable zoo of different add-on classes to pick from.

Further we will use some data structures. Unlike the many data structures added to Java 2, streams have been integral since Java version 1. Finally we will learn another term, abstract, and what that implies and means.

A stream is a sequence of bytes or characters. Input streams are read from and output streams are written to. Historically 8-bit bytes are used, but today the 16-bit Unicode characters are becoming more popular and hence Java offers special classes for handling these characters transparently to the programmer.

Consider a pipe providing one byte, either from time to time as from a keyboard that is used infrequently, or at a high speed from reading a locally stored file. In both cases, there is only one byte at a time provided, you have to remove it to give room for the next one. That is what the most basic stream looks like, which leads us to the InputStream. The OutputStream is the exact opposite, a pipe that can take one byte at a time, but we will come to that class in a moment.

InputStream and Reader
As both the byte streams (InputStream) and the character streams (Reader) provide almost the same functionality, I discuss them together. Later we will see how the two groups differ in usage. A basic input stream provides these methods: void close                 // abstract in Reader void mark int read                   // abstract in InputStream int read(char[] buf) int read(char[] buf, int offset, int length) // abstract in Reader void reset void skip(long n) A method may not be implemented in a specific class and then be declared abstract, as public abstract int read That makes the entire class abstract. In fact such a class is more of a framework, and in some aspects it is close to an interface. But where interfaces only declare the methods to be used, an abstract class may have one or several methods that in fact are implemented, and some that are not.

Abstract classes cannot be instantiated, thus they are always subclassed and the subclass(es) then implement the missing method bodies as they want to. This is quite close to overriding methods, only this time implementation is most certainly needed. Such classes may also act as a label on certain available streams so that we know what we can expect from them.

Why is this? It is obvious when looking at the differences between data streams and text streams, one is using byte and the other char, which leads to two different implementations.

Still, any non abstract java.io subclass of InputStream</tt> and Reader</tt>, has the methods mentioned!

In addition to these there are a few methods that differ between the classes. The most important methods are the read</tt> methods, the basic one returning an int</tt> (I presume an int</tt> is chosen since Reader</tt> may return an int</tt> in the range of 0 to 65535 (0x00-0xffff</tt>), or -1 if the end of the stream is found), and the other ones filling a provided char</tt> array with characters one at a time.

reset</tt> is used to start over from the place mark</tt> was used to put kind of a "book-mark", if the stream provides "book-marks". We may also skip(long n)</tt> bytes or characters if we want to. When finished reading the stream we close</tt> it so the system resources will be returned.

These methods differ in InputStream</tt> and Reader</tt> only in the former processing bytes and the latter characters. The characters are represented internally within your Java application in Unicode format, but externally the encoding depends on the underlying system and the actual stream processed.

Now an observation: read</tt> is a rather dumb method, it will sit and wait at the end of the input stream for more data. At least until an <tt>IOException</tt> arrives. Hence, if you read from a stream, your application will freeze if the stream stops for a while, as a stream over the Internet may do. Later we will see how to take care of that.

OutputStream and Writer
As input streams had their common methods the output streams do as well where x is <tt>byte</tt> or <tt>char</tt> respectively: void close                      // abstract in Writer void flush                      // abstract in Writer void write(int x)                 // abstract in OutputStream void write(x[] x) void write(x[] x, int off, int len)     // abstract in Writer void write(String str)                  // Writer only void write(String str, int off, int len) // Writer only The <tt>write(...)</tt> methods are self-explanatory, as is <tt>close</tt>. <tt>flush</tt> tells the system to write the data right away, if buffering is used.

Three pipes from the <tt>System</tt> class are always available, <tt>System.in, System.out </tt> and <tt>System.err</tt>. The former two we have used, especially <tt>System.out</tt>. <tt>System.in</tt> is an <tt>InputStream</tt> while the latter two are of the <tt>OutputStream</tt> type.

However, not all of the methods mentioned so far are implemented, see the abstract box. All these methods will be implemented by the classes we wrap these basic streams up in. Hence we may consider these basic stream classes a plain pipe, and we have to wrap it up in convenience classes if we do not want to do a lot of tiresome coding ourselves.

Convenience classes
Below is a table of most of the IO-classes. Remember that a subclass also has the properties of its superclass(es). That is for example, <tt>BufferedInputStream</tt> is a <tt>FilterInputStream</tt>, that is an <tt>InputStream</tt>, that is an <tt>Object</tt>. Hence you may for example use an <tt>InputStream</tt> reference from somewhere. And from that abstract object we may make ourselves an <tt>InputStreamReader</tt> that is a <tt>Reader</tt> so we can make a <tt>BufferedReader</tt>.

I have coloured the base IO classes so it can be easily seen where they may be used in constructors. Let us start with <tt>File</tt>, a handy object that represents either a file path and file name, or only a path to a directory. Its implementation differs from one operating system to another, as the Unix path separator / is represented with \ on an OS/2 machine. To play safe, we use the <tt>File.separator</tt> static variable when writing paths. <tt>File</tt> is constructed with only a file name, a complete path, or a path as one string and the file name as another string. The instantiated object can now answer many questions, such as <tt>canRead, canWrite, exists, isDirectory,</tt> etc. Hence, if you have to create a file yourself, it may be a good idea to instantiate a <tt>File</tt> object.

Since <tt>read</tt> can only read one data packet a time, sitting in a loop until maybe the end-of-line (EOL) character arrives, it seems convenient to use classes that have methods like <tt>readLine. BufferedReader</tt> and <tt>LineNumberReader</tt> have such methods, thus when reading text files, one of these two classes is most often used, mainly the <tt>BufferedReader</tt>. We used such a reader in Into Java 4 and No 5 and we will use <tt>BufferedReader</tt> today too.

Word frequency counter
We will make ourselves an application that can count how many times each of the words of any provided text files are used. Hence we need to read a text file. We will use a data structure that can help us find equal words and we will write the result to a file.

A speedy data structure that provides a way to tell the difference between objects is <tt>Hashtable</tt>, two equal words will produce the same hash code, and we may use <tt>containsKey(Object key)</tt> to find out if a word is used more than once. But where do we store the hits since a <tt>String</tt> cannot hold hits? It looks like we have to make ourselves a helper class that holds one word and a counter. Let us start with that one.

Since that helper class is to be used in a <tt>Hashtable</tt> it must implement <tt>hashCode</tt> and <tt>equals</tt>. On the other hand, since we work with <tt>String</tt> objects, we may use the methods of that class and we will just do a call-back to the <tt>String</tt> class' methods on each of the methods mentioned. (If we use Java 2 and would like to get a sorted output, we must implement the java.lang.Comparable interface, and that has only one method to implement, compareTo(Object other). That is because the static Collections.sort method demands that the objects to be sorted support that interface.)



Having this helper class we may continue with the WordCounter class. We will settle for a tiny terminal window version, although it could be increased into a GUI application, using this class as an invisible engine. The class we are making shall have a <tt>Hashtable</tt>, hence we must import the <tt>java.util</tt> package. We must also import <tt>java.io</tt> to get the file readers.



I think I have mentioned that there are system dependent characters and <tt>line.separator</tt> is one. A way to support system independence in Java is to use the variables available through the <tt>System.getProperty</tt> method. There is a list of such variables near that method in the Java API. The first line of <tt>main</tt> makes it valid.

The next task is to get the filename from the input argument, we must assure ourselves that there is a parameter to read, else we notify the user and quit automatically. Once we have a valid input, we instantiate an object of the <tt>WordCount</tt> class type, passing the input argument as a parameter to the constructor. So far we know that we have a valid argument from the user, but wait, didn't we use <tt>throws IOException</tt> the last time we worked with file reading?

This time we instantiate a handle to the file we want to access even though we are not yet certain if there is a file to read from. Fortunately <tt>File</tt> is an abstract handle to a file and does not need a file, hence we may create ourselves a file handle to use. The next step is to ask the handle if the file <tt>exists</tt>, and if not, tell the user and exit.

A small note, last time I mentioned briefly that using primes as the starting value for <tt>Hashtable</tt> gives better results. I will not argue further on that but please note that I looked one up and am using that prime, 2671, and that will be used for a while. Naturally, if you would like to count a huge file, you will need to increase this to a much bigger prime.

So far you may compile without error and try the error messages. Nothing else will work since we have not done anything to the <tt>count</tt> method.



We chose to have the <tt>count</tt> method public, thus it may be used by GUI apps. (To do this, you would instantiate an object with the file name as parameter and call <tt>count</tt>. Unfortunately I will not make it quite that easy, you will have to change some lines to redirect the output to a text area.) Now we need a <tt>try/catch</tt> block since <tt>readLine</tt> may go weird, and if so, an error message needs to be printed. This time we use the <tt>System.err</tt> that is the standard pipe to print error messages to. At this point it is still equal to <tt>System.out</tt>, but it might very well be redirected to a log file or any other stream.

We use the <tt>File</tt> handle to make a <tt>FileReader</tt>, which is used to make a <tt>BufferedReader</tt>, a convenience class that has some useful methods like <tt>readLine</tt>, which are preferred to the low level methods of <tt>FileReader</tt>. Next, we start reading the file line by line as long as there are more lines to read. Please, note the parentheses in the <tt>while</tt> clause.

Every line read is sent to <tt>processLine</tt> which simply wraps a <tt>StringTokenizer</tt> around the line. This useful piece of code is located in <tt>java.util</tt> and gives you tokens delimited by blanks, or any other whitespace, if you do not specify your own delimiters. As long as the line has more tokens (words) the <tt>while</tt> loop continues. Finally we <tt>close</tt> the file, although this time it is not strictly needed since we are only playing with it, in the future you might be working on networks where you should be more polite.

Recall that a <tt>Hashtable</tt> needs two things, a <tt>key</tt> to map from, and the <tt>value</tt> to store. They need not be the same thing, as in this case. We use the words as keys to the table, but we store the <tt>Word</tt> instances made out of the actual tokens.

We use the token as a <tt>key</tt> and want to know if it is already stored in the table, if so we increment the word count. Else we make a new <tt>Word</tt> instance, use the token as <tt>key</tt> and <tt>put</tt> the object as <tt>value</tt> in the table. In the end, the file will be read and all the words <tt>put</tt> into the table and counted.

Now we will continue with the "more code to come" part. What to do with the output? I have made two versions available, one for Java version 2 (that is Java 1.2 and above) and the one actually used here for the prior versions. If you use a later flavour of Java, please remove the appropriate lines and make a few changes to the code as explained, both in this class and in the <tt>Word</tt> class where <tt>implements Comparable</tt> must be visible.

This time we get ourselves an <tt>Enumeration</tt>, that is an abstraction of any data structure that is enumerable. This interface has two methods, <tt>public boolean hasMoreElements</tt> and <tt>public Object nextElement</tt> that operate on the underlying structure.

Using such interfaces hides the actual data structure and you may conveniently change from a <tt>Vector</tt> to an <tt>ArrayList</tt> (Java 1.2) without too much work. For example, there is not a single line to be changed in the <tt>while</tt> loop, <tt>Enumeration</tt> still works.



The very next thing is to get ourselves an output filename. We use a <tt>StringBuffer</tt> which does not cause any overhead to the JVM (as concatenating and mixing with <tt>String</tt>s do) and it has many useful methods. Since we do not know if we can lengthen the filename (maybe you use 8+3 FAT) we alter the first two characters. Another option would be to change the file extension, if we know there is one.

Another <tt>try/catch</tt> block encompasses a new <tt>FileWriter</tt> that is set to not append to an existing file if there is one. For example, if you have a log file that is added to once in a while, you instantiate a <tt>FileWriter</tt> object with boolean <tt>true</tt>, and it will append to the end of the existing log file.

The while loop is mainly self explanatory due to the narrative method names. Please note how convenient the <tt>toString</tt> method may be from time to time. <tt>toString</tt> is not only a good debugging method, it can serve our purposes this time as well. It does not provide splendid output but it is speedily implemented.

Finally the output file is closed and the <tt>count</tt> method is finished. Compile and go for it. Optionally you may add a System.out.println("The table size is: " + table.size); to the method and you will see how close to the prime we got.

Summary
This time we used a speedy data structure that is good for quick input and lookup. The object was to count words, and we expected to find a lot of duplicates and in fact, we were not interested in a sorted structure. At least not from the beginning. Hashtable is very good for such tasks. Java 2 offers a method to sort our data structures as well.

The <tt>put</tt> and lookup (<tt>containsKey</tt>) do not depend on the size of the table, the time is constant with increasing sizes.

Further we found that is was not hard to get a list of the contents, we used an <tt>Enumeration</tt> and found a nice interface, having only two methods.

We have touched on the useful <tt>File</tt> class, a handle to a file, existing or not. Instances from <tt>File</tt> may be used with some other classes, as we did.

<tt>BufferedReader</tt> encapsulating a <tt>FileReader</tt> showed itself to contain many useful methods. Reading a stream always follows the same pattern:
 * open the stream
 * read some data
 * process the data
 * repeat until no more data is available
 * close the stream

Finally we used <tt>FileWriter</tt> to print the results, and as with input streams, the output streams follow the same procedure:
 * open the stream
 * write some data
 * repeat until finished
 * close the stream

Next time we will continue with streams and look into Exceptions. We have used the <tt>try/catch</tt> mechanism before but there is more to say about that. Exception handling is one of the bigger strengths of Java and using them in a good, sensible way may makes your apps reliable and robust.

I must also announce that I have to make this column much shorter in the future, mostly because my time is limited and I do not have unlimited strength. Still I will try to do my best, and I hope you enjoy future instalments.