In the previous part we saw how to read files and print them. This isn't terribly interesting. Typically, you are going to read files and then do some processing on the contents of the file and possibly save the results. Let's do that in a set of examples.
Clojure is a functional language and you'll realize that as you learn the language and start writing functions. There is actually more to it than that but for now remember that you are building your program by creating and composing functions. In a purely functional language each function would not have a side effect. In other words the function would accept parameters and return a result without any change to the environment around it. As soon as we started talking about writing a function to process a file and do something with the contents we introduced the idea of a side effect. In our simple example previously we printed the lines. Next we'll process the lines and write the results to another file.
How about for a totally contrived example we write a function that takes a file, reads and write a copy where everything is uppercased. We know how to read a file let's learn how to write one.
If we return to the clojure.java.io namespace we'll find there is a writer that wraps a java.io.BufferedWriter. This should work. The following example makes a copy of .bashrc as FOO. A couple things to note. First we need to call the write method on our writer. See that java.io.Writer doc to verify this. Also, see the dot macro in action. To call the write method on our wrt instance we use .write. See Java Interop page on Clojure.org page for more details.
(with-open [rdr (io/reader "/Users/brad/.bashrc")
wrt (io/writer "/Users/brad/FOO")]
(doseq [line (line-seq rdr)]
(.write wrt (str line "\n"))))
Now we need to know how to upper case a string. We'll do this to each string before we write it. The function we'll use is \link{href{http://clojuredocs.org/clojure_core/clojure.string/upper-case}\text{upper-case}} in the clojure.string namespace. To access it use the following in our repl.
(use '[clojure.string :as str])
And as a test try the following.
user=> (str/upper-case "this is lower case")
And finally finish up our file copy upper cased routine.
(with-open [rdr (io/reader "/Users/brad/.bashrc")
wrt (io/writer "/Users/brad/FOO")]
(doseq [line (line-seq rdr)]
(.write wrt (str (str/upper-case line) "\n"))))
Reading local files is nice but there is a whole world out on the Internet which is readable as well. Nicely, the reader function can read from URLs so we can slurp or read any URL we have access to. So as a quick example read my web site.
(slurp "http://beaconhill.com")
Slurp is fine for quick things like this but what if we want something a bit more functional. What if for example we want to download stock quotes from Yahoo. For this we need to know the URL format to get all the stock quotes for a given symbol. I happen to know it looks like this.
http://ichart.finance.yahoo.com/table.csv?s=SYMBOL&ignore=.csv
So we'll need a function to build this URL given a stock symbol and then a function to read that URL into a local file. Say we'll save the data to a file with the SYMBOL as it's name.
Before now we've written our routines and run them in the repl. At this point you should learn how to build functions so you can call them from other functions. To do this you'll with a macro called defn. This takes a name and a function definition creates the function and assigns it to your name so you can call it later. Let's start with a function to build a url given a symbol.
(defn build-url [sym]
(str "http://ichart.finance.yahoo.com/table.csv?s=" sym "&ignore=.csv"))
Notice how we can use the str function simply concatenate the parts of the URL we need with the symbol parameter. Next, we should take our earlier example and modify it read the url and write the data to a local file. Let's call the data file SYMBOL.csv. Let's assume our data can be large so slurp is not a good idea. We'll use our reader writer example.
(defn download-historical-quotes [sym]
(let [url (build-url sym)
filename (str sym ".csv")]
(with-open [rdr (io/reader url)
wrt (io/writer filename)]
(doseq [line (line-seq rdr)]
(.write wrt (str line "\n"))))))
In this example, our let block sets up two variables, the url to read from and the filename to write to. The url is created using our build-url function and we create filename by appending .csv to our symbol. Then we create our reader and writers inside a with-open as before. With those we we call line-seq as before on our reader and doseq across the sequence to get each line. Each line is then written using our writer which writes the string to our file.
Try the following.
(download-historical-quotes "goog")
You should see a goog.csv file in your local directory containing all of the vailable Google historical quote data.
See https://github.com/bradlucas/quote-downloader for a working version of the example presented here. As a bonus the uploaded project includes support to run as a standalone program from the command line.