Building Command Line Applications With Clojure

March 9, 2012

To be able to package your Clojure application as a single jar and to run it from a script or the command line is a very handy thing. At first it might not be all that apparent how to do this if you are staring to learn Clojure and are happily hacking away in the Repl or evaluating code from Emacs using Slime. But, it is possible and very easy to do. You'll need to be using Leiningen so at this point I'll assume you are.

Java and One-JAR

Back in the day when you wanted to deliver your Java application you had to collect all the dependent Jar files and your own Jar or class files and then come up with a script to kick the whole thing off. You had to make sure things were all in the right place and your Classpath was correct.

Then On-JAR came along. At the time I thought this was fantastic. What it did was provide an Ant task that could package up your code with all the dependent files into a single jar file. You ran your application like so; "java -jar your-app-cmdline.jar". I used it quite often for a number of applications.

Leiningen Uberjar

Clojure, being built on Java would have similar issues if you wanted to package things up yourself and deliver class files and Jars. Nicely, the build tool Leiningen has the 'uberjar' command. This packages your app with it's dependencies into a single stand alone jar just like One-JAR. To enable this you only need to do a few things in your project.

1. Edit project.clj


Continue reading →

Loading Data With Clojure

March 4, 2012

If you are coming to Clojure from Java you are probably familiar with JDBC. If not do a bit of research and you'll find that it is a layer that allows you to program against a database agnostic layer that communicates to your database through a database specific library. In the following example we'll use the clojure.java.jdbc wrapper for JDBC-based access to databases. This library is not part of what you would automatically have when you just install Clojure. For use to use it we'll have to download it ourselves or use a build tool to do it for us. To keep things simple here I'll assume you are using Leiningen as a build tool. If not you can find clojure.java.jdbc on GitHub. Also, you will have to have a database. Here I'll assume MySQL and explain how to connect to that. The example we'll build here will be an enhancement to the stock quote downloader presented in Reading Files article. Instead of writing the quotes to a file we'll load them into a database. You might want to review the article before continuing.

Database

Here we'll use MySQL but you could be using another. The only difference in the following will be the setting up of your database's JDBC library.

Install and Setup JDBC Driver

You'll need to install and reference the MySQL JDBC driver. At this point you should consider using a build tool. Manually you'll need to download the jar file and make sure it is on your class path before starting your Clojure Repl or running Clojure to evaluate your file. Both techniques are worth understanding thoroughly but once you do understand you'll find using a build tool to be more efficient and satisfying.

Manual Install

You'll need to download a copy of the JDBC Driver for MySQL. You can do that on the MySQL Connectors page (http://www.mysql.com/products/connector/). Grab the one for Java. They call it Connector/J. (Download link http://www.mysql.com/downloads/connector/j/)

Inside the package the file you want is the jar named mysql-connector-java-5.1.18-bin.jar. Before you run your Clojure repl you'll want this file on your Classpath.

Using a Build Tool


Continue reading →

Writing Files In Clojure

March 2, 2012

In the previous post we saw how to read files and then introduced a function to write them. The example presented downloaded stock quotes from Yahoo and then writes the data to a local file. Let's look a bit more into writing to files in Clojure.

Writing Short Strings

The opposite of slurp is spit. This is for writing short strings to files. You call the function with the name of the file and the string.

user=> (spit "testfile.txt" "This is my string")

After running the example you should see a file called 'testfile.txt' with the contents of 'This is my string'.

Writing Large Files

To write larger amounts of data we'll need to use a writer. To review, we can use clojure.io.writer to get a java.io.BufferedWriter. This is used within a with-open call. Suppose for the purposes of demonstration we want to create a file that is simply a list of sequencial numbers from 1 to 999 with each number on it's own line. The first thing we'll need is the list of numbers from 1 to 999. This can be gathered with the range function. For example here are the numbers from 1 to 10.

user=> (range 1 10)

We'll need to iterate over the list so lets print them as an example.


Continue reading →

Yahoo Quote Downloader In Clojure

February 26, 2012

Yahoo provides downloadable historical quote data if you exercise a properly formatted URL. The data is returned in CSV format and is easily stored in a file. I wrote quote-downloader to accept stock symbols from the command line and request and store the data in symbol.csv files locally.

The program demonstrates reading from the command line, building a unique URL and reading from it as well as saving the results to a file and

The code is on github here https://github.com/bradlucas/quote-downloader


Continue reading →

Reading Files In Clojure – Part 2

February 22, 2012

In the previous part we saw how to read files and print them. This isn't terribly interesting. Typically, you are going to read files and then do some processing on the contents of the file and possibly save the results. Let's do that in a set of examples.

Functionally

Clojure is a functional language and you'll realize that as you learn the language and start writing functions. There is actually more to it than that but for now remember that you are building your program by creating and composing functions. In a purely functional language each function would not have a side effect. In other words the function would accept parameters and return a result without any change to the environment around it. As soon as we started talking about writing a function to process a file and do something with the contents we introduced the idea of a side effect. In our simple example previously we printed the lines. Next we'll process the lines and write the results to another file.

File Copy and Upper Case Example

How about for a totally contrived example we write a function that takes a file, reads and write a copy where everything is uppercased. We know how to read a file let's learn how to write one.

If we return to the clojure.java.io namespace we'll find there is a writer that wraps a java.io.BufferedWriter. This should work. The following example makes a copy of .bashrc as FOO. A couple things to note. First we need to call the write method on our writer. See that java.io.Writer doc to verify this. Also, see the dot macro in action. To call the write method on our wrt instance we use .write. See Java Interop page on Clojure.org page for more details.

(with-open [rdr (io/reader "/Users/brad/.bashrc")            
            wrt (io/writer "/Users/brad/FOO")]  
    (doseq [line (line-seq rdr)]    
       (.write wrt (str line "\n"))))

Now we need to know how to upper case a string. We'll do this to each string before we write it. The function we'll use is \link{href{http://clojuredocs.org/clojure_core/clojure.string/upper-case}\text{upper-case}} in the clojure.string namespace. To access it use the following in our repl.

(use '[clojure.string :as str])

Continue reading →

Reading Files In Clojure – Part 1

February 20, 2012

There are a few ways to read files in Clojure.

slurp

If you want to simply read a small file into a string then you can use slurp. This function works with local files as well as URLs.

An example:

user=> (print (slurp "/Users/brad/.bashrc"))

clojure.java.io

The clojure.java.io namespace contains a reader function that returns a java.io.Reader. Make sure to call this inside a with-open call.

To use clojure.java.io place the following namespace declaration at the top of your file.

(ns your-namespace-name
   (:require [clojure.java.io :as io]))

Continue reading →

S3cp Now On Github

February 8, 2012

To make it easier for others to work on I've moved my most popular open source project to GitHub.

Already, I've had one pull request with some fixes to allow s3cp to run under Windows more easily.

See the project at https://github.com/bradlucas/s3cp


Continue reading →

Debugging S3 Authorization Issue

May 24, 2011

Recently found an issue with some code running against Amazon’s S3 service. All of a sudden the application, s3cp, was not working as part of our downloader service. The app has been working flawlessly for over a year and suddenly stopped working.

As I tried to debug it I found that the system worked fine locally but not on the server. It was giving an error message around not being able to find one of my accounts buckets. That was odd. First I tried updating the jets3t library used underneath to communicate with S3. The I noticed the server’s Java as bit old but still 1.6. After upgrading both of these still no go.

After taking a break from this I turned on debugging and saw that the authorization interaction with Amazon was returning a 403 error.

To make a long day’s session short for others here is the punch line. Turned out the server’s date was off and Amazon will reject the call if it is too far from the correct time. After setting the servers date and time to the proper value my issue went away.

Sometimes when you debug you need to look at things outside of your code especially if all of a sudden it doesn’t work. In this case the turning point was when the code worked on one machine and not the other. Sadly, we sometimes start upgrading and changing things too much and lose sight of other possibilities.

Of course, if the error message returned by Amazon had more of a specific reason this would have been more quickly solved as well.


Continue reading →

Error NT_STATUS_BAD_NETWORK_NAME printing from Mac to Linux Printer

May 11, 2011

Just fixed. Was getting

Tree connect failed (NTSTATUSBADNETWORKNAME)

when printing from my Mac to a printer hanging off a Ubuntu machine.

Having recently upgraded Ubuntu that was probably the culprit.

This link has more details http://ubuntuforums.org/showthread.php?t=1578421.

The short answer was to restart the following on the Ubuntu machine.

$sudo service cups restart
$sudo service smbd restart

Continue reading →

Website Grader Report for Beaconhill.com

February 22, 2011

https://website.grader.com/results/beaconhill.com


Continue reading →

Blog up and running

February 22, 2011

Getting the blog up and running after a long hiatus


Continue reading →

Ben Engber In The Wsj

April 21, 2010

Ben Engber over at Thumtack was interviewed by the Wall Street Journal.

"The biggest takeaway was that it's a different world than the one I'm used to," Mr. Engber says, adding that government agencies "want a specific service, and have set criteria to evaluate you." With commercial work, "you can go in and explain what you do and why it's superior," he says.

The article: http://online.wsj.com/article/SB10001424052748704671904575194172673354864.html


Continue reading →

New Version Of Jskeleton Is Available

February 7, 2010

Just released an update to JSkeleton. This version 0.1.3 has a minor bug fix in the final renaming of the todo-app.txt and app.el files.

See http://www.beaconhill.com/opensource/jskeleton.html for download links.


Continue reading →

New S3cp Version

February 3, 2010

The new version of s3cp is available. The release is labelled 0.1.9 and includes a number of internal improvements based on feedback from users.

Any existing users of s3cp should upgrade to this version.

The download page is http://www.beaconhill.com/opensource/s3cp.html


Continue reading →

New Version Of S3cp Available

January 21, 2010

The new version of s3cp is available. The release is labelled 0.1.8 and improves on the previous with a bug fix for downloading to a local file.

Any existing users of s3cp should upgrade to this version.

The download page is http://www.beaconhill.com/opensource/s3cp.html


Continue reading →