The other day at lunch, I heard this interesting phrase, "It's an optics game". It was spoken by a friend I was having lunch with. He was detailing his current work environment which is less than positive. The story went like a lot of stories I hear from people in the same company as my friend, in that things are not positive. The company appears to be failing, a lot of people have left, no raises, etc.
My friend's boss was recently working on a series of marketing messages and powerpoint presentations. What made this odd to my friend was that my friend is in BizOps and his manager is in charge of the entire department. Why does BizOps need to market itself was my friends' first thought. Sensing this oddness, my friend asked the boss why he was doing the presentations and got back the answer, "It's an optics game".
Continue reading →Work continues on the Ads.txt crawler with a focus on error handling and reporting. A new version (0.0.2) is not available and it works well against the 'Top 100 domains' file.
One particular issue was found with http-kit. It seems to have difficulty with sites that are SNI-enabled. When such a site is encountered you need a SNI-aware client.
In my top 100 list the url http://elpais.com/ads.txt causes the following error:
Error: javax.net.ssl.SSLException: Received fatal alert: handshake_failure for https://elpais.com/
I recently discovered the Ads.txt specification and the IAB Tech Python crawler. Being more interested in Clojure I decided last weekend to write a crawler for Ads.txt files in Clojure. The first pass is available at the following repo on the release/0.0.1 branch.
Currently as of this writing, the 0.0.1 version supports passing a target domain list to have the sites crawled, their content parsed and output written in comma delimited format to STDOUT. The Python crawler from IAB saves it's output to a SQLite database. I decided that would be a feature for a later release. With my current version it is simple enough to pipe the output to another program or file for subsequent processing.
If you find this post at a later date than the release/0.0.1 branch feel free to investigate any progress I've made.
How to print to STDERR in Clojure came up the other day.
It turns out there are two vars *out* and *err* that are bound to STDOUT and STDERR. There vars are instances of `java.io.PrintWriter' and you can simply write to them.
For example, here is a string being printed to STDOUT.
user> (.println *out* "hello")
hello
nil
Some new Mac notes concerning Python. Out of the box the Mac has Python and when I ran pip list I saw a bunch of stuff that I didn't recognize. When I tried to remove things they looked baked in because they were installed as root.
Turns out you can ignore all that. Just install a new Python using brew.
$ brew install python
This gives you a new Python called Python2. That's not that helpful because who wants to remember to always use Python2?
Continue reading →