All Jobs Tend To Face Time

June 26, 2017

I've noticed that to some extent all jobs tend to face time. Sure, there are exceptions but there is a tendency.

When you hear things like, "90% of the job is just showing up", there might be a clue there.

Large organizations seem to tend to not trust their employees and in return force a culture of face time because that is the simplest way to hold someone accountable.

"Well, at least he/she shows up" keeps many people employeed.

It's the 10% that I'm interested in. But, is it worth the effort in environments what have lowered the bar of accountability to just existing?


Continue reading →

Yardwork

June 25, 2017

Weekend chore. The Mac has no more space.

$ df -k
Filesystem                1024-blocks       Used Available Capacity    iused     ifree %iused  Mounted on
/dev/disk1                  243924992  219810484  23858508    91%   55016619   5964627   90%   /
devfs                             180        180         0   100%        626         0  100%   /dev

Show directory sizes.

$ du -sh *

Show where the large directories are.

$ du -a . | sort -n -r | head -n 5

Continue reading →

Showip Example In Clojure

June 24, 2017

I found a small example I wrote last year. It shows the ip address of the machine from which the program is running. It does so by calling http://httpbin.org/ip. This url returns the callers IP address in JSON.

The example shows get-ing a url and parsing the JSON.

To access the url I'm using clj-http. This returns a map from which you want the value of :body.

The JSON return value will look like this:

{
    "origin": "8.8.8.8, 72.80.131.168"
}

To parse this I'm using clojure.data.json and it's read-str function which converts the result into a map. Once in a mapp you grab the "origin" value.

Lastly, you want the IP so I split the string on the comma and return the second value.

Code

The main function contains the following:


Continue reading →

Python Virtualenv

June 23, 2017

Introduction

Virtualenv supports the creation of isolated Python environments. This allows you to create your project with all of it's dependencies in one place. Not only does this allow for a simpler deployment path when you release your project but it also makes trying different versions of libraries and experimenting safer.

The following is a good intro for virtualenv.

https://papertrailapp.com/systems/751509772/events

Notes

Step 1 is to install virtualenv. When you run the following you'll be installing virtualenv on your machine.

$ pip install virtualenv

You should be able to execute the command virtualenv afterwards.

When you can you can move on.

Usage


Continue reading →

Building A Weather App In Clojure

June 22, 2017

Overview

I stumbled upon this service called Dark Sky the other day which supports an API to return weather reports. I thought it would be interesting to create a small command line application which would query this API and print out a short summary weather report. Yesterday, I took up the challenge to build such an application in Clojure.

The results are up on GitHub in the following repo.

https://github.com/bradlucas/weather

For those who are interested I'll go through a process to build the application.

Requirements

First, investigate the Dark Sky API. The documention is here:

The main function is here:


Continue reading →

Gist mode for the win

June 21, 2017

If you have a GitHub account you are probably aware of gists. Small mini repos which allow you to send links to their content easily. You can create secret and public gists and I've found them to be excellent for sending examples, lists and notes to others in a more contained format than email.

For example, you create a todo list and want to share it with your team. You create a gist of the outline and send everyone the link. The team can all see the outline, add comments and see updates much like a full git repo. All of this without the full setup of a repo.

Now if you use Emacs there is gist mode which makes creating and editing gists even easier.

The mode is available here https://github.com/defunkt/gist.el.

Once installed and setup try gist-list to see all of your gists. It will look like Dired mode and you'll pretty much know already what to do. To start with an existing gist hit Enter on an entry to edit.

To create a gist start with a local buffer or file. Start there, select the entire buffer (region) and then execute gist-region. This will create a public gist. The related gist-region-private command creates a private gist

There is more you can do so feel free to explore. Oh, and one last tip. Whenever you create a gist you'll find that the resulting url to the gist has been copied to your clipboard so you'll find it pasteable right after you great the gist and need to send the link to it.


Continue reading →

Competition Vs Comparison

June 20, 2017

Competition is something we are all aware of in our society. It is taught to us and bred into us from an early age. In fact you are pretty much setup to think that to get ahead you need to be competitive. Be better than anyone else and you will win. What you'll win isn't really mentioned just the idea of competing with others.

This of course makes sense in sports. You need to compete to win and to win is competing. That's fine. But how about in life? In work? Are you competing with others? I suppose you are in a general sense. You interview for a job and get it. Supposedly there are others wanting the job so you did compete and win in that situation.

But what about in other parts of your life? Are you competing with your friends, your neighbors the guy you just walked by? I don't think so but you might and by this I think you might because you compare.

I think constantly comparing yourself to others is a hidden form of competition that you learned throughout your life. I addition to learning to compete at sports or in school or in other places you also learned through marketing and advertising. Think about all the ads and TV shows that show others in more favorable situations. I say more favorable because the message is typically just that. Look at this person who has something you don't and is so happy because they do. The message is clear in that others have things, good things, expensive things that become things you want. You compare yourself to these other people and that is when you end up feeling bad.

This comparing and then failing is a cycle I think can be broken if you learn to not compete by comparing. Why should I feel bad if so and so has a nice car or house. Why should I compare the way I dress to someone who walked by. Think about this for a moment. What if you didn't compare. What if you either appricated what you had and didn't compare yourself to anyone else. What would you think about instead. How would you feel?


Continue reading →

Escape Games NYC

June 19, 2017

Yesterday, I had the pleasure of trying to escape a room over at Escape Games NYC on Leonard Street in New York. For a first timer at this sort of thing I can say it was well worth it and very fun.

Without giving away too much I can tell you that the idea, 'escape the room', is pretty much what you are in for. You and a small group are put in a room and have to figure out how to get out. Beyond that I can say each room has a theme and you need to just figure it out. There are 'things' in the room and you need to investigate and go from there.

The scenario for the room I signed up for had an outer space theme. There are others and each is rated on a difficulty scale. In learning about the room upon arrival I found that each room is custom built and upon finishing can say the people who built it are very clever.

Turns out that room escape games are a thing and there multiple places in New York and others throughout the country. Definitely, something to do. Which one is next?


Continue reading →

Father's Day

June 18, 2017

Today, our culture where I live celebrates fathers. Today, fathers will be treated a little bit better. They may allow themselves to sleep a bit later and may be absolved of chores for a day. Some may get a gift or two and may end up feeling a little bit special for the day. Others may not.

For me, I found myself this morning remembering the two father figures in my life. The first, my own father and the other, my grandfather who was my mom's father were the two male role models for me growing up.

My dad was very gregarious, at least as I remember him. He was friendly, quick to laugh and had many friends. My grandfather was quiet, stoic and kept to himself. While physically big and strong in a John Wayne sort of way, he could handle himself and generally did things for himself. My dad on the other hand relied on a group of associates and friends to get things done.

Dad

My dad grew up in a semi-wealthy family, went to school, was drafted and returned to work in a bank. There he made is way up and was eventually number two. He seemed to have done well for himself while at the time having a good time. People in the bank liked him. People in the community liked him. He ran the loan department and in a time that was more flexible was able to bend rules to give loans to people who returned the favor many times over as they became the members of a large group of people he helped who would gladly help him.

Being friendly and having a good time was to me my dad's model. He liked sports and played softball with a team at his work. He was the pitcher (of course), was well liked and hung out after games drinking with his work friends. My mom would bring us to games and we knew other families and were friends with the kids of the other families. Everyone had nicknames in my father's circle and when they came by our house for parties and cookouts they all seemed to laugh and have a good time.

My dad's bank was a small regional bank. The kind where they knew every customer and customer knew the people in the bank. The camaraderie at work mimicked the business model and the world around him. This worked until the world changed and banks consolidated. Like a massive wave, a bigger bank bought my dad's bank, the president was 'retired' and my dad was working for a lawyer assigned to merge the smaller bank into the bigger one. Since, my dad was well liked and was basically the operational manager of the bank he was kept for a while as the new president gutted the middle management of the bank. Each and everyone of his friends was let go to a world they were not ready for. Every single one of my dad's friends including himself had spent their entire career at the bank. This did not prepare them for moving on at all. Each had a hard time and most moved away as they gasped for breath to make it till they could retire. My dad hung on until there was nothing left and he found himself at home with nothing to do.

This point coincided with his kids moving on to college and my mom working more. Not the best time for him and I'll skip this part for another day. It doesn't need to be part of my father's day remembrance as I like it more to see the old him having fun with his friends.

Grandfather


Continue reading →

Git Scripts - First Commit Authors

June 17, 2017

Overview

When you work in an organization with a large number of repoositories created over time there is sometimes a need to figure who created what. You want to know who was the author of each repo.

The following three scripts can help with this.

The first get-first-commit-log.sh will show you who was the first committer to a repo. You run it in the directory of a repo.

#!/bin/bash
# @see http://stackoverflow.com/a/5189296

# If the repo has no commits you'll get the following
# fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
# Use '--' to separate paths from revisions, like this:
# 'git <command> [<revision>...] -- [<file>...]'
#
# Using 2> /dev/null to ignore these

git log $(git rev-list --max-parents=0 HEAD 2> /dev/null) 2> /dev/null

This is perfect for a single repo. But, what about when you've cloned each of your organization's repos into a directory. Here, as I write this there are 194 directories in my work directory. To find out the authors of each I could use another script. The following will help. It is called get-all-local-authors.sh.

#!/bin/bash
(for f in `find . -maxdepth 1 -type d`;
 do
   pushd > /dev/null $f
   # only do something if there is a .git directory
    # do something only if there is a .git directory
    if [[ -d ".git" ]]
        then
            get-first-commit-log.sh | grep -m 1 Author: | cut -f2- -d' ' | awk -F'[<>]' '{print $2}'
   fi
   popd > /dev/null
 done) | sort --ignore-case | uniq

But this script produces all the unique repo authors for the organization. This would answer the question, who here has created a repo. How about who and which repo did they create.

For this, we can have a script called get-all-local-repos-authors.sh.

#!/bin/bash
echo ""
printf "| %-55s  | %-40s | \n" "-------------------------------------------------------" "----------------------------------------";
printf "| %-55s  | %-40s | \n" "Repo" "Author";
printf "| %-55s  | %-40s | \n" "-------------------------------------------------------" "----------------------------------------";

for d in `find . -name .git -type d`; do
    pushd $d/.. > /dev/null; 
    # do something only if there is a .git directory
    if [[ -d ".git" ]]
        then
            AUTHOR_EMAIL=`(get-repo-first-author.sh)`
            REPO_NAME=`basename ${PWD}`;
            printf "| %-55s  | %-40s | \n" $REPO_NAME $AUTHOR_EMAIL;
    fi
    popd > /dev/null; 
done
printf "| %-55s  | %-40s | \n" "-------------------------------------------------------" "----------------------------------------";
 

Continue reading →

Up Early New York

June 16, 2017

Up Early

3, 4, 5

Train

Grand Central at 6 am

More people than I'd thought

Outide

Trash still out

Sidewalks getting washed

Construction blocking roads


Continue reading →

Clojure NYC Report

June 15, 2017

Making SQL More Like Datomic With Clojure

Clojure NYC had another great talk last night with Michael Gaare of Ladders introducing his library sqlium. Michael has taken his experience with Datomic and has created a DSL for querying SQL databases with a Dataomic-like flavor.

The source for the library was just put up on GitHub last night at https://github.com/TheLadders/sqlium. It will be interesting to see how the library does as others start using it.

As a next step, it would be good if the source Michael demonstrated last night which used the MusicBrainz data set was published somewhere for review as well.

Background

Michael wrote the Datomic chapter of Professional Clojure

Meetup Details

https://www.meetup.com/Clojure-NYC/events/239281967/


Continue reading →

Release Procedures

June 14, 2017

Some Rules

  • No releases after hours unless previously negotiated
  • Reponsible parties need to be present during the release
    • ie, Developer and Ops
  • Different systems have different release requirements
    • All have dependencies
    • Need to know roll back procedure
    • And, how to know if a roll back is necessary
  • The release may have ‘other‘ tasks. These should be documented and have been previously tested
    • Any env settings, configuration setting changes
    • Database changes
    • Systems that need to be stopped and started as part of the release
    • The ‘recipe‘ should be published
    • The ‘recipe‘ should have been followed in a ‘testing/staging‘ environment

Source Control

  • Use Git Flow
  • Releases are from release/VERSION_NUMBER branches
  • All code should be reviewed if necessary and merged to ‘develop‘ prior to release
    • ie, releases are branched from a stable state
  • Releases are numbered with a x.x.x format
    • Each release should increment intelligently
    • If any confusion get it sorted out ahead of time
  • DevOps will deploy the branch unless previously negotiated

Checklist

  • [ ] All features have been merged back into develop
    • [ ] Merging can be done using Pull Requests for secondary approval
    • [ ] All features have been tested in a Test environment
      • Tests can be done off origin/feature/my-new-feature
      • or if a final feature form a set of features origin/develop
  • [ ] A release branch has been created with the new version number
    • [ ] Release branch is pushed to it's remote origin/release/VERSION_NUMBER
  • [ ] All involved know of the schedule release time
    • [ ] Ops sets a time agreeable to all
    • [ ] Developer will be present or reachable at said time
    • [ ] A rollback procedure has been defined
    • [ ] All dependent changes have been documented. The ‘recipe‘ has been published

Reference

Previously saved as a Gist

https://gist.github.com/bradlucas/2536c5771bb08b7ad149


Continue reading →

Juxt

June 13, 2017

Overview

While cleaning out some old Gists I found a function I wrote six years ago using Clojure's juxt function.

The gist is here:

https://gist.github.com/bradlucas/1233763

To save a click I'll put the function here after fixing the embarassing typo in the comment.

(defn find-new-members
  "Return the new people who are only in the new-list but not in the current-list.
Both lists are lists of maps where they have at least :first_name and :last_name keys.
"
  [current-list new-list]
  (let [existing (set (remove #(nil? (second %)) (map (juxt :first_name :last_name) current-list)))]
    (remove #(existing ((juxt :first_name :last_name) %)) new-list)))

Comments

In reviewing this I remember why I wrote it. I was building some functions to process email lists. The scenario was that there was a current saved list which needed to periodically updated upon receiving a new list. Often the new list was very similar to the old or current list save for a few updates. Identifying the new members from the new list was the purpose of the function.

I notice that in my function I was also concerned about empty last name values in the current list. It looks like I wanted to ignore them for consideration but I wasn't concerned about empty last names in the new list. I'm not sure why.

juxt


Continue reading →

Working With A Local Clojure Project

June 12, 2017

Overview

A common workflow when developing a Clojure project is to leverage one or more libraries written by others. When found, you'll review the main documentation page for a library and learn the information you'll need to add to your project.clj file's :dependencies vector so Leiningen can find and load that library. These libraries will typically come from Clojars but can also be loaded from Maven Central.

Behind the scenes, when you build your project, you should know that your configured libraries will be installed in your local Maven repository. Look inside of the .m2 directory off of your home directory.

So far so good. This will be all you need if you are good using these libraries without modification.

The question does arise though when you want to try changing one of these libraries. How can you set things up to allow this is the question.

Clone the repo

The first step is to clone and build the project you want to work with. Make sure you can do that. Then give this local project a new version number. Here I use a naming convention of setting the major version number to 99. This way when I reference this version in my project I have a version number that really sticks out. Also, I've got a situation where the version number only changes at the major level. I don't have to change the others. Next, run lein install for this library. This will install the 99 version in your local .m2 directory.

Point to the new version

Go back to your project which uses the example project. Make sure it still builds and runs. Then, update your dependencies list and change the project library's required version number to this new 99 version.

At this point you should be all set.


Continue reading →