Snippets of Useful Information and Tools

Here’s a collection of useful snippets that I’ve picked up or written over the years. Most are small, but in the grand picture, contribute greatly to my productivity.

Shell tips

Searching through history

I’m a command-line junkie, so naturally, I spend quite a bit of time there. One of the best tricks I’ve learned over the years, is to edit your .inputrc file to support searching through history easier. Yeah, yeah, you can hit a couple magical strokes and do it, but that’s not what I’m talking about. I’m talking about typing the first few letters of your command, and then hitting up and down to scroll through every command that starts with those letters. Add the following to your .inputrc:

"\e[A": history-search-backward
"\e[B": history-search-forward

This tip has to be one of my biggest time savers. I feel like I’m stumbling without this feature.

Changing to the top-level folder of my project

I work on lots of stuff, and I constantly find myself needing to crawl back from the depths of a project to the root of the project. I structure my projects so that the top-folder always resides in my home’s project folder. So, my directory structure may look like this:

projects/
    subversion/
    jython/
    django/
    ...

When I’m way down in a tree, I type cdt, and that flings me back to the top of the project. It’s a handy little shell function for bash:

function cdt
{
    local new_dir=`pwd | sed "s|\(.*/projects/[^/]*\).*|\1|"`
    cd $new_dir
}

Subversion tips

Bringing my entire working copy up-to-date

I’m an avid user of Subversion, as are many of the customers that I interact with. One of the most useful—and annoying—things about Subversion is that when you commit, it only bring portions of the tree up-to-date. I like having my working copy all at the same revision, so that make it easier for myself, I have handy alias:

alias svnup='svn up `pwd | sed "s|\(.*/projects/[^/]*\).*|\1|"`'

It works similar to cdt, except it takes to resultant directory and runs “svn up” on it. I should probably make my svn ci just bring the entire tree up-to-date after committing, but there’s enough times when I don’t want that to happen that I haven’t bothered to write said script.

Ignoring .svn folders while grepping

Make sure to include this in your .bashrc:

export GREP_OPTIONS='--exclude=\*.svn\*'

Update:

Use this instead:

export GREP_OPTIONS='--exclude-dir=.svn'

Grep will pick it up and refrain from searching down the .svn working copy admin area.

Generating log messages

Log messages are a big deal when committing your changes. It communicates the intent of the developer, warns others of what has changed, and provides enough context for someone to get started reviewing your code offline. My script doesn’t solve any of those problems, but it does help to format the list of files that have been modified, so that you can get started writing the meat of the message. I find an occasional bug in it, but it’s very useful on the whole. Here’s the script I use:

#!/usr/bin/python
# -*- coding: UTF-8 -*-
import os, string, sys, re
from urllib import unquote

trunk_re = re.compile(r'^(?:URL|Depot).*/(?P<project>[^/]+)/(trunk|wiki)/?(.*)$',
                      re.MULTILINE) 
branch_re = \
  re.compile(r'^(?:URL|Depot).*/([^/]+)/(branches|releases|tags)/[^/]+/?(.*)$',
             re.MULTILINE) 

def get_info(path, use_svk):
  if use_svk:
    info = os.popen("svk info \"" + path + "\"").read()
  else:
    info = os.popen("svn info \"" + path + "\"").read()

  for r in [trunk_re, branch_re]:
    m = r.search(info)
    if m:
      break

  if not m:
    raise "Error trying to obtain path from: svn info %s" % path

  project = m.group(1)
  type = m.group(2)
  repo_path = m.group(3)

  return [project, unquote(repo_path)]

if len(sys.argv) < 2:
    statpath = "."
else:
    for x in sys.argv[1:]:
        os.stat(x)
    statpath = string.join(sys.argv[1:])

# fetch the list of modified data items.
if not os.path.exists('.svn'):
    raise "%s doesn't appear to be an SVN working copy" % statpath
    # possibly an svk working copy... let's check
    p = os.popen("svk st -q " + statpath)
    is_svk = True
else:
    p = os.popen("svn st -q " + statpath)
    is_svk = False
filelist = p.read()
p.close()

# Create a list of filenames/statuses
filelist = filelist.rstrip().split('\n')
modified_list = []

# strip any trailing white space from the entries
for i in xrange(len(filelist)):
  status = filelist[i][0:2]
  if is_svk:
    filename = filelist[i][4:].rstrip()
  else:  
    filename = filelist[i][7:].rstrip()

  # strip off the ./
  if filename[0:2] == './': filename = filename[2:]

  if status.strip():
    modified_list.append([status, filename.strip()])

filelist = modified_list

no_modifiers = re.compile(r"^(subversion|svn|svn-\d+\.\d+\.\d+)$")

print "\n"

for file in filelist:
  [project, file[1]] = get_info(file[1], is_svk)

  if no_modifiers.match(project):
    status = ''
    spacing = ''
  else:
    status = " [" + file[0].replace(' ', '.') + "]"
    spacing = ' '
  print spacing + "* " + file[1] + status;
  print "\n"

Or you can download it here.

Linkinus

Ignoring join/part messages

defaults write com.conceitedsoftware.Linkinus ignoreJoinPart 1

Running a Mac OS X fat binary for a particular architecture

I found this out while rooting around trying to determine how to get PyQt4 working on my laptop. The short form is that the Qt libraries weren’t (at the time) built with 64-bit support, but Snow Leopard wanted to run my apps in 64-bit mode. In the end, it took one small change to how my app was launched. You have to do it through the arch tool. For instance, to fire up python2.6 in 32-bit mode, run:

arch -i386 python2.6

Interacting with Keychains from the command line

You can use the security command to manipulate keychains, and to set and retrieve passwords from the command line. Here are a couple of links pointing to how to use it:

• http://ttt.ggnore.net/2007/01/18/neat-stuff-the-os-x-keychain-12/
• http://blog.macromates.com/2006/keychain-access-from-shell/

The man page is also useful: security(1).

Dump DNS from the Mac OS X command line

scutil --dns

Disable user list in Fedora

sudo -u gdm gconftool-2 --type bool \
    --set /apps/gdm/simple-greeter/disable_user_list true

Breaking out of a stuck SSH session

Turns out SSH has an escape sequence, just like telnet’s ctrl-]. In ssh, it’s simply ~, but only after a newline. Here’s the output from ~?:

Supported escape sequences:
  ~.  - terminate connection (and any multiplexed sessions)
  ~B  - send a BREAK to the remote system
  ~C  - open a command line
  ~R  - Request rekey (SSH protocol 2 only)
  ~^Z - suspend ssh
  ~#  - list forwarded connections
  ~&  - background ssh (when waiting for connections to terminate)
  ~?  - this message
  ~~  - send the escape character by typing it twice
(Note that escapes are only recognized immediately after newline.)

So if your session gets stuck, hit Enter, and type ~.. Then you’ll have control of your terminal again.

Finding which package has a file in Ubuntu

There’s a nify tool called apt-file which can be used to find which package provides a file. To install it, simply do:

sudo apt-get install apt-file

You’ll need to fetch the file data from the apt repositories. Do that by running:

sudo apt-file update

Once that’s complete, you can do things like:

apt-file search add-apt-repository

Perl locale errors when using Git

I ran across this recently where I was cloning a repository and I would see the following:

Cloning into szakmeister-site...
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
    LANGUAGE = (unset),
    LC_ALL = (unset),
    LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
remote: Counting objects: 2185, done.
remote: Compressing objects: 100% (972/972), done.
remote: Total 2185 (delta 1155), reused 2185 (delta 1155)
Receiving objects: 100% (2185/2185), 1.19 MiB | 1.07 MiB/s, done.
Resolving deltas: 100% (1155/1155), done.

There is a fair amount of information out there in Google-land. Most recommend making sure you have a language pack installed, and locales are up-to-date. That translates into the following for me:

apt-get install language-pack-en
dpkg-reconfigure locales

Well, it didn’t fix my problem. It turns out, that it was the remote machine that had the issue! So I installed the language pack on the remote machine, and that got rid of the error message.

Helpful Ubuntu PPAs

add-apt-repository ppa:git-core/ppa
add-apt-repository ppa:nginx/stable
add-apt-repository ppa:chris-lea/python-django
add-apt-repository ppa:uwsgi/release