Command-line tools on Mac OS X, Snow Leopard

allmemory, fs_usage, fuser, latency, sar, sc_usage, spindump: Display various system usage statistics.

bless, disklabel, disktool, diskutil, drutil, fsck, hdiutil, pdisk: Create, identify, manage, and fix Mac OS X disks, file systems, and disk images.

ioreg, kextstat: Show device drivers and other kernel extensions in use.

mDNS, dns-sd: Test Bonjour service discovery with these diagnostic tools.

open: Invoke launch services on an arbitrary document or application as if doubleclicking it in the Finder.

pbcopy, pbpaste: Move data between stdin/stdout and the Mac OS X pasteboard.

say: Convert text to audio output or a file via Speech Synthesizer.

shasum: Compute or validate SHA message digests.

sips: Manipulate the format and color space of bitmap images (for example, rotate,

sw_vers, uname: Display Mac OS X version information.

syslog and logger: Send, view, and manage system log messages with these modern

xattr: List, display, set, and delete file system extended attributes

You can find more information on apple.com in this documentation.

snoop file opens on Snow Leopard

Opensnoop tracks file opens.
sudo opensnoop

or you can watch only a process
sudo opensnoop -n Google Chrome

HTML5 heading elements

"In HTML 4, the only way to create a document outline was with the <h1>–<h6> elements. If you only wanted one root node in your outline, you had to limit yourself to one <h1> in your markup.
But the HTML5 specification defines an algorithm for generating a document outline that incorporates the new semantic elements in HTML5.
The HTML5 algorithm says that an <article> element creates a new section, that is, a new node in the document outline. And in HTML5, each section can have its own <h1> element."

more of Pilgrimさん's Dive into HTML5
or check the outline of your document with the HTML5 Outliner

activemq and the java.io.EOFException: Chunk stream does not exist at page: 0

If you start activemq and get an error like that

INFO | Scheduler using directory: activemq-data/localhost/scheduler
ERROR | Failed to start ActiveMQ JMS Message Broker. Reason: java.io.EOFException: Chunk stream does not exist at page: 0
java.io.EOFException: Chunk stream does not exist at page: 0

It means that there is something wrong with the scheduler.

If you don't use the built in persisten scheduler you can easily turn it off by changing the activemq.xml config file (/opt/local/share/java/activemq/conf/activemq.xml)

<broker xmlns="http://activemq.apache.org/schema/core" ....
to
<broker xmlns="http://activemq.apache.org/schema/core" schedulerSupport="false">

Open a new tab with AppleScript in Chrome

tell application "Google Chrome"
 set myTab to make new tab at end of tabs of window 1
 set URL of myTab to "http://google.com"
end tell

more AppleScript Chrome goodies at http://laclefyoshi.blogspot.com/2010/10/google-chrome-ver.html

Get user id on a remote server

you have to use SSH public/private key pairs to make this work.

#!/bin/bash
uid=`ssh <your.server> 'id -u'
echo $uid

more stuff here http://bashcurescancer.com/run_remote_commands_with_ssh.html

Google App Engine Development Console from command line

When you develop on localhost the interactive console that Google provides is not convenient.
In my project I had to run few scripts in the console quite often, so instead of going to the interactive console all the time I could do it from command line.

This example executes the 'gen_pilot_data.py' script on the interactive console.

curl --data-urlencode "code=`cat src/gen_pilot_data.py`" http://localhost:8079/_ah/admin/interactive/execute

Please notice that you may have to the change the port number.

Latitude and longitude of cities

http://www.getlatlon.com

 cities = [
  { 'name': 'Tokyo', 'lat': 35.6894875, 'lon': 139.6917064 },
  { 'name': 'Seoul', 'lat': 37.566535, 'lon': 126.9779692 },
  { 'name': 'Beijing', 'lat': 39.904214, 'lon': 116.407413 },
  { 'name': 'New Delhi', 'lat': 28.635308, 'lon': 77.22496 },
  { 'name': 'Moscow', 'lat': 55.755786, 'lon': 37.617633 },
  { 'name': 'Kiev', 'lat': 50.45, 'lon': 30.5233333 },
  { 'name': 'Budapest', 'lat': 47.4984056, 'lon': 19.0407578 },
  { 'name': 'Berlin', 'lat': 52.5234051, 'lon': 13.4113999 },

  { 'name': 'Paris', 'lat': 48.8566667, 'lon': 2.3509871 },
  { 'name': 'Madrid', 'lat': 40.4166909, 'lon': -3.7003454 },
  { 'name': 'New York', 'lat': 40.7143528, 'lon': -74.0059731 },

  { 'name': 'Mexico City', 'lat': 19.4270499, 'lon': -99.1275711 },
  { 'name': 'Honolulu', 'lat': 21.3069444, 'lon': -157.8583333 },

  { 'name': 'Sydney', 'lat': -33.8599722, 'lon': 151.2111111 },
  { 'name': 'Taipei', 'lat': 25.091075, 'lon': 121.5598345 }
 ]

Zoom to fit all markers with Google Maps V3

// creating the map
var map = new google.maps.Map(document.getElementById("map_canvas"), {
  mapTypeId: google.maps.MapTypeId.ROADMAP
});

// latitude, longitude values
var latLons = [ { lat: 35.6453962, lon: 139.7117893 }, 
                           { lat: 35.645076, lon: 139.709183 }  ];

// this is the bounding box container
var bounds = new google.maps.LatLngBounds();

// iterating through the points
latLons.forEach( function (element, index, array) {

  var point = new google.maps.LatLng(element.lat,element.lon);
  
  // extending the bounding box
  bounds.extend(point);

  // creating the marker on the map
  var marker = new google.maps.Marker({
        position: point,
        map: map
    });
});

// zooming on the map
map.fitBounds(bounds);

Reload the browser with AppleScript

This post is obsoleted, here is the new simplified version.

AppleScript is a weird language but you still can do cool things with it.

tell application "Google Chrome"
        activate
end tell

tell application "System Events"
 tell process "Google Chrome"
  keystroke "r" using {command down}
 end tell
end tell

Please feel free to replace "Google Chrome" with your choice.

https://github.com/yikulju/MacGyver/blob/master/reload-browser.applescript

removing trailing whitespace with sed


sed -i '' -e's/[ \t]*$//' $1


the full script can be found here
https://github.com/yikulju/MacGyver/blob/master/rwh.sh

is activemq running?

Activemq is listening on the 61616 port so a simple telnet can help us.

telnet localhost 61616

If you get a response that starts with '?ActiveMQ?' it means that activemq is running.

Simulating cellular network bandwith

To turn it on

sudo su
ipfw add pipe 1 src-port http
ipfw pipe 1 config delay 200 bw 700kbit/s


To reset

ipfw flush


Infinite blinking effect with jQuery

I'm not sure this is the shortest way to do this but it works.

setInterval( function () {

// this is the element you want to blink
var box = $(".blinking_thing");

if (box.data("fade") === 1) {
box.fadeIn().data("fade", 2);
} else {
box.fadeOut().data("fade", 1);
}
}, 800)


activemq and the java.io.EOFException: Chunk stream does not exist at page on broker start

Deleting the activemq-data directory seems to be the only recovery solution (which is not an option in production)

rm -rf /opt/local/share/java/activemq/data


Changing the computer name on terminal


sudo scutil --set HostName [NewMacComputerName]

Next flush the DNS cache on the Mac by entering the command:

dscacheutil -flushcache


Removing 1Password agent on Snow Leopard

months after i uninstalled 1Password something was still spamming my log files.

It seemed like launchd has some problems, so i headed over one of the following folders

/System/Library/LaunchDaemons
/System/Library/LaunchAgents
/Library/LaunchDaemons
/Library/LaunchAgents
~/Library/LaunchAgents

I found this entry

ws.agile.1PasswordAgent.plist

let's delete it and refresh launchd

launchctl remove ws.agile.1PasswordAgent

and the problem has gone.

iTunes search with command f

To use Command-F to get to the search field in iTunes you have to execute the following in Terminal.

defaults write com.apple.iTunes NSUserKeyEquivalents -dict-add "Target Search Field" "@F"

You have to restart iTunes to make it work.

gitignore and already tracked files

If a file is already being tracked by Git, adding the file to .gitignore won’t stop Git from tracking it.

You’ll need to do

git rm --cached <file>

to keep the file in your tree and then ignore it.

SVG JavaScript Libraries

SVG has been in development since 1999 and in 2010 we can say that all major modern web browsers except Microsoft Internet Explorer, support and render SVG markup directly. The Internet Explorer 9 beta supports SVG.

But dealing with SVG documents remained painful, don't despair there are JavaScript libraries to help.


Protovis
Great for data visualization.
It only works in browsers that have native SVG support.


Raphael
It's a good starter library, easy to do a LOT of things with SVG quickly. Well written and documented. Lots of examples and Demos. Very extensible architecture. Great with animation.

But note that there are ways of expressing things in SVG that are not possible in Raphael. There are no "groups". This implies that you can't implement layers of Coordinate Transfomations. Instead there is only one coordinate transform available.
If your design depends on nested coordinate transforms, Raphael is not for you.

It supports Firefox 3.0+, Safari 3.0+, Chrome 5.0+, Opera 9.5+ and Internet Explorer 6.0+.


jQuery SVG
Well written and documented. Lots of examples and demos. Supports most SVG elements, allows native access to elements easily.

It only works in browsers that have native SVG support.


SVG Web
It uses flash to render in non-SVG compliant browsers.

List open ports on Mac OS X

sudo lsof -i -P | grep -i "listen"

FDTG Principles for Developers

this is a small reminder for me no to forget these principles.

Flexibility
Be ready to change your plans when they’re not working the way you expected; don’t count on things remaining stable.

Decentralization
Centralized systems look strong, but when they fail, they fail catastrophically.

Transparency
Don’t hide your systems; transparency makes it easier to figure out where a problem may lie. Share your plans and preparations, and listen when people point out flaws.

Graceful Failure
Failure happens, so make sure that a failure state won’t make things worse than they are already.

HTML Comments as strings

Reading HTML comments from a DOM tree with JavaScript is easy.

var nodes = document.body.childNodes,
    comments = [];

for (var i = 0; i < nodes.length; i++) {

    // if the nodeType is 8
    if (nodes[i].nodeType == 8) {
          
          // this is a comment
          comments.push(nodes[i]);
    }
}

Let's say that the comments array is not empty, so the type of comments[0] is a Comment. That's great but how can you read its content?

This is what the W3C DOM-Level-1 spec says. It implements the Comment interface.

interface Comment : CharacterData {
};

This does not really help us but don't despair. It also implements the Node interface.

  ...
  readonly attribute  DOMString            nodeName;
           attribute  DOMString            nodeValue;
  ...

Yes, it's read-only but you can get the comment as a string by using its nodeValue.

comments[0].nodeValue

W3C DOM-Level 3 specifies something called textContent. This can also do the job, you can read about them here, textContent vs. innerText

Blame does not fix bugs

It's always seems hard to deal with people. Here are few thoughts that I'd like to share.

Blame does not fix bugs, the outcome is important, not the credit, the blame, or the ongoing intellectual superiority contest.
You want to focus on fixing the problem, instead of blaming others.

Criticize ideas, not people, keep it professional, not personal.
Ask questions that allows someone to figure out the problem for themselves.

and there is one more thing, negativity kills innovation so be constructive.

simpleHTTPServer, a simple web server

If you wish to have a simple web server that shares files inside your local network there is a good news. You can use simpleHTTPServer which is a Python (2.5+) builtin HTTP Server.

Go to the folder that you want to share and start the server.

python -m SimpleHTTPServer 8000

It will start the server at 8000 and it will serve the files that the current folder contains.

Finding in files

Finding a string in files is useful and not that straightforward. But don't despair, we have bash.

find . -exec grep -q 'foobar' '{}' \; -print

This searches in the current directory including its subdirectories and finds files that contain foobar.

Smart Keyword Search with Omnibox in Chrome

Omnibox is a prominent feature of Google’s Chrome Browser. Omnibox is Chrome’s combined search and address bar feature, which allows you to use the address bar as a search bar.
Type a search term in the address box (Omnibox) and press enter to see results from your default search engine.

But it happens that you want to use Youtube search or Wikipedia search instead.

Open Chrome/Preferences/Basic
Click to the Manage button next to the Default Search label.



Here you can see what search engines Google Chrome can use (you can remove from the list or you can add a new item to the list)



Double click on an item shows up a dialog where you can set up the keyword.



If you type this keyword and your search query to the Omnibox, Chrome will search with that engine.

I changed youtube's keyword to yt so if I type yt Humcrush to the Omnibox it'll show me Humcruch youtube videos.

That's smart.

Here is my keywords.

What changed in your home folder today.

You can perform a simple Spotlight search from the shell

mdfind [query]

for instance

mdfind Ruby
mdfind -name stdlib.h

and then you pipe the result as you want

mdfind Ruby | grep 'pdf'

You can redefine your search by specifying any of the metadata attribute keys.
Here is a helpful list of metadata attributes that you can use.

You'd like to see what changed in your home directory today, easy.

mdfind -onlyin ~/ '(kMDItemFSContentChangeDate >= $time.today)'


Video for IPAD

Problem: You have a DIVX/DVD video on your computer and you want to watch it on your iPad.

Solution: There are a few options how to convert your video:

• If the original file is supported by QuickTime it would be your first and simplest choice. Just load your file into QuickTime X and choose Save As from the File menu. Then choose HD 480p as a desired format and wait till the conversion is finished (this may take a while)



• If you have some esoteric data format or you have to get more control over the encoding process or simply you want to get better quality than HD 480p, then you have to use something else.
  Basically there are plenty of choice (I assume you are coming from Unix world):

  
  
  
  
  
  1. Mencoder (part of Mplayer)
  
  
  2. ffmpeg
  
  
  3. x264 - this is free implementation of h.264 codec. The first two use it as a depndecny anyway
  
  
  
  
  

  I had a luck with tool number 2. Here is an example how I converted some video files to iPad format:

  
  
  

  for i in *.avi; do ffmpeg -i "$i" -acodec libfaac \
  -ab 96k -vcodec libx264 -s 1024x576 -vpre lossless_ultrafast \
  -crf 20 -threads 2 "$i".mp4; done
  

  
  
  Probably you'd like to set higher audio bitrate etc. But the command above will get you started. If you need explanations what these params do, please refer to the links above.

The only thing left now is to add the newly converted file to your iTunes library and sync with your iPad.

Enjoy!

Monitoring file system activity on OS X

My hard disk is quite slow, so It seemed a good idea to see what's going on under the hood, why the filesystem is so busy.
I came across fslogger, which is a great user-space program that subscribes to the kernel's file system change notification service.

Sounds great and actually monitoring file system activity with fslogger is quite easy. It must be run as root but that's it.

The output is little bit verbose but you can cut it with awk.

# changed files
sudo fslogger | awk '/FSE_ARG_STRING/ { print $5 }'

# file change type
sudo fslogger | awk '/type.*=/ { print $3 }'

# process that caused the change
sudo fslogger | awk '/pid.*=.*\(.*\)/ { print $4$5 }'



Helpful.

Unix sort by column

Sort is a very useful unix utility. Simplest usage is here::

ls | sort

but that's not very interesting. Let's say we want to sort all the files by size:

ls -l | sort -k5 -n

What this command does exactly sort by 5th field using numeric order. Another useful option you may use is -t which allows you to select the field separator.

&& and || bash operators

I guess you already know about && operator, which is very common. But || is not so common.

To better illustrate how it works here is an example:

>(exit 1) && echo '0' || echo '1'
1


>(exit 0) && echo '0' || echo '1'
0

also you can do tricks like that

>t=`( echo 'some output'; exit 0) && echo $t || echo 'Error!!!'
some output

>t=`( echo 'some output'; exit 1) && echo $t || echo 'Error!!!'
Error!!!

Very tasty usefulness! Isn't it ?

Ruby memory leaks

So we had this problem that some batch processes in our production environment were leaking memory... and the funny part is that it was working fine on our developer machines.

In order to find out what's actually happening to it I've used the tool called

bleak_house, which is very good to examine how your object space is evolving.

Istallation is fairly simple:

gem install bleak_house

And then just run you application using the ruby-bleak-house instead of a regular ruby.

Basically what bleak_house does is it dumps an object space picture to a disk when you press Ctrl-C, or send a USR2 signal to the process.

So I used the second method from crontab to get an image of the process every minute.

Then you can run bleak command to analyze the dump and see what has changed.

In our case it happened that logs were not flushed (sic!), but since we are currently running our application with debugging logs enabled - it led to very fast process growth.