Get list of URLs that Google Search has indexed

Here is a quick and dirty way to get the list of google results for a domain.

1. Do a google search for   site:example.com
2. Open your browser's console
3. copy paste:

var elems = document.getElementsByTagName('cite')
for(var i=0; i<elems.length; i++) {
console.log(elems[i].innerText)
}

Mounting S3 on an Ubuntu EC2 intance

Compile S3FS

sudo apt-get install build-essential libfuse-dev fuse-utils libcurl4-openssl-dev libxml2-dev mime-support

curl -O http://s3fs.googlecode.com/files/s3fs-1.61.tar.gz

tar -xzvf s3fs-1.61.tar.gz

cd s3fs-1.61/

configure

make

sudo make install

setup security

vim /etc/passwd-s3fs

accessKeyId:secretAccessKey

sudo chmod 640 /etc/passwd-s3fs

mount it

Everything a Web Developer needs for OSX

Setting Up New Computer.

Within the last month received a new MacBook Pro, and had to re-install another. So I decided to list the software I installed.

• Browsers
• Chrome www.google.com/chrome/
• Firefox www.mozilla.org/firefox
• FireBug
• LastPass lastpass.com/
• Command line Tools
• Git git-scm.com/
• php 5.4 php-osx.liip.ch/
• nmap nmap.org/
• homebrew mxcl.github.com/homebrew/
• Xcode (3.22 for SnowLeopard | 4.4 for Lion)
• IDEs
• NetBeans netbeans.org/
• OSX Java
• PhpStorm www.jetbrains.com/phpstorm/
• Sublime Text www.sublimetext.com/
• GitX(L) gitx.laullon.com/
• SourceTree www.sourcetreeapp.com/
• DataBase
• Mysql www.mysql.com/
• Sequel Pro www.sequelpro.com/
• JumpCut jumpcut.sourceforge.net/
• Other Apps
• Minuteur www.phg-home.com/index_mac.html
• ScreenFlow www.telestream.net/screen-flow/ 
• Tweaks
• Terminal
• Change theme to home brew
• Add command key
• When the shell exists: Close if the shell exited cleanly
• Keyboard ShortCuts Prefs
• All controls for tab key
• Shell Tweaks
• edit ~/.profile
• function parse_git_branch_and_add_brackets {

git branch --no-color 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/\ \[\1\]/'

}

PATH="/usr/local/php5/bin:/usr/local/mysql/bin:$PATH"

PS1='\[\e[01;32m\]\u@\h:\[\e[01;34m\]\w\[\e[0;31m\]$(parse_git_branch_and_add_brackets)\[\e[0m\]$ '

alias ls='ls -G'

OpenSSL - Create your own Certificate of Authority and sign your SSL Certs

Creating your own CA for Signing Dev Certs

Here are some handy commands to create a CA and sign your own cert for development.

The commands are fully scriptable: all key passwords have been bypassed with the -nodes flag, and the interactive openssl prompts not necessary with in the the -subj flag.

tldr; Short version for the Impatient:

# Create Certificate of Authority
SUBJ="/C=US/ST=CA/L=San Diego/O=Lance Rushing/OU=Development/CN=Dev Certificate of Authority"
openssl req -subj "$SUBJ" -nodes -new -x509 -extensions v3_ca -keyout devCA.key -out devCA.crt -days 365

# Create Key and CSR
HOST="appname.local"
SUBJ="/C=US/ST=CA/L=San Diego/O=Lance Rushing/OU=Dev Web Sites/CN=$HOST"
openssl req -subj "$SUBJ" -newkey rsa:2048 -nodes -keyout $HOST.key -out $HOST.csr -days 365

# Create the Cert by signing the CSR with our CA
openssl x509 -req -CA devCA.crt -CAkey devCA.key -CAcreateserial -in $HOST.csr -out $HOST.crt

Add the appname.key and appname.crt to your apache/nginx/IIS config, and add the devCA.crt to your browser.

Long Version.

When developing webservices that will be using SSL in production, I like to use SSL in development.

The old way

The traditional way to do this is create a self-signed cert. Where you sign your Certificate Signing Request (CSR) with the same key that created it. (Thus "self-signed".)

HOST="appname.local"
SUBJ="/C=US/ST=CA/L=San Diego/O=Lance Rushing/OU=Dev/CN=$HOST"
openssl req -subj "$SUBJ" -x509 -newkey rsa:2048 -nodes -keyout $FILENAME.key -out $FILENAME.crt -days 3650  ## look mom! One line.

Then add it to the apache.conf

<VirtualHost 172.16.1.1:443>
   ServerName appname.local:443
   DocumentRoot "/Users/lance/Sites/AppName/src/webroot"

   SSLEngine on
   SSLCertificateFile "/Users/lance/Sites/AppName/certs/appname.local.crt"
   SSLCertificateKeyFile "/Users/lance/Sites/AppName/certs/appname.local.key"
</VirtualHost>

This works great, but now with every new ssl cert I generate I have to add a trust exception in my browser.

A better way

To avoid multiple trust exceptions of you self-signed certs, is to first generate a "Certificate of Authority" (CA), add that CA to your browser, and then sign all of your certs with that CA.

# Create CA
SUBJ="/C=US/ST=CA/L=San Diego/O=Lance Rushing/OU=Development/CN=Dev Certificate of Authority"
openssl req -subj "$SUBJ" -new -x509 -nodes -extensions v3_ca -keyout devCA.key -out devCA.crt -days 3650

# Create Key and CSR (same as above)
HOST="appname.local"
SUBJ="/C=US/ST=CA/L=San Diego/O=Lance Rushing/OU=Dev Web Sites/CN=$HOST"
openssl req -subj "$SUBJ" -newkey rsa:2048 -nodes -keyout $HOST.key -out $HOST.csr -days 365

# Create the Cert by signing the CSR with our CA
openssl x509 -req -CA devCA.crt -CAkey devCA.key -CAcreateserial -in $HOST.csr -out $HOST.crt

# cleanup CSR
rm $HOST.csr

Thoughts

These commands are the shortest way I've found to create keys, CSRs, and signing. Other guides often use three steps (1 genrsa, 2 key export w/o pass, 3 req ) to generate the CSR, whereas it is possible to do it with 1 step.

Hint: By using the -subj flag we can bypass openssl's interactive prompts

Hint: I use Virtual IPs for each https web service I need:
OsX: $ ifconfig lo0 alias 172.16.1.1
Linux: $ ifconfig lo:0 172.16.1.1
windows: http://support.microsoft.com/kb/236869

http://gagravarr.org/writing/openssl-certs/ca.shtml
http://www.g-loaded.eu/2005/11/10/be-your-own-ca/

Fixing my Bash Prompt on OSX

I have been having a problem with the bash prompt. When I use the up and down arrows to scroll throw the history buffer, the output of the previous commands would get garbled and not line up. After messing around for 20 minutes I found a solution.

My old prompt:

PS1='\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[0;31m\]$(parse_git_branch_and_add_brackets)\[\033[00m\]\$ '

My new prompt:

PS1='\[\e[01;32m\]\u@\h:\[\e[01;34m\]\w\[\e[0;31m\]$(parse_git_branch_and_add_brackets)\[\e[0m\]$ '

Something about changing the \033 to \e works. Why? I don't know, if you do please comment.

BASH Prompt in OSX with git branch

I found others doing the same.

Here is the content of my ~/.profile.

function parse_git_branch_and_add_brackets {
git branch --no-color 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/\ \[\1\]/'
}

PS1="\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[0;31m\]\$(parse_git_branch_and_add_brackets)\[\033[00m\]\\$ "

Quickly rename files to lower case

Here is a simple bash oneliner to rename all folders/files in a directory to lower case

for old in * ; do new=`echo $old | tr "[[:upper:]]" "[[:lower:]"`; mv $old $new;  done;