Pandemonium Illusion

Entries from May 2008

Bulk Add/Replace from Command Line

May 31, 2008 · Leave a Comment

I had to do this like 5 times today, so I thought I’d put it out there (so I can reference it).

Note: you might want to backup your stuff, just in case you screw up your regular expression.

Basically, I have a bunch of files that need to have changes, such as email address, or include paths.

So, for example, in one I had to change a bunch of email addresses so I used `sed`. To do this on one file you would simply run the sed command directly:
sed -e s/info@olddomain.com/info@newdomain.com/g -i file.html
-e tags the regular expression you’re using
-i means we’re editing the files inline (not outputting the results to standard out)
Don’t forget g, because this will ensure that the expression is greedy and will find all the instances of this expression.

Of course, you probably want to run this on a bunch of files, so you would use `find`, because it’s awesome:
find -name "*.html" -exec sed -e s/info@olddomain.com/info@newdomain.com/g -i {} \;

Categories: Uncategorized
Tagged:

Starting a new Django Project with Versioning

May 12, 2008 · Leave a Comment

Having an efficient working environment can be essential to the success of any project and can save hours, if not days of extra work in the long run. So, I’ve put together a little “getting started guide” to putting the full package in place with Django, SVN, and Trac.

Create your django project anywhere

~/django/projects$ ../src/trunk/django/bin/django-admin.py startproject myProject

Set up your SVN
This can be done either on your local machine or on another server. I like /var/svn/repos/myProject/.

$ svnadmin create /var/svn/repos/myProject

I use svnserve to serve my svn so I update the permissions in /var/svn/repos/myProject/conf/svnserve.conf:

[general]
anon-access = none
auth-access = write
password-db = passwd

/var/svn/repos/myProject/conf/passwd

[users]
user = passwd

(re)Start the svnserve daemon

svnserve -d -r /var/svn/repos

Finally, we’re ready to check in the project tree

$ svn import myProject file:///var/svn/repos/myProject/trunk/myProject -m "Initial import"

Now just import it into Eclipse and your good to go! ;)

Oh, and you might want to read about excluding .pyc files in svn.

Categories: django · webdev
Tagged: , ,

Compress a PDF with pdftk

May 7, 2008 · 5 Comments

Ok, so I don’t know why this works, but it does and I haven’t seen any decrease in quality. I suspect converting from PDF to PS and back just optimizes the PDF.
Requirements

The Process

  1. Convert your PDF to PS (this creates a large file
  2. Convert the new PS back to a PDF


pdf2ps large.pdf very_large.ps
ps2pdf very_large.ps small.pdf

Results
large.pdf : 6.3MB
very_large.ps : 53.4MB
small.pdf : 2.4MB
Looks like pretty good compression to me.

If anyone knows why this works so well, please let me know.

Categories: Uncategorized

Ignore .pyc files in Subversion

May 7, 2008 · 3 Comments

Subversion allows you to exclude certain files using regular expressions. This needs to be applied to each directory individually. So, if you add a new directory I think you will need to run the command again. To apply the svn:ignore property to your project you just need to do two things:

Create a file (.svnignore) with the regular expressions

*.pyc
*~

Run svn propset on the project

svn -R propset svn:ignore -F .svnignore .

-R indicates that you are doing this recursively so all the directories ignore these.
-F indicates the file we just created with the regular expressions.

EDIT 5/27/08:
If you’ve already got yourself into the mess of versioning your compiled files, then you might want to do a few things.
1) Revert all the .pyc files (you can always recompile them). You don’t want there to be any local changes when you try to delete them:
find -name "*.pyc" -exec svn revert {} \;
2) Delete all the .pyc files using `svn delete`:
find -name "*.pyc" -exec svn delete {} \;
3) Finally recompile and re-run the svn ignore code above

Categories: django · webdev
Tagged: ,

Backing up with BackupPC

May 5, 2008 · Leave a Comment

BackupPC is an awesome tool for backing up a number of different machines. BackupPC has a web application that manages your backups and a separate backup application that runs periodically based on the settings you have in the web admin.

The design is ingenious because it can back up a variety of machines with unique settings. For windows machines I use a protected share that can only be accessed by a user called ‘backup’. The backup application then connects periodically to the share and runs incremental backups. For the linux machines I run an rsync demon on the pc. This is one of the few cases where it’s as easy on windows as linux. (In case there’s any doubt… everything is easier on linux 99% of the time.)

Given that I use linux exclusively, here’s the documentation on setting up an rsyncd connection with Backuppc.

1) Install rsyncd: $ sudo apt-get install rsync (ubuntu)
2) Configure rsyncd with a backup option (/etc/rsyncd.conf)

[backup]
path = /home/jamstooks
exclude = "/home/jamstooks/.mozilla/firefox/***"
comment = Backup
uid = jamstooks
gid = jamstooks
read only = true
auth users = backup
secrets file = /etc/rsyncd.secrets

I excluded the firefox directory, because I was getting issues with the cache being accessed during backup. It would probably be best to include the bookmarks though.
/etc/rsyncd.secrets stores the passwords in plain-text, so make sure you adjust your permissions appropriately:

backup:passwd

3) Ubuntu sets up an init.d setup file, but you want to make sure the daemon is running…
4) Finally, you are ready to set up BackupPC. In the BackupPC admin, simply select rsync for this machine and fill in the password you selected.

More details can be found in the official BackupPC documentation.

Categories: Uncategorized
Tagged: , ,

Quick image resizing with python

May 4, 2008 · 2 Comments

I typically build automatic resizing into any django models that use ImageField by overriding the save function like so:

def save(self):
super(GalleryImage, self).save() # Call the "real" save() method
if self.image:
# now change the size of the image
path = settings.MEDIA_ROOT + self.image
img1 = PIL.open(path)
img2 = PIL.open(path)
img2.save(self.get_thumbnail_path())

size = 600,600
img1.thumbnail(size, PIL.ANTIALIAS)
img1.save(path)

# create the thumbnail
size = 150, 150
img2.thumbnail(size, PIL.ANTIALIAS)
img2.save(self.get_thumbnail_path())

def get_thumbnail_path(self):
path = settings.MEDIA_ROOT + self.image
return self.convert_path_to_thumbnail(path)

def get_thumbnail_url(self):
path = self.get_image_url()
return self.convert_path_to_thumbnail(path)

def convert_path_to_thumbnail(self, path):

basedir = os.path.dirname(path) + '/'
base, ext = os.path.splitext(os.path.basename(path))

# make thumbnail filename
th_name = base + "_tn"
th_name += ext
return urlparse.urljoin(basedir, th_name)

This ensures that all my images conform to the sizes I need. However, even if the server is automatically resizing the images for me, I don’t want to spend hours uploading huge images only to have them resized. It’s just a waste of bandwidth and time. So, I’ve written a quick python script that resizes all the images in a given directory and places the copies in a new directory called resized:


#! /usr/bin/env python

"""
What:
Resize all the jpg images in a directory
All images will be placed inside a directory called "resized"
This directory must exist
Usage: ./resize.py
"""

import PIL.Image as PIL
import re, os, sys, urlparse

SIZE = 600,600
JPG = re.compile(".*\.(jpg|jpeg)", re.IGNORECASE)

def get_new_path(path):
basedir = os.path.dirname(path) + '/resized/'
base, ext = os.path.splitext(os.path.basename(path))
file = base + ext

return urlparse.urljoin(basedir, file)

try:
image_dir = sys.argv[1]
except:
print "You must specify a directory"
exit(0)

files = os.listdir(image_dir)
for file in files:
if JPG.match(file):
f = image_dir.rstrip("/") + "/" + file
print f
img = PIL.open(f)
img.thumbnail(SIZE, PIL.ANTIALIAS)
img.save(get_new_path(f))

Categories: django · webdev
Tagged: , ,