The Ubuntu Incident

Problem

I use Dropbox a lot; it’s very useful to have a large folder that is automatically synchronized and made available on all my machines, be it either a Linux or a Windows box.

However, I have problems with file permissions. Unfortunately it’s not treated by Dropbox in a platform-independent way. What I mean is the following: when I log in to Windows, Dropbox “nicely” flattens all my files, i.e. it removes the executable flag from my scripts… When I log back to Linux, all my scripts (*.sh, *.py, etc.) are non-executable. Great! :(

Solution #1 (naive approach)

First I wrote a simple bash script to correct the directory and file permissions:

#!/bin/bash

HERE=$HOME/Dropbox
cd $HERE

find . -type d -print0 | xargs -0 chmod 700
find . -type f -print0 | xargs -0 chmod 600
find . | grep ".py$" | xargs chmod u+x
find . | grep ".sh$" | xargs chmod u+x
chmod u+x $HERE/xmind-portable/XMind_Linux/xmind
chmod u+x $HERE/xmind-portable/XMind_Linux/xmind-bin
chmod 755 $HOME/Dropbox
chmod u+x git.projects/others/upskirt/upskirt

I put it in crontab and called the script once in an hour.

The script gets the job done but it has a great price. In my Dropbox folder I have more than 30,000 files. This script updates all of them with “chmod”. Even if there is no change (for ex. a file had 0600 permissions and we set 0600 again), Dropbox notices the update (!) and synchronizes all the files with its server! Thus, even if the script modified the permissions of few files, all the files will be synchronized. Result: high CPU and RAM usage for several minutes when the script is launched…

Solution #2 (sophisticated approach)

To overcome the previous problem, we need to do the following: look at the permissions of a directory/file and IF its permissions are not good THEN modify the permissions. That is, if permissions are OK, we do not touch the file with “chmod” and thus Dropbox will not take any actions. Dropbox will synchronize only those files whose permissions really changed.

So here is my script, written in Python. Just put it in your Dropbox folder and launch it. By default it’s in “dry” mode, i.e. it only prints the changes without applying them. If the result is OK, set DRY to False. The script also verifies if it’s in the Dropbox folder. Since it changes permissions recursively, you don’t want to run it in your HOME folder :)

#!/usr/bin/env python

# dropbox_permissions.py
# https://ubuntuincident.wordpress.com/2011/05/08/setting-file-permissions-in-your-dropbox-folder-recursively/

import os
import sys
import stat

# dry run, make no changes just show them
DRY = True
#DRY = False

# verify if we are in the Dropbox folder
VERIFY_DROPBOX = True
#VERIFY_DROPBOX = False

ignore_dirs = ('.git', '.svn', '.eric4project', '.ropeproject')
executable_file_extensions = ('.py', '.sh', '.pl')
executable_files_with_relative_path = (
    './xmind-portable/XMind_Linux/xmind',
    './xmind-portable/XMind_Linux/xmind-bin'
)

changes = 0


def chmod_ux(file):
    set_mode_to(file, 0700)


def set_mode_to(file, permissions):
    global changes
    f = file
    mode = get_oct_mode(f)
    if mode != oct(permissions):
        try:
            if DRY:
                print "# chmod {0} {1}".format(oct(permissions), f)
            else:
                os.chmod(f, permissions)
            changes += 1
        except OSError:
            print >>sys.stderr, "# cannot chmod the file {0}".format(f)


def get_oct_mode(entry):
    entry_stat = os.stat(entry)
    mode = oct(entry_stat[stat.ST_MODE] & 0777)
    return mode


def process_dir(directory):
    set_mode_to(directory, 0700)


def process_file(file):
    f = file
    file_name = os.path.split(f)[1]
    file_ext = os.path.splitext(file_name)[1]

    if (file_ext in executable_file_extensions) or (f in executable_files_with_relative_path):
        process_exe_file(f)
    else:
        process_other_file(f)


def process_exe_file(file):
    chmod_ux(file)


def process_other_file(file):
    set_mode_to(file, 0600)


def traverse(directory):
    """Traverse directory recursively. Symlinks are skipped."""
    #content = [os.path.abspath(os.path.join(directory, x)) for x in os.listdir(directory)]
    content = [os.path.join(directory, x) for x in os.listdir(directory)]
    dirs = sorted([x for x in content if os.path.isdir(x)])
    files = sorted([x for x in content if os.path.isfile(x)])

    for d in dirs:
        if os.path.islink(d):
            continue
        dir_name = os.path.split(d)[1]
        if dir_name in ignore_dirs:
            continue
        # else
        process_dir(d)
        traverse(d)
    
    for f in files:
        if os.path.islink(f):
            continue
        # else
        process_file(f)


def verify_dir(directory):
    d = os.path.abspath(directory)
    if 'dropbox' not in d.lower():
        print >>sys.stderr, """
It seems that you are not in the Dropbox folder. If you launch this
script in a wrong folder, it may do more harm than good since it
changes file permissions recursively.
If this is a false alarm and you really want to execute the script
here, disable this verification by setting the variable VERIFY_DROPBOX
to False.
"""
        sys.exit(1)


def main():
    start_dir = "."
    if VERIFY_DROPBOX:
        verify_dir(start_dir)
    traverse(start_dir)
    #chmod_ux(sys.argv[0])
    print "# changes: {0}".format(changes)
    if DRY:
        print "# >>> it was a dry run, no changes were made <<<"

####################

if __name__ == "__main__":
    main()

The up-to-date version of the script is available here, in the dropbox/ folder.

You can safely call it from crontab at every hour, it will not eat up your system resources.