Trim Seconds of a Video with FFMPEG

Google failed me. How do you trim x amount of time off a video when you don’t know its length (imagine a script scenario). All this whilst preserving its quality, i.e. copy only, no transcoding.

First we need to find its length. We do this via ffprobe:

ffprobe -v quiet -show_entries format=duration movie.mp4

-v quiet shuts off ffprobe’s info vomit and just gives us the line we want. Let’s get rid of the lines we don’t care about:

ffprobe -v quiet -show_entries format=duration movie.mp4 | grep "duration"

Now let’s use cut to get the figure:

ffprobe -v quiet -show_entries format=duration movie.mp4 | grep "duration" | cut -d "=" -f 2

We have finally got a number in seconds for the video length!

Trim the length of time you desire via bc:

echo $(ffprobe -v quiet -show_entries format=duration movie.mp4 | grep "duration" | cut -d "=" -f 2) - 30.5 | bc

Now we can take this and plug it into our ffmpeg command:

ffmpeg -i movie.mp4 -t $(echo $(ffprobe -v quiet -show_entries format=duration movie.mp4 | grep "duration" | cut -d "=" -f 2) - 30.5 | bc) -c:v copy -c:a copy movie_out.mp4

Done!

Automatically Transfer Torrents from Deluge to rTorrent

I’ve been working on a script (due to personal interest and requests from various private tracker threads) that auto-moves torrents from Deluge (better at downloading & uploading simultaneously) to rTorrent (better for long-term seeding). Now that it’s completely “out of beta” so-to-speak, I’m going to share it! I hope someone finds this useful.

Use your favourite text editor to save the script below somewhere. Make sure you chmod +x delugeExport.sh (or whatever you end up calling it).

The script has several dependencies. You need to install/configure them.

Download this other script and put it somewhere. I put it in the same directory as delugeExport.sh. In order for this Perl script to function, it requires the Bencode module: sudo perl -MCPAN -e 'install Convert::Bencode'. Finally, don’t forget to chmod +x rtorrent_fast_resume.pl.

Configure the configurable things in delugeExport.sh. They’re all important so don’t miss one.

Ensure rTorrent is setup as you would like. You basically just need the watch folder setup (schedule = watch_directory,5,5,load_start=/your/directory/*.torrent). You also need to set rTorrent’s download directory to the directory where Deluge sends completed torrents. Any other stuff like ruTorrent can wait if you want.

Now you need to run the script from the command line. Every single *.torrent file that you have loaded in Deluge will be processed by rtorrent_fast_resume.pl and will have fastresume data added to it. This will only happen for 100% complete downloads. Partial, or still-downloading torrents will not be moved across. All your Deluge torrents are now in rTorrent’s watch folder and have been added if rTorrent was running!

Go to Deluge and remove (without data) all your completed torrents.

Go to Preferences –> Plugins. Turn on Execute.

Turning on Execute in Deluge Preferences

• Restart deluged (and deluged-web).

• Go to Preferences –> Execute. Do your version of this (i.e. enter the path to the script!):

Configuring Execute plugin

Click Apply/Okay/etc.

You may optionally configure a cronjob to clear out the *.torrent files out of rTorrent’s watch directory every 24 hours. Simply execute crontab -e and enter the following line: 0 0 * * * rm $HOME/path/to/rtorrent/watch_folder/*.torrent > /dev/null 2>&1

Unfortunately, due to a bug in the Execute plugin (version 1.3.5 and earlier only) concerning Unicode handling, before you start testing out your new configuration you must follow this post of mine here. If not, you’ll find that random torrents stay resting in Deluge, even after they’re completed.

Finally, you’re done! Add a torrent to Deluge and test it!

Also, you can further minimise your time dealing with Deluge by using this UserScript (use it with NinjaKit for Safari or Chrome).

#!/bin/bash
####################################################################
###             Please fill in the variables below!              ###
####################################################################

# Default is "deluge", but you should have changed it...
password=deluge

# Your domain/IP here. For example, google.com or localhost or 127.0.0.1
domain=localhost

# The port you use to access the WebUI (default 8112)
port=8112

# Do you use http or https to access the WebUI (default http)?
scheme=http

# Which folder are you keeping rtorrent_fast_resume.pl in?
# Please include trailing slash.
dir=$HOME/torrents/

# Where do your torrents eventually download to? If they auto-move
# please specify that directory. Please include trailing slash.
download=$HOME/torrents/complete/

# rTorrent's watch folder. Please include trailing slash.
watch=$HOME/torrents/watchr/

# Do you want to backup whatever's in Deluge's state directory before
# you rm it all? (Maybe useful the first time for the mass-transfer
# just to be sure but probably not afterward).
backup=no

# Do not change this unless you have knowingly changed Deluge's
# state folder.
files=$HOME/.config/deluge/state/

###							         ###
# This script is designed to be used alongside the Execute plugin! #
###								 ###

####################################################################
###   Please remember to create the watch folder for rTorrent!   ###
###   Please remember to have rTorrent fully setup and config'd! ###
###   Remember to chmod  x this script and the perl script!	 ###
####################################################################

# Backup if you told it to.
date=`date  "%m-%d-%Y-%H.%M.%S"`
if [ "$backup" = "yes" ]; then tar -cf $HOME/deluge_state.$date.tar $files; fi

# Every torrent, "f", will have fastresume data added and then moved
# to rTorrent's watch directory, $watch, for addition.
# $1,$2 received from Execute plugin. Need specifics when not en masse.
# If block ensures these lines don't run except via Execute plugin.
# Authenticating to deluge-web.
# Remove torrent using $1 var passed from Execute.
if [ -n "$1" -a -n "$2" -a -n "$3" ]; then
	torrentid=$1
        torrentname=$2
        torrentpath=$3
	perl $dir/rtorrent_fast_resume.pl "$torrentpath" < "$files/$torrentid.torrent" > "$watch/$torrentname.$RANDOM.torrent"
	cookie=`curl -vskm 1 -H "Content-Type: application/json" -X POST -d "{\"id\": 1\,\"method\": \"auth.login\",\"params\": [\"$password\"]}" $scheme://$domain:$port/json 2>&1 | grep -i "Set-Cookie:" | cut -d '=' -f2 | cut -d ';' -f1`
	curl -vskm 1 -b _session_id=$cookie -H "Content-Type: application/json" -X POST -d "{\"method\":\"core.remove_torrent\",\"params\":[\"$1\",false],\"id\":2}" $scheme://$domain:$port/json
else
	for f in $files/*.torrent
        do
          name=`basename $f`
          perl $dir/rtorrent_fast_resume.pl $download < $f > $watch/$name
        done
fi

Please leave a comment if you find a bug!

And, yes, if you so desire, you can use this script as a simple way to move from Deluge to rTorrent completely. Simply execute the script once from the command line and everything will be moved over, imported into rTorrent via the watch directory.

Making Deluge’s Execute Plugin Support Unicode

The Execute plugin that comes bundled with Deluge has a problem with Unicode characters such as à, é, ü, î, etc. So you’ll find that if you download a torrent that has those sorts of characters in its main name, or even file names, the Execute plugin handler will fail and your script will not run. The torrents will simply rest, completed, in Deluge. This is irritating.

Luckily, I have found a solution! Here’s what to do.

Edit this file: /usr/share/pyshared/deluge/plugins/Execute-1.2.egg/execute/core.py

Go to line 122 and modify the following (add-in the lines marked by the “+”):

# Go through and execute all the commands
        for command in self.config["commands"]:
            if command[EXECUTE_EVENT] == event:
                command = os.path.expandvars(command[EXECUTE_COMMAND])
                command = os.path.expanduser(command)
                log.debug("[execute] running %s", command)
+               if isinstance(torrent_name, unicode):
+                   torrent_name = torrent_name.encode('utf-8')
                d = getProcessOutputAndValue(command, (torrent_id, torrent_name, save_path), env=os.environ)
                d.addCallback(log_error, command)

Restart Deluge and it a try! Magic. Thankfully this fix will be found in version 1.3.6 so we won’t need to re-do it each time Deluge gets updated.

How to Hide the TypeKit Logo

A quick mini-post on how to hide the TypeKit logo that appears on your page when you’re using the Trial plan. Here’s how to do it with WordPress.

Download the TypeKit Fonts for WordPress plugin. Now, in the Custom CSS Rules box, simply enter .typekit-badge {display:none !important;} and hit Save Settings. Voilà!

If this isn’t working for you, make sure you’ve cleared your X/Page/W3-Cache caches, and your browser’s cache. These caused me to think it wasn’t working for a while.

As you may imagine, this is against TypeKit’s T&C’s, but pfff.

Create an OpenELEC USB on Mac OS X

If you’re seeking to create an OpenELEC install USB on OS X, you’ll be disappointed to find that there is no easy “create_installstick” script for Mac as there is for Windows/Linux.

Luckily this has already been worked out in the OpenELEC forums. You can download a convenient little image here!

After you’ve downloaded the OpenELEC .tar.bz2 file, unarchive it, and then simply download the above file and put it in the newly created OpenELEC folder.

Now, cd to the OpenELEC folder and chmod +x create_installstick_osx.sh to make it executable. Before you run the script make sure you have your USB disk plugged in so the script sees it. Beware, all data on it will be deleted during the script’s execution!

Now, sudo su and then ./create_installstick_osx!

You will be asked to choose the right disk (don’t get this wrong!) and to confirm your understanding that its current contents will be lost and overwritten. The script will then unmount the disk, format it, download some small tools, and then copy the OpenELEC install image over. At the end of it, the USB can be safely removed as it is already unmounted. Here’s a little shot of what’ll happen:

OpenELEC Install Script

Happy XBMC’ing!

Creating a Bootable USB on a Mac for Flashing The BIOS on an HP N40L Microserver

I thought I would write a few words on this to hopefully save some other Mac users some time. I had to try so many times to get this to work.

I recently bought an HP N40L Microserver and I wanted to flash the BIOS to an unofficial version which enables more features (like AHCI full-speed SATA on all available ports – configure this as shown here). There are a variety of ways to do this, but seeing as the model I was working with didn’t have a CD drive, I wanted to boot FreeDOS off of a USB.

After various problems with Unetbootin, I discovered that it doesn’t actually create the boot image properly on a Mac. Bizarre.

What we’ll do is simply download a FreeDOS image prepared by someone else and then dd it to the device.

The image I used is FreeDOS 1.1 and I found it here. Download this and unarchive it. Whilst it’s downloading, take your USB device and format it to FAT32 in Disk Utility. Now go to Terminal and type diskutil list. You should see output similar to the following:

Terminal Screenshot

Here you can see that that I have already formatted the disk and titled it FREEDOS. My disk identifier is disk2s1. Yours will be something similar, you must work out what it is.

Unmount (read: not eject) the disk you want to use. For example, I used diskutil unmount /dev/disk2s1 – perhaps you would use diskutil unmount /dev/disk3s1. Word of caution: Don’t try and use a drive with multiple partitions. They will get destroyed.

Now that the drive is available for our exclusive use, fire up dd to copy across FreeDOS image that you downloaded onto the USB stick using this command: dd if=/path/to/image/FreeDOS-1.1-USB-Boot.img of=/dev/diskX where “X” = the disk identifier (2 and 3 respectively in the above example. After just less than a minute your new bootable USB should be complete (and automatically re-mounted)!

Simply download the unofficial BIOS from here (I simply removed two unneccessary files from the original, it still contains the latest modded HP BIOS), unzip it, and copy it on to the root of the USB drive.

Now stick it in the machine, fire it up, and simply type flash.bat at the DOS command prompt. Voilà.

Ext3 Recovery on Mac OS X Advice

Don’t try and do it on a VM. It’s never worked for me. And now it’s so easy to do it on a Mac with built-in read support from 10.6 onward. The command to check your poor ext3 drive is this: fsck -fy /dev/diskXsX

Where X is the replaced by the number given to you by: df -h

Post a Week

It’s time to breathe some life back into this old blog! I’ve been in Paris for 12 months now, and my head is filled with advice, stories and useful tidbits. And not just of Paris, coding too. Lots of Parisian and coding things all bouncing around my head. But as ever, my procrastination has stopped me doing anything useful with them – like writing them down and sharing them with people (as I know you’re all desperate to hear about them). But no more!

Yes, it is time for me to commit to something. Something like posting at least once a week. Even if it’s just 100 words or so. And what better week to start with than this one right here? Expect a new post (this one not included, hah) before the clock strikes midnight on Sunday.

Multiple Deluge users

It was recently just me using Deluge on my server, but now my friend (and his friend) would like it too. I thought there might be a way to use one web server and three different daemons (along with three init.d scripts), but I tried this and it’s not as secure and robust as I’d like. So I basically create three entire deluge daemon/webserver combos running alongside each other. It’s horribly kludgy, but it works. As soon as Deluge develop some sort of conf-based multiuser setup or something, I’m there.

Assuming you’ve got one Deluge instance set up properly (for details on that, see here) you now just need to create a couple more by duplicating all the init.d scripts, etc. and affixing them to the correct existing users (or you can create a new user if you want, it doesn’t really matter). For reference, both my friends were using rtorrent, so I’m transitioning them from that to Deluge.

This wiki page is very important. It contains the init.d scripts and log details that we’re going to use to setup Deluge for another user. I’m assuming you already have a user setup with the default “deluge” name. Let’s say the user we want to set Deluge up for is called “bob”.

Init Scripts

Thus for DELUGED-USER you’ll want to put “bob”. Also, you’ll need to change the file’s name as we already have the /etc/default/deluge-daemon file. I like to name them in the follwing style: /etc/default/deluge-bobd.

Save the init.d script as /etc/init.d/deluge-bobd. Make sure to do sudo chmod 755 /etc/init.d/deluge-bobd.

You should change the following lines:

# Provides:          deluge-daemon
#                    /etc/default/deluge-daemon.
NAME1="deluged"
NAME2="deluge"
DAEMON1_ARGS="-d"    # Consult `man deluged` for more options
DAEMON2_ARGS=""    # Consult `man deluge-web` for more options
PKGNAME=deluge-daemon

To something like this:

# Provides:          deluge-bobd
#                    /etc/default/deluge-bobd.
NAME1="deluged_bob"
NAME2="deluge_bob"
DAEMON1_ARGS="-d -p 58847 -L info -l /var/log/deluge_bob/daemon/warning.log"    # Consult `man deluged` for more options
DAEMON2_ARGS="-p 8113 -L info -l /var/log/deluge_bob/web/warning.log"    # Consult `man deluge-web` for more options
PKGNAME=deluge-bobd

We have specified a particular listening port for the daemon and for the webserver. These ports are simply the default ports + 1 – it means they won’t conflict (you’ll need to check these ports fit your own configuration). Next, we can create those log paths.

Logs

Simply adjust the paths it has in the wiki to the something like the following:

sudo mkdir -p /var/log/deluge_bob/daemon
sudo mkdir /var/log/deluge_bob/web
sudo chmod -R 755 /var/log/deluge_bob
sudo chown -R bob:bob /var/log/deluge_bob

Then let logrotate.d know how you want the logs rotated, fire up nano /etc/logrotate.d/deluge_bob and enter:

/var/log/deluge_bob/*/*.log {
        weekly
        missingok
        rotate 7
        compress
        notifempty
        copytruncate
        create 600
}

Save it and you’re done, logging-wise.

Starting Up

Now start the daemon and enter the default pass of “deluge”, and change whatever preferences you want to (more information here). The Connection Manager will pop-up and display the default Deluge daemon connection on port 58846. This is not your daemon as we have it running on 58847. In order to add our daemon to the Connection Manager, select the default entry and click Remove. Now click Add. The host is “localhost” and the port is 58847 in this case. The username and password are contained within Deluge’s auth file. This is kept in the Deluge user’s home directory at this path: ~/.config/deluge/auth. It contains three strings separated by colons. The first part is the username, the second is the password, and the third is the permissions level (normally 10 which means admin). Enter the username and password into Connection Manager! Now click “Add” and you’ll see it added. Now you can click “Connect”. You’re ready to add torrents!

Free SSL Certificate with StartSSL and Mac OS X

Start SSL by Lewis Miller

Up until now I have been using a self-signed certificate for securing and encrypting areas of my website that I wish to keep away from prying eyes. However, I found that I wasn’t able to stream media that I have contained within these HTTPS’d directories to VLC. I think VLC refuses to connect to sites with a self-signed certificate, but I am not 100% sure on that – it may be the case that I didn’t realise I had to place some certificates somewhere. In any case, this lack of connectivity, and thus streaming, was my main motivation in procuring a real, signed SSL certificate. But of course, I couldn’t deal with paying for it!

I found a free SSL CA in the form of StartSSL (another one I found is CAcert, but I didn’t end up using them). Unfortunately their FAQ and installation steps aren’t that great, which is why I am going to try and break it down nice and easy!

To start using StartSSL is relatively simple. Simply click Sign Up and enter the few details that the site requests. Next, you’ll be brought to a page that requests a verification code that has been sent to the email that you signed up with. After entering it you’re normally told that you have to wait 6 hours or less for your account to be verified. I found it took much less time than that. After your account is verified, and you enter the code, your browser (in my case, Safari) will download a .crt file. This certificate is very important! It is your login to the StartSSL control panel. As soon as it’s downloaded, double-click it. Keychain Access will now prompt you to install the certificate – do it!

Safari 5.1 (or the site) has a bug whereupon the browser will crash when you try to access the control panel even though you have the correct authentication details. In order to overcome this, you must use a different browser. The simplest one to use is Google Chrome as it pulls the certificate data directly from Keychain, but in an email exchange, I was informed by StartSSL that Chrome “has other issues”. The only browser that seems to work perfectly is Firefox. However, the certificate and private key must be imported into Firefox in the form of a .p12 file.

In order to get this .p12 file, go into Keychain Access, select the login keychain, and the All Items category. Now select the certificate you downloaded from the site and installed (its name will be the email address you used) and your private key (probably called “key from www.startssl.com”. With only these two items selected, right-click on one of them and select “Export 2 items…”. Make sure the File Format is Personal Information Exchange (.p12). Put in whatever password you like for exportation and then enter your admin password. Now in Firefox go to Preferences –> Advanced –> Encryption –> View Certificates. Make sure you’re on the “Your Certificates” tab. Now click “Import…”. Select the .p12 file you created and enter the password you chose when you exported it. It’s now imported and you can now use Firefox to access the StartSSL control panel!

So, back at the StartSSL control panel, we’re going to create a certificate to add in to something like Lighttpd or Apache. First we have to validate a domain. Click the Validations Wizard tab. Select Domain Name Validation. Now enter your domain name and its TLD. The vast majority of TLDs are selectable, but unfortunately .co.cc (a free domain name) isn’t (I recreated my domain under another free domain name, .tk, which is selectable). After you click Continue you’ll be asked to select an email address (from a finite list) that you wish a verification code to be sent to. If you don’t have one of these email addresses that they ask for set up, you’ll have to set it up (I use Google Apps which makes it incredibly easy to add extra email address aliases, and domain aliases too).

Now we have a domain ready to go! What we need to do now is prepare a Certificate Signing Request (CSR). I did this with Lighttpd in mind, but this stage should work for Apache too. Drop into your server’s CLI, cd to wherever you want to keep your certs (like /etc/lighttpd/certs) and do sudo openssl req -new -newkey rsa:2048 -nodes -keyout server.key -out server.csr and answer the questions – make sure at the Common Name or Domain Name step you type the domain name that you want to secure, like “theendoftheuniver.se”. cat the server.csr file and copy exactly what’s inside it.

Back in the StartSSL control panel, go to the Certificates Wizard tab. Select Web Server SSL/TLS Certificate. As we’ve created a CSR, we can click Skip. Now paste in the CSR that you copied. Unless you already did this at the domain name validation step, you’ll have to choose a subdomain in addition to your standard domain name. I recommend www, but you can choose any. For free validations, you can only have one subdomain. Now you should finally have your SSL cert! Woohoo! Copy the SSL cert and paste it into a file on your server. Save As the root and intermediate CA certificates (I wget’d them straight to the server). Now create your final .pem by combining your server’s private key with the SSL certificate you just copy/pasted: sudo cat server.key ssl.cert > domainname.tld.pem

Make sure you do a sudo chmod 0400 on the certs so no-one but root and the webserver can read them. If it doesn’t work when you restart the webserver you probably need to sudo chown webserver-user:webserver-user on the certs too.

Lastly, you need to add the certificates to your webserver’s configuration. For Lighttpd, you’ll want a block similar to this in your .conf:

$SERVER["socket"] == ":443" {
        ssl.engine = "enable"
        ssl.pemfile = "/etc/lighttpd/certs/domainname.tld.pem"
        ssl.ca-file = "/etc/lighttpd/certs/sub.class1.server.ca_int.pem"
}

It’s better to use the intermediate certificate as it adds another level of security.

Now simply restart Lighttpd and enjoy your signed HTTPS! Leave a comment if you encounter any problems.

And relating to this post, you can use your signed SSL certificate with Deluge by simply cat‘ing the domainname.tld.pem file and saving the RSA Private Key section to daemon.key and the Certificate section to daemon.cert. Then put these in the relevant folder (something like ~/.config/deluge/ssl/), chmod them to 0400 again, chown them to Deluge’s user, and then restart the Deluge daemon.