Skip to content

Making duplicity and ncftpput play nice with vsftpd FTP daemon

I’ve been setting up yet another remote backup lately (see associated problem here). For this purpose (on Unix), the duplicity solution looks ideal. However, I’ve tried it on a couple of lightweight FTP servers (a TL-WDR3600 and a Raspberry Pi) and neither of them work. I keep getting a “Permission Denied” message.

I did quite a bit of investigation, and found that it’s related to the way that duplicity’s FTP backend, ncftp/ncftpput sends files. Or really, a quirk of vsftpd, where it won’t let you put a file with an absolute path (and more specifically a file with a directory name). I’ve seen other reports of this on the Internet here and here.

Here’s a sample ncftpput debug log:

2014-06-25 00:56:34  331: Please specify the password.
2014-06-25 00:56:34  Cmd: PASS xxxxxxxx
2014-06-25 00:56:34  230: Login successful.
2014-06-25 00:56:34  Cmd: PWD
2014-06-25 00:56:34  257: "/"
2014-06-25 00:56:34  Logged in to as Poojan.
2014-06-25 00:56:34  Cmd: FEAT
2014-06-25 00:56:34  211: Features:
2014-06-25 00:56:34        EPRT
2014-06-25 00:56:34        EPSV
2014-06-25 00:56:34        MDTM
2014-06-25 00:56:34        PASV
2014-06-25 00:56:34        REST STREAM
2014-06-25 00:56:34        SIZE
2014-06-25 00:56:34        TVFS
2014-06-25 00:56:34        UTF8
2014-06-25 00:56:34       End
2014-06-25 00:56:34  Cmd: PWD
2014-06-25 00:56:34  257: "/"
2014-06-25 00:56:34  Cmd: CWD folder1/duplicity/Public
2014-06-25 00:56:34  250: Directory successfully changed.
2014-06-25 00:56:34  Cmd: CWD /
2014-06-25 00:56:34  250: Directory successfully changed.
2014-06-25 00:56:34  Cmd: TYPE I
2014-06-25 00:56:34  200: Switching to Binary mode.
2014-06-25 00:56:34  Cmd: PASV
2014-06-25 00:56:34  227: Entering Passive Mode (192,168,1,2,225,89).
2014-06-25 00:56:34  Cmd: STOR folder1/duplicity/Public/duplicity-full.20140625T055614Z.vol1.difftar.gpg
2014-06-25 00:56:34  550: Permission denied.
2014-06-25 00:56:34  ncftpput folder1/duplicity/Public/duplicity-full.20140625T055614Z.vol1.difftar.gpg: server said: Permission denied.
2014-06-25 00:56:34  Cmd: QUIT
2014-06-25 00:56:34  221: Goodbye.

However, I ran a few tests, and I found that if one cd‘es to the directory and then puts (STOR‘es) the file, everything works. My guess this is a chroot-style feature. So, I created a small Python script that will parse the arguments that duplicity sends, and inserts cd (to the target directory) command before uploading the file. This works well. Here’s the script:

import argparse
import subprocess
import sys
import os

# ncftpput looks something like this:
# 'ncftpput -f /tmp/duplicity-yRftm3-tempdir/mkstemp-PiYFTL-1 -F -t 30 -o useCLNT=0,useHELP_SITE=0  -m -V -C '/tmp/duplicity-yRftm3-tempdir/mktemp-sWc1AA-3' 'folder1/duplicity/Public/duplicity-full.20140625T041147Z.vol1.difftar.gz
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='wrapper for ncftpput command, to allow absolute target paths on vsftpd')
parser.add_argument('-f', dest='credentials_file')
parser.add_argument('-F', action='store_true')
parser.add_argument('-t', dest='timeout')
parser.add_argument('-o', dest='ftp_options')
parser.add_argument('-m', action='store_true')
parser.add_argument('-C', action='store_true')
parser.add_argument('-V', action='store_true')
args = parser.parse_args()

# print(args)

dest_dir = os.path.dirname(args.dest)
dest_file = os.path.basename(args.dest)

#cmd_args = ['/usr/local/bin/ncftpput', '-d', 'ftp-debug.log', '-W', 'cd {0}'.format(dest_dir)] +  sys.argv[1:-1] + [dest_file]
cmd_args = ['/usr/local/bin/ncftpput', '-W', 'cd {0}'.format(dest_dir)] +  sys.argv[1:-1] + [dest_file]

The script is called ncftpput, and is placed in a directory called ~/duplicity_bin. I then add this directory to my path before running duplicity. Here’s a shell script that does that:

export PATH="/home/Poojan/duplicity_bin:$PATH"
duplicity -v 9 --encrypt-key=DEADBEEF full /tank/Users/Public ftp://Poojan@

Be the first to like.

Post a Comment

Your email is never published nor shared. Required fields are marked *