How to connect to an FTP server using Python

FTP (File Transfer Protocol) needs no presentations: it is among the most used file transfer methods between one or more clients and a server. By design it supports both anonymous access and authentication, but in its most basic form it doesn’t provide data encryption, that’s why it is often secured via TLS.

A lot of FTP client applications are available on Linux, as for example Filezilla (graphical) or lftp (command line). Sometimes, however, we may want to access an FTP server programmatically, perhaps to schedule file transfers. One easy way to do this is by using a programming language like Python. In this tutorial we will learn how to use the ftplib library to interact with an FTP server.

In this tutorial you will learn:

  • How to create an instance of the ftplib.FTP class
  • How to list files on a remote FTP server
  • How to upload files in binary and “lines” mode
  • How to download files in binary and “lines” mode
  • How to create,delete and rename directories and files
  • How to change working directory

How to connect to an FTP server using Python

How to connect to an FTP server using Python

Software requirements and conventions used

Software Requirements and Linux Command Line Conventions
Category Requirements, Conventions or Software Version Used
System Distribution independent
Software Python
Other No other permissions required
Conventions # – requires given linux-commands to be executed with root privileges either directly as a root user or by use of sudo command
$ – requires given linux-commands to be executed as a regular non-privileged user

The ftplib library

The ftplib module is part of the Python standard library, and provides two main classes to abstract working with an FTP connection: ftblib.FTP and ftplib.FTP_TLS. The latter is a subclass of the former and adds support for TLS. Let’s see some of the most common use cases of the library.

Connecting to an FTP server

To connect to an FTP server, the first thing we have to do is to create an instance of the FTP class. The class supports the with statement so it can be used with a context manager: this way, the connection will be automatically closed when we finish working or an error occurs. Here is an usage example:

with ftplib.FTP('ftp.somehost.com') as ftp:
    # codehere


All the parameters of the FTP class constructor are optional, however here we provided the first argument accepted by it, which is the host we want to connect to. If the argument is provided, the connect method, used to establish a connection with the server, is implicitly called with the specified host passed as argument, otherwise it should be called explicitly:

with ftplib.FTP() as ftp:
    ftp.connect('ftp.somehost.com')

The second argument accepted by the FTP class constructor is the user we want to login as into the ftp server. Providing this argument will cause the login method to be called implicitly with the user, the password and acct values passed as arguments (they are the third and fourth parameters of the class constructor, and default to an empty string as value):

with ftplib.FTP('ftp.somehost.it', 'testuser', 'testpassword') as ftp:
    # codehere

If the argument is not provided, the login method must be called explicitly:

with ftplib.FTP('ftp.somehost.it') as ftp:
    ftp.login('testuser', 'password')

Getting a list of files on the server

Once an FTP object is created, we basically have three ways of obtaining a list of the files stored on the FTP server we are connected to. First of all we can use the dir method, which produces a directory listing as returned by the LIST command:

>>> with ftplib.FTP('ftp.somehost.it', 'user', 'password') as ftp:
...     ftp.dir()

The dir method accepts an optional argument, which is the directory to list (the default is the current working directory one, so in this case the FTP root). The above code produces an output similar to the following:

drwxr-xr-x    2 ftp   ftp         4096 Oct 13 14:37 .
drwxr-xr-x    2 ftp   ftp         4096 Oct 13 14:37 ..
-rw-------    1 ftp   ftp           10 Sep 10 06:04 .ftpquota
-rw-r--r--    1 ftp   ftp      5306756 Oct 18 01:32 file.csv

The second method we can use to obtain a list of files, is nlst. As its name suggests, this method, under the hood, sends a NLST command to the server; it returns a Python list containing the name of the files as members:

>>> with ftplib.FTP('ftp.somehost.it', 'user', 'password') as ftp:
...     ftp.nlst()
...
['.', '..', '.ftpquota', 'file.csv']

The third method we can use to obtain to list the content of a directory is mlsd. This method uses the MLSD command (so for it to work, the server must support it), and accepts two optional arguments:

  • The path of the directory that should be listed
  • A list of the information we want to be included in the result

The method returns a generator which yields a two-elements tuple for each file: the first element of each tuple is the file name; the second a dictionary containing the requested information and their values. Let’s see an example:

>>> with ftplib.FTP('ftp.somehost.it', 'user', 'password') as ftp:
...     for filename, information in ftp.mlsd():
...             print(filename, information)


The output of the code above is the following:

. {'type': 'cdir', 'sizd': '4096', 'modify': '20201013123732', 'unix.mode': '0755', 'unix.uid': '1809', 'unix.gid': '1811', 'unique': 'fd04g58e0a67'}
.. {'type': 'pdir', 'sizd': '4096', 'modify': '20201013123732', 'unix.mode': '0755', 'unix.uid': '1809', 'unix.gid': '1811', 'unique': 'fd04g58e0a67'}
.ftpquota {'type': 'file', 'size': '10', 'modify': '20200910040430', 'unix.mode': '0600', 'unix.uid': '1809', 'unix.gid': '1811', 'unique': 'fd04g58e0a9d'}
file.csv {'type': 'file', 'size': '5306756', 'modify': '20201017233245', 'unix.mode': '0644', 'unix.uid': '1809', 'unix.gid': '1811', 'unique': 'fd04g58e020a'}

Notice that the server is not guaranteed to respect the list of information we request.

Retrieving files from the server

To retrieve files from the server, we can use the retrbinary or retlines methods.  Let’s see how they work.

The retrbinary method retrieves a files in binary transfer mode: this is what you want to use to simply download a file from the server to your local machine and don’t need to interact with its content. Let’s see an example of its usage. Say we want to download the file.csv from the server; we would simply write:

>>> with ftplib.FTP('ftp.somehost.it', 'user', 'password') as ftp:
...     with open('file.csv', 'wb') as downloaded_file:
...         ftp.retrbinary('RETR file.csv', downloaded_file.write)
...
'226-File successfully transferred\n226 0.823 seconds (measured here), 6.15 Mbytes per second'

In the example above we opened a local file for writing in binary mode
(file.csv) using a context manager, then called the retrbinary method passing
an appropriate RETR command as first argument (RETR nameofthefile), and the
write method of the file object downloaded_file as the second argument, which
is a callback applied to each chunk of data received.

Speaking of data chunks, the maximum block size used for the transfer
of data, by default, is 8192 bytes. This, however, can be changed via the
optional third parameter of the retrbinary method.

The retrlines method works a little bit differently, since it retrieves files in “line” mode. The first argument of this method, can be a valid RETR command, just like the one we used in the previous example, but also a LIST (to retrieve a list of file names and information about them) or NLST (retrieve just filenames). The second argument of the method is optional and is a callback which is applied to each retrieved line (default behavior is to print lines to stdout). It is important to notice that each line is stripped of the end of line character, which on Linux is \n.

Let’s see an example. If we use the retlines method, we can retrieve the content of the file.csv file line by line:

>>> import os
>>> with ftplib.FTP('host', 'user', 'password') as ftp:
...     with open('file.csv', 'w') as csvfile:
...         ftp.retrlines('RETR file.csv', lambda x: csfile.write("".join([x,os.linesep])))
...

In the example above we imported the os module, then, just as before, we created a file locally, this time in textual mode. With the ftp.retrlines method we retrieved the file.csv remote file line by line. The callback we used as second argument of the retrlines is a lambda function which takes the line as argument and calls the write method of the csvfile object to write the line joined with the linesep character appropriate for the Os, which we accessed by os.linesep.

We can use the callback to also modify the content of the file on the fly. As a trivial example, imagine we want to uppercase each words contained in the remote file when we store it locally. We could write:

[...]
...   ftp.retrlines('RETR file.csv', lambda x: csfile.write("".join([x.upper(),os.linesep])))

This method, as we already mentioned, can be used to also work with the lines returned by the LIST or NLST commands. Suppose we want to save the result of listing a directory on the remote server into a local file:

>>> with ftplib.FTP('host', 'user', 'password') as ftp:
...     with open('list_result', 'w') as localfile:
...         ftp.retrlines('LIST', lambda x: localfile.write("".join([x, os.linesep])))

The local file list_result will be create (or truncated and overwritten if it already exists), and its content will be something similar to:

drwxr-xr-x    2 ftp   ftp         4096 Oct 13 14:37 .
drwxr-xr-x    2 ftp   ftp         4096 Oct 13 14:37 ..
-rw-------    1 ftp   ftp           10 Sep 10 06:04 .ftpquota
-rw-r--r--    1 ftp   ftp      5306756 Oct 18 01:32 file.csv

Uploading files to the server

When we need to upload a file to an FTP server, we can also choose to do it in binary or “lines” mode. The two methods we can use to accomplish the task, are respectively: storebinary and storelines.

The storebinary method of the FTP class takes two mandatory arguments which are a valid STOR command, and the file object created from a local file opened in binary mode. Suppose we want to upload a file; we would write:

>>> with ftplib.FTP('host', 'user', 'password') as ftp:
...     with open('linuxconfig.txt', 'rb') as file_object:
...         ftp.storbinary('STOR linuxconfig.txt', file_object)


Really simple! Of course, we can also store the file on the server with a different name. The file object passed as the second argument of the storbinary method is read until EOF. Just like in the case of the retrbinary method, its possible to change the data chunk size, with the optional third argument (the default, is, again 8192 bytes). The fourth argument accepted by the storbinary method, is an optional callback function which is applied to each chunk of data.

To upload a file line by line, we can use the storlines method instead. In this case the file we want to upload will be read line by line. The first two arguments are the same accepted by the storbinary method, while the third (and last) is a callback that is applied to each line.

Navigating, creating directories, deleting and renaming files

The FTP class (and the FTP_TLS class which extends it) provides also some very useful methods to perform some of the most common operations. For example, to create a directory on the remote FTP server, we can use the mkd method which takes the pathname of the directory to create as its sole argument:

>>> ftp.mkd('newdir')
'newdir'

To change the working directory we can use the cwd method, passing the name of the directory we want to move into as argument:

>>> ftp.cwd('newdir')
'250 OK. Current directory is /newdir'

To delete an existing directory, we can use the rmd method, passing the name of the directory to be removed:

>>> ftp.rmd('newdir')
'250 The directory was successfully removed'

To delete a regular file we can use the delete method instead, passing the name of the file to be deleted as argument:

>>> ftp.delete('file.csv')
'250 Deleted file.csv'

To rename files or directories, we can use the rename method. It accepts two arguments: the first is the current name of the file or directory, the second is the new one. To rename file.csv to file0.csv, for example, we would write:

>>> ftp.rename('file.csv', 'file0.csv')
'250 File successfully renamed or moved'

Closing a connection manually

As we already learned, the FTP class can be used with a context manager, so that the connection is automatically closed when the interpreter exits the with statement block. In cases where we have to close the connection manually, however, we must use the quit method: it calls the close method internally, and sends a QUIT command to the server to try to close the connection gracefully.

Conclusions

In this article we learned how to use the python ftplib module in order to connect to an FTP server and interact with it. We saw how to create an instance of the FTP class and what are the methods we can use to list the content of a remote directory and upload/download files. We also saw how to create, delete, rename and remove directories or files and how to change the working directory. In this tutorial we explored the most common use cases, for a complete feature list, please visit the official libftp page.



Comments and Discussions
Linux Forum