File Transfer and Management (FTM)
The File Transfer and Management (FTM) System offers simple, uniform, and secure access to the storage resources of the HLRS over local or remote network connections. The functionality offered is analogous to that of the Unix-based systems of the HLRS machines, extended somewhat to enhance its use in a distributed environment. Clients of FTM can be either Unix or Windows based systems. Both command-line (CLI) and Graphical (NIFTI) interfaces are offered.
The topology of the FTM system is depicted in Figure 1. The specification of a file or directory in this network topology requires the specification of:
- a StorageServer: a machine which stores resources (files and directories) which may be accessed by the user,
- a Subspace : the directory within a StorageServer's storage, in which relative paths reside (analogous to the Unix Current Working Directory (CWD)),
- the path of the resource, either relative to the subspace or absolute.
To illustrate the principles and terminology the topology shown in Figure 1 includes four StorageServers:
- the storage of the client itself,
- the storage of a "neighbor" platform (such as other Workstations within the user's local environment), accessible from the client without use of SSL or the Safe server,
- StorageServers within the HLRS domain (a Virtual Organisation or Vorg), accessible using the Safe server as an intermediary via SSL (two are shown in the Figure, but any number can be supported).
Any FTM command acts on one or more resources on a StorageServer specified by the PartnerStorageServerName, in a subspace specified by the PartnerSubspaceName. FTM supports the commands on resources on any of the StorageServers in the topology, including those within HLRS, on neighbor platforms, and on the client itself.
For transfer operations, the PartnerSubspaceName refers to the source(s) of Get requests or the target of Put requests, whereas the OwnSubspaceName specifies the target of Get requests or the source of Put requests. Both the PartnerSubspaceName and the OwnSubspaceName may be taken from the defaults from the cli.props file (typically HOME), or may be overridden on the command line.
FTM supports actions on resources in any of these domains, and transfers between any two domains.
FTM can also support multiple Virtual Organisations (Vorgs); however, for this documentation, we treat only the HLRS as the (single) Vorg.
Access Control and Security
FTM handles user authentication and security based on a Public Key Infrastructure (PKI), in which the user maintains a secret "private key" (usually on the local workstation), and asserts a personal identity in the form of a matching "public key" within a "certificate". To be valid, the certificate must be signed by the HLRS Certificate Authority (CA) to verify its authenticity (i.e. the user's identity). This user certificate, serves to identify the user on all machines and environments supporting FTM, and is also used to construct the Secure Socket Layer (SSL) connection to the server for secure access control and transmission of data.
The use of the PKI mechanism removes the need for filtering individual IP addresses, so that supported resources can be accessed from any workstation at any location by any user posessing the proper certification.
The FTM system supports essentially all Unix functions which can be carried out by a command line function without interactive feedback from the user during the command. For example, functions for ls, cat, rm etc. are supported, while vi, man and more are not. Other than these interactive functions, FTM supports the combined functionalities of both ssh and scp, plus enhancements, thus offering the essence of a uniform and secure network operating system.
The following table summarizes the functionality of FTM and compares it with that of ssh and scp.
Functional Comparison of FTM with SSH and SCP
|single firewall access||independent of client IP address||YES||NO|
|user authentication||global certificate||password, or individual certs installed on each StorageServer|
|file transfer to/from client or Neighbor||YES||NO||YES|
|single sign-on for a series of commands||YES||YES||NO|
|support setting of StorageServer for subsequent commands||YES||NO||NO|
|support setting of remote Subspace for subsequent commands||YES||NO||NO|
|transfer of directories||YES||N/A||YES|
|support for wildcards||YES||YES||YES|
|transfer of list of files||YES||N/A||NO|
|transfer between StorageServers||YES||NO||YES|
|support for scripts, using return codes, etc.||YES||YES||YES|
|control overwrite of existing files||YES||NO||NO|
|optional automatic backup of overwritten files||YES||NO||NO|
|Progress monitor for transfers||YES||N/A||YES|
|support of compression for transfers||YES||N/A||YES|
|encryption is optional||YES||N/A||NO|
|optional limits for file transfers||YES||N/A||NO|
- Peggy Lindner, Jim Almond
An Integrated Global Service for File Transfer and Management in a Network (FTM)
Published at PDPTA'04 - The 2004 International Conference on Parallel and Distributed Processing Techniques and Applications, Las Vegas, 2004.
- Jim Almond, Peggy Lindner
FTM Global File Transfer and Management in a Network
Presented at PDPTA'04 - The 2004 International Conference on Parallel and Distributed Processing Techniques and Applications, Las Vegas, 22.6.2004.