You are in the main area:Organization
Headerimage for: Command Line Interface (CLI)

Command Line Interface (CLI)

Overview

The File Transfer and Management (FTM) System offers simple, uniform, and secure access to remote storage resources over local or remote network connections. The Command Line Interface (CLI)  accesses FTM commands from a Unix terminal window or from shell scripts on the Unix client. FTM functionality is supported for both Unix and Windows clients using the Nifti GUI interface described elsewhere.
CLI supports interactive commands for carrying out a wide variety of functions on files and directories - much like Unix commands - but on distributed systems (both local and remote). For use in shell scripts, all functions uniformly return a code (with non-zero indicating an error condition) which can be tested by the script. For use from the command line, there is a "verbose" mode, which returns additional human-readable output summarizing the result of the request. Although the functionality of the CLI is based on that of Unix, command names and syntax are not identical, and functionality is in some cases a subset, in others an enhanced superset of the Unix functionality.

Default values for parameters not specified on the command line are kept in a cli.props file in a special "hidden" directory ".cli" in the user's home space on the client. The parameters within this file are explained in Appendix A.

FTM handles user authentication and security based on a Public Key Infrastructure (PKI), in which the user maintains a secret "private key" on the local workstation, and asserts a personal identity in the form of a matching "public key" within a "certificate" which is signed by the appropriate Certificate Authority (CA) to verify its authenticity (i.e. the user's identity). Both the private key and the user's signed certificate are stored in a keystore kept in the .cli directory. The name of the keystore file is specified in the cli.props file. When resources (files and directories) stored at a participating organization, or "Vorg" are to be accessed via FTM, requests are  accompanied by the user's signed certificate. This certificate, together with the user's private key, is used to construct a Secure Socket Layer (SSL) connection to the server (called SAFE) which handles all requests on behalf of the user. Since this mechanism offers state-of-the-art security, it may be used to access resouces within a security firewall.

In order to access the user's keystore, the FTM client must have access to the user's password(s) for the keystore and  the private-key within it (these passwords are often identical). The user may supply a password as an argument with the FTM command (in this case the current version assumes the keystore and  private key passwords are identical), or the passwords may be specified in the cli.props file, so the user doesn't have to enter them repeatedly in CLI commands. Since in this version, these passwords are stored in clear text (!) the user should not use  passwords which control access to any resources other than FTM CLI!  More secure mechanisms are being defined. In order to minimize the exposure of passwords kept in the cli.props file in this way, there are mechanisms for removing them when FTM CLI is not being used (Fexit), and for entering them again (Finit).
If there is no password in the cli.props file, and none is supplied on the command line, the user will be prompted to enter a keystore name and password via a dialog box. The specification of a password on the command line or in the dialog box does NOT store the password permanently in the cli.props file or elsewhere. To store the password as a "persistent" default in the cli.props file, the Finit command is used.

The FTM2 command line interface supports two modes of operation:

  1. "Single Command" mode, in which a connection to the Safe server is constructed for each CLI command, and then released when the command is finished. No preliminary  "sign-on" is required. Since each command is seen by the client shell environment:
    • CLI commands must have names different from local Unix commands; these always begin with "F" - e.g. Fls, Frm, Fget.
    • names and paths of remote file or directory resources must always be prefixed by a special character (the colon ":") to prevent possible wildcard expansion by the local shell. For example, to list all files and directories on the currently set StorageServer which begin with "foo", the FTM command would be Fls :foo* 
    Since this mode carries the overhead of establishing the connection for each command, it is appropriate when only one or few commands are to be issued, or for shell scripts. 
  2. "Session" mode (single sign-on), in which the user first establishes a persistent connection to a Safe Server at the Virtual Organisation (Vorg) via the Fconnect command, and can then issue a series of commands using the same connection. For example,
                    "Fconnect @HWW:hwwfs3.hww.de/HOME

    would create an SSL connection to Safe within the firewall of the HWW Virtual Organization (Vorg), with commands directed to the StorageServer  "hwwfs3.hww.de" and remote working directory set to the user's home space on the specified StorageServer. If CLI defaults in the cli.props file are set to HOME on hwwfs3.hww.de within HLRS, then the command would be simply Fconnect.
    Since these commands are not seen by the client shell environment:
    • Plain (non-interactive) Unix commands can be issued for the remote environment (as in an ssh session)
    • Certain FTM commands can also be issued within a session.  This allows the user to take advantage of additional functions such as get, put, and l   (not available in Unix or ssh) and of enhanced functionalities such as verbose mode, overwrite control, and the automatic backup of overwritten resources. 
    • Names of remote file or directory resources are NOT prefixed by the colon character in session mode. 
    The user may open multiple simultaneous connections, e.g. to different StorageServers. Since the keystore password is required only for the original connect, the user can avoid storing the password in the cli.props file, and enter it only when prompted by the Fconnect request.

Global Sepcification of Resources

The topology of the FTM system and its connections are depicted in Figure 1.

FTM Client Server Topology

In this network topology, the complete specification of a resource (a file or directory) requires four values:

  1. A Virtual Organisation (Vorg): a set of resources accessible by the user using a single loginID and a single security policy, accessed via an FTM server called SAFE (Secure Access File Executive) at a given port and IP address. Safe accepts  requests from the user's client, authenticates the requesting user using th user's public X509 certificate,  maps this certificate to the loginID by which the user is known, and carries out the request in the name of the user. In the context of this document, the HLRS is the only known remote Vorg.
  2. A StorageServer:  a machine within the Vorg which stores resources accessible by the user. The example topology shown in Figure 1  includes  four StorageServers;
    • The storage of the client itself (this may be a Unix or a Windows system),
    • The storage of a "neighbor"  platform (such as another Unix Workstation or storage system within the user's local environment), accessible from the client without use of SSL or the Safe server,
    • StorageServers (Unix) within the HLRS domain (a Virtual Organisation or Vorg), accessible using the Safe server as an intermediary via SSL (two are shown in the Figure, but any number can be supported).
  3.  A Subspace :  the directory within a StorageServer,  in which relative paths reside (analogous to the Unix Current Working Directory (CWD)). (Absolute paths are used as is, without reference to the Subspace value.) A subspace may be specified as:
    • The NAME of a subspace, which is mapped in a seamless manner by FTM to the actual path of the subspace on the StorageServer. Currently only "HOME" is implemented, signifying the user's homespace on the StorageServer.
    • The path of a directory relative to the current subspace on the StorageServer, e.g. if the current subspace is HOME, then myfiles/dir2 denotes /userhome/myfiles/dir2.
    • An absolute path on the StorageServer. (This is of course not "seamless", since it requires the user to know the precise absolute path.)
    • The path of a symbolic link to the actual subspace, e.g. if the link is defined in the user's homespace as
             MYDOCS -> journals/archives/msDocs 
      then specifying the subspace as "MYDOCS" serves as a shortcut to the actual path. See documentation on Flink and FlistDirLinks below.
    •  The name of a Workspace defined for the user on a StorageServer which supports Workspaces. See documentation on FmakeWS, FlistWS, and FdeleteWS below.
  4. <>The path to the resource, either relative to the subspace or absolute, as in Unix. Ftm supports both wildcard characters (*?[]) and special characters treated as "literals" in path names.  To protect possible wildcard characters from expansion by the client's shell, all path specifications must be preceded by a semicolon (e.g. :foo? refers to paths matching foo?on the remote StorageServer). To avoid confusion, the user is advised to avoid path names beginning with ":". In order to be treated as literals, characters from the list    @&()|><!\"'$;*?[]\   and the blank character (e.g. a file name foo@my space) must be escaped or be protected by enclosing the path in quotes, as in:
               :foo\&my\ space  or   ':foo&my space'
    The use of special characters is summarized in Table I in Appendix A. Since - as in Unix - the use of special characters for path names can be tricky, the user is advised to use them with care, and only when needed and understood.

FTM unary commands (e.g. Fls, Frm, Ftest ) operate on target resources at a (logically remote) partner, specified by a PartnerSpecification (Pspec), Vorg, StorageServer, and Subspace within which the resource path(s)  are located.
Get requests transfer (copy) files from the partner domain to the client domain; Put requests transfer (copy) files  from the client domain to the partner domain. Analogous to the partner domain, the client domain is specified by a OwnSpecification (Ospec). The Ospec typically specifies the client itself, and is only used when using "neighbor" storage as shown in Figure 1.


unary operation
Get from partner
Put to partner

target source
target
source target
PartnerStorageServer
x
x


x
PartnerSubspace
x
x


x
OwnStorageServer

x
x

OwnSubspace

x
x

While these variables may appear confusing at first, they are not normally seen by the user.  More typically:

  • The default for the PartnerStorageServer may be set in the cli.props file, or specified within the command. 
  • The PartnerSubspace is typically simply left as HOME, resolving to the homespace of the user on the PartnerStorageServer. 
  • PartnerStorageServer and PartnerSubspace are easily set using the :Fcd Pspec command presented below (a kind of global cd) or may be overridden on the command line. 
  • Unless specified otherwise, OwnStorageServer refers to the client itself. If specified as the name of a neighbor platform registered in a database within the client's .cli directory, and defined as StorageServerDBpath in the cli.props file, the OwnStorageServer becomes the neighbor platform. 
  • If left unspecified, OwnSubspace is always taken as the Current Working Directory (CWD) from which the FTM request is issued. 

For the specification of the partner on the command line, we define the Pspec parameter in the format:  
       @[PartnerVorgName]:[ PartnerStorageServerName]/[PartnerSubspaceName] :

   - where PartnerVorgName for the Stuttgart High Performance facility might be "HWW".

Some Introductory Examples:
Assume cli.props defines:
 - PartnerVorgName=HWW
 - PartnerStorageServerName=volvox.hww.de
 - PartnerSubspaceName=HOME   (i.e. the user's home space on volvox)
 - OwnStorageServerName and OwnSubspaceName are undefined, which resolves to the user's current working directory on the client machine:

    Fpwd                         displays current Pspec as     @HWW:volvox.hww.de/HOME
    Fls :foo?                    lists files matching  <userhome>/foo? on volvox.hww.de 
                                       Note the use of ":" to prevent the client shell from expanding the foo?.
    Fget :foo1  'local foo*'    copies  <userhome>/foo1  from volvox.hww.de to  local foo*  in the user's
                                       local CWD on the client. Note the use of blank and the literal "*" protected within
                                       single quotes as special  characters of the path.  While this demonstrates the use of
                                        special characters, the user is advised to be cautious in the use of such characters.
    Frm :foo1                    removes  <userhome>/foo1 on volvox.hww.de
    Fcd  :hwwfs1.hww.de/         PartnerStorageServer is now hwwfs1.hww.de; Subspace is still HOME
    Fcd  -                                    Pspec reverts to @HWW:volvox.hww.de/HOME
    Fcd someDir           changes PartnerSubspaceName to <userhome>/someDir ;
                                         (PartnerStorageServer remains volvox.hww.de)
    Fpwd                       displays the current Pspec as      @HWW:volvox.hww.de/someDir
    Fcd                          reverts to the default  @HWW:volvox.hww.de/HOME  as specified in cli.props
    Fls -v  @:hwwfs3.hww.de/mydir  :foo*  returns a verbose (i.e. long) list of all files in
                                                <userhome>/mydir on hwwfs3.hww.de which begin with "foo".
                                                  (The values of PartnerStorageServerName and PartnerSubspaceName
                                                     are NOT changed for subsequent commands.)

In the notation below:

  • :path is an absolute or relative specification of a remote path containing no wildcard characters 
    If path is relative (does not begin with a file separation character ("/" in Unix), it is made absolute on the remote platform by prepending the current SubspacePath. Otherwise (path is absolute) the SubspacePath is not used. Note that the ":" is NOT included in the directory name.
  • :path* is an absolute or relative specification of a remote path which may contain wildcard characters
  • . . .  indicates that multiple paths may be specified

API for the Command Line Interface; Single Command mode

Options are prefixed with "-", and may occur in any order. If not specified, a default value (possibly overidden in the  cli.props file) is used. Any options must be specified before the specification of a Pspec or path, as written in the syntax below.
General options: 

  • -v  is an option requesting a "verbose" result (for Fls, this produces a detailed list)
    -nv specifies a non-verbose result even if default specifies verbose
  • -pw  passwd specifies the password for the user's keystore containing an identifying Unicore certificate, and for the user's private key . The specification of a password on the command line is not necessary if the user has executed a Finit command (see below) for storing the password as a default. If no password is present, the user is prompted by a dialog box.
  • Pspec in the format above can be used to specify the partner storage space on the command line.

Options for data transfer operations:

  • -en specifies that ssl transfer streams are to be encrypted
    -ne specifies that ssl transfer streams are not to be encrypted
  • -c cl  specifies that transfer streams should be compressed at level cl (0-9) (default is that used by the Java jdk). 
  • -ow n specifies overwrite mode n for transfers and for renaming files. 
    • n = 0 (default): existing files are simply overwritten, as in Unix;
    • n = 2: existing files are NOT overwritten, but no error is reported; 
    • n = 3: an attempt to overwrite an existing file results in an error. 
  • -bu specifies that any overwritten file filename will first be backed up as filename~ 
    -nb specifies that overwritten files will not be backed up 
  • -pr specifies that the properties of the source are to be preserved onto the target.
    These properties are:
    • The last modification time of the files or directories being transferred
    • The Unix access mode for source files in a Unix system. For non-Unix clients,
      this property is ignored.

Functions of the FTM CLI Overview Overview

Unary operations operate on resources within a (Unix) StorageServer:

Fcd [Pspec]
Change the current Pspec. If a Pspec is specified, the new  values are written to a ".CuPspec" file in the .cli directory, and used for current partner values. If only Fcd is specified, any ".CuPspec" file is removed, so that the partner specification reverts to the default  values from the cli.props file.
Fcd - 
    returns the CuPspec to the previous one.
Fcd ..  (or Fcd /..) 
    change the Subspace to the parent of the current subspace if possible. (Cannot Fcd to the parent of HOME.)

The CLI currently has no analogous single command mode function for changing the "own" storage space.
(The current Unix working directory CWD is used unless overridden by

Fpwd
Query the current partner storage space given by default values from cli.props, replaced by any from a CuPspec file. Output is in the form of the Pspec given above.

Fls [-v
[op]] [-nv[op]] [-pw PW] [Pspec]  [:path*] . . .
 
Return a listing of file(s) or directory(s) specified by  path* . . .
 The -v option returns a detailed list (like Unix ls -l). -nv overrides any default for a detailed list, producing a simple  list like ls.  op can consist of a number of ls options as in Unix. For instance, Fls -va produces a "long" list including hidden files (those beginning with ".").   If no path is specified, the entire subspace is listed

Frm [-v] [-nv] [-pw PW] [Pspec]  :path* . . .
 Remove the file(s) specified by  path* . . . 
Frm cannot be used to remove directories (see FrecursRm). It CAN remove symbolic links.
The -v option returns a list of files actually removed.

Fmkdir  [-v] [-nv] [-pw PW] [Pspec]  :path . . .    
Make directory(s) specified by path . . .  (Note that the ":" is NOT included in the directory name.)
The -v option returns a list of directories actually created.

FrecursRm  [-v] [-nv] [-pw PW] [Pspec]  :path . . .    
Recursively remove the file(s) or directory(s) specified by  path . . .
If path specifies a directory, it is removed along with its entire contents.
The verbose option returns a list of file(s) or directory(s) actually removed.

Fchmod  [-v] [-nv] [-pw PW] [Pspec] [-R]  newMode  :path* . . .    
 Set the Unix access modes of file(s) or directory(s) specified by  path* . . . to the octal value newMode. 
The verbose option returns a list of file(s) or directory(s) whose mode was changed.

Fchgrp  [-v] [-nv] [-pw PW] [Pspec] [-R]  newGroup  :path* . . .    
Set the Unix group of files or directory(s) specified by  path* . . . to the value newGroup.
The verbose option returns a list of file(s) or directory(s) whose group was changed.

Fmv  [-v] [-nv] [-pw PW] [-ow n] [-bu]  [Pspec]  :path :newPath
 Move the path to newPath  (semantics are those of the Unix mv command).
 If newPath is a pre-existing file, its "overwrite" will be controlled by [-ow n];  a backup copy may be made as specified by [-bu] ;
  The verbose option returns a message verifying the rename and reporting any overwrite or backup action.

Ftest  [-v] [-nv] [-pw PW] [Pspec]   testSpec  :path
 Test the specified path (according to the Unix test command) according to the value testSpec.
  (e.g.  Gtest e foo  tests whether file foo exists) Note that there is no "-" or ":" preceding the tesSpec argument.  The verbose option returns a message indicating the result as true or false. Otherwise, the return code  of the   command can be used to evaluate the result (as with the Unix command).

Ffile  [-pw PW] [Pspec]  :path* . . .  
Return a description of the resource(s) denoted by   :path* . . .  

Fgrep [-pw PW] [Pspec]  pattern :path* . . .  
Search for the pattern in the files denoted by :path* . . .
Note that there is no ":" preceding the pattern argument.

Fcat [-pw PW] [Pspec] :path* . . .  
Return the contents of the file(s) denoted by :path* . . .

Ffind  [-v] [-nv] [-pw PW] [Pspec] :path  
Find and list paths corresponding to path, starting the search at the current subspace. This corresponds to the Unix        find startDir "searchPath" -print        command.
If verbose mode is specified, the returned message includes the "Permission denied" message for directories which do not permit the search.

Flink [-d dir] [-v] [-nv] [-pw PW] [Pspec] :path :linkName 
Create a symbolic link "linkName" within remote directory dir, as a symbolic link to the specified (relative or absolute) :path. If dir is not specified, or is specified as ".", the link is created within the current Subspace. If dir is specified as "HOME", then it is created in the user's home space. (This is most useful as a "shortcut" to commonly used directory paths.) NOTE: Flink will produce an error if a link or path of name linkName pre-exists in either the current Subspace or in the target dir.
Links can be deleted using the Frm function.

FlistDirLinks [-v] [-nv] [-pw PW] [Pspec] [:linkName*]
List all links within the current subspace, which correspond to linkName* and link to a directory.
If linkName is not specified, list all such links within the current subspace.

FallocateWS [-F filesystem] [-pw PW] [Pspec]  :WSName :d
Allocate a Workspace with name = WSName and duration = d in the specified <filesystem> (or in the
default on the current StorageServer) for the current user. The duration d in days must be specified.

FlistWS  [-F filesystem | -a | -l] [-pw PW] [Pspec]
If neither -F nor -a nor -l are specified, list all workspaces on the default filesystem.
If -F is specified, list all workspaces on the specified filesystem.
If -a is specified, list all workspaces on all filesystems.
If -l is specified, list all available workspace filesystems.

FreleaseWS [-pw PW] [Pspec] :WSName
Release the Workspace specified by WSName.

FregisterWS -F filesystem [-pw PW] [Pspec] :dir
Create symbolic links to all workspaces belonging to the user in the specified filesystem. If filesystem=ALL, then workspaces in all filesystems will be registered. The resulting links will be stored in the remote directory path specified by :dir.

In the following transfer requests, Fget,Fput, Fzipput, and Fxfer;

  • the [options] supported are
    [options] = [-v] [-c cl] [-e E] [-pw PW] [-ow n] [-bu]
      [-pr]  as presented above.
  • [Pspec] specifies the Vorg, StorageServer, and Subspace on the "Partner" to the transfer. For a Put (or a Zipput), this is for the target; for a Get it is for the sources; for a Fxfer transfer, it is for the sources (remember, the Fxfer request is being carried out by the server at the target of the transfer. 
  • [Ospec] specifies the Vorg, StorageServer, and Subspace on the "Own" system. For a Put (or a Zipput), this is for the sources; for a Get it is for the target; for a Fxfer transfer, it is for the target (the target is in the server's "Own" space).

Fget [options] [Pspec]  :path* . . .  [Ospec]  localTargetPath
 Copy the partner (e.g. remote)  file(s) or directories specified by path* . . . to the local path specified by localTargetPath within the current local subspace on the client. This is typically on the client itself, in the Unix client's Current Working Directory (CWD). This can be changed:

  • by altering the OwnStorageServerName to that of a StorageServer defined in the ss_db.xml database in the user's .cli directory, either in the cli.props file, or specifying an Ospec on the command line. Such "neighbor" clients (see Figure 1 above) must be accessible from the user's primary client via rsh or ssh, using the same login value as that of the user on the client itself. Normally the value of OwnStorageServerName is left undefined, so that the client StorageServer defaults to the user's local client itself.
  • by altering the OwnSubspaceName to that of a directory within the the CWD, either in the cli.props file, or specifying an Ospec on the command line. If the OwnSubspaceName is left undefined or blank, it defaults to the user's CWD on the client. The user might however wish to set OwnSubspaceName to "HOME" or "someSpecialDir"  so that files from a Get are always stored in the specified location, regardless of the client's CWD.

 The -v option for a Get returns a list of transferred files, reporting any overwrite or backup actions. 

Fput [options]  [Ospec] path* . . .  [Pspec]  :partnerTargetPath
Copy the local file(s) or directories specified by path* . . . to the partner (e.g. remote) path specified by partnerTargetPath. The local resources defined by path are expected in the current local subspace on the client. This is typically on the client itself, in the Unix client's Current Working Directory (CWD). This can be changed by specifying OwnStorageServerName and/or OwnSubspaceName in cli.props or as an Ospec as described for Fget above.

The -v option returns a list of transferred files, reporting any overwrite or backup actions.

Fzipput [options] [Ospec] path* . . .   [Pspec]  :partnerTargetPath 
Copy the local file(s) or directories specified by path* . . . to the partner as the .zip file specified by  partnerTargetPath. The partnerTargetPath must have a .zip extension.
The -v option returns a list of transferred files, reporting any overwrite or backup actions.

Fxfer [options] [-bg] [Ospec] :path* . . .    [-ow n] [-bu]  [Pspec]  :TargetPath
 Copy the partner (e.g. remote) file(s) or directories specified by path* . . . to the path specified
 by :TargetPath
 Ospec specifies the global location (Vorg, StorageServer, and Subspace) of the source(s) .
 Pspec specifies the global location (Vorg, StorageServer, and Subspace) of the target .
 (This Pspec is the current global location to which FTM is logically connected, i.e. that
   returned by a Fpwd command.)
The -v option returns a list of transferred files, reporting any overwrite or backup actions.
The -bg option causes the transfer to be carried out in background or asynchronous mode, as a consigned "job"; the request returns a unique xferID to the user client as soon as it is accepted, and the connection to the server is broken.  The user is then notified by email when the transfer has completed - or alternatively,  the user may query the status of such background transfer requests using the FqueryJob or FqueryJobs commands described below.

Finit [-pw PW]
Sets the user's keystore password to the given value. If no PW is specified, a  PasswordDialog box appears prompting the user to enter keystore and  private key passwords, and an alias for the certificate to be used in the keystore. Any existing alias is offered as a probable choice in the PasswordDialog.

Fexit

Removes the keystore password stored within CLI. You might want to do this before leaving your machine for any length of time.

Fsjp -v [-pw PW] [-ow n] [-bu]  ftmJob.xml
Calls a SessionJobProcessor to execute the job defined in the XML document ftmJob.xml in interactive mode. 
All interactions with the server on which ftmJob is to be performed are done "interactively" by the client, using a single Connection to the Server. Any parameters for the job must be  included within the XML document.

 Fjc -v [-pw PW] [-ow n] [-bu]  safeJob.xml
Consigns the safeJob.xml to the Safe server for execution under control of the server in background or asynchronous mode, as a consigned "job". The request returns a unique JobID to the user client as soon as it is accepted, and the connection to the server is broken.  The user is then notified by email when the job has completed - or alternatively,  the user may query the status of such jobs using the FqueryJob or FqueryJobs commands described below.
The description of jobs in xml format as a safeJob.xml may specify the use of the batch facility of the execution platform, or may may consist of a sequence of commands to be run under control of the server. The description and use of the xml job format is beyond the scope of this document.

Control of background requests:
FqueryJobs
Returns a list of all jobs and xfer requests which have been consigned by the user for execution in background mode. Each item listed has the format 
<userID><jobID>              for jobs which are still running,
<userID><jobID>.aborted for jobs which have been aborted by the user,
<userID><jobID>.failed    for jobs which have failed
For background xfer requests, the job name is always xfer, giving an xferID of the form xfer<n>, where n is chosen for uniqueness among all consigned user transfers.
The list is contained in a frame which also includes buttons which allow single selected item to be aborted or purged (removed completely from the server's database).

FqueryJob  jobID
Returns the status of the specified jobID.

FabortJob  jobID
Aborts the specifed jobID.

FpurgeJob  jobID
Purges the record of the specified jobID from the system.  A running job cannot be purged, but must be aborted first.

The Use of Session mode

Session mode is entered using the command:
Fconnect [-v] [-c cl] [-pw PW] [-ow n] [-bu]  [Pspec]
- after which the [:StorageServer/Subspace]:>> prompt appears. The user may then enter

  • Plain Unix commands (e.g. ls, rm, cp, cat, grep - - -) with Unix options 
  • CLI commands:
    • get (or Get)     supports  -v, -c, -e, -ow, -bu, -pr  options 
    • put (or Put)     supports  -v, -c, -e, -ow, -bu, -pr  options
    • xfer (or Xfer)  supports  -v, -c, -e, -ow, -bu, -pr options
    • Rm                  supports  -v  option
    • Mv                   supports  -v, -ow,   -bu  options
  • Special session commands:
    • SS (or ss) SSName
      changes the partner StorageServer of the connection to SSName
    • cd (or CD) directoryPath
      changes the subspace (and therefore the partner (remote) working directory)
      for the session to directoryPath. 
      directoryPath may be any value allowed for the Unix cd function
      (except ~<userLogin>)  and the strings HOME and TMP.
    • pwd
      reports the current working directory on the (remote) partner
    • lcd
      changes the current working directory at the local client
    • lpwd
      reports the current working directory at the local client
    • prompt (or Prompt)  promptLevel
      sets the information displayed in the session prompt e.g.:
      • promptLevel  = vorg:        [@VorgName:StorageServer/HOME]:>>
      • promptLevel = server:        :/StorageServer/HOME]:>>
      • promptLevel = subspace:    /HOME]:>>
    • exit or quit ends the session

Session mode is only supported for connecting to resources at a Safe server (i.e. NOT to another local Workstation).

Example:
a typical session might look something like:

[unixPrompt]$  Fconnect -pw <myPasswd>  -bu  @:hwwfs1.hww.de/
            - (default) PartnerVorgName is  is HLRS
            - (default) Subspace is "HOME"
            - (default) prompt mode is "server"  (show only StorageServer and Subspace)

-[:hwwfs1/HOME]:>>ls -l foo*
           (lists files beginning with foo in user's  home on hwwfs2)
-[:hwwfs1/HOME]:>> cd nuDir
           (changes to <usersHome>/nuDir on hwwfs1 )
-[:hwwfs1/nuDir>> rm foo2
-[:hwwfs1/nuDir>> Rm -v foo3
           (removes <usersHome>/nuDir/foo3, returning verbose information)
-[:hwwfs1/nuDir>> lpwd
           ownPWD:  HOME
-[:hwwfs1/nuDir>> lcd foodir
         
  (changes local cwd on client to foodir)
-[:hwwfs1/nuDir>> get -ow 2 foo4 foo4Got
           (copies <usersHome>/nuDir/foo4 to local foodir/foo4Got with overwrite prevented)
-[:hwwfs1/nuDir>> cd
            (changes subspace to HOME)
-[:hwwfs1/HOME>>  SS crossi.hww.de
           (changes StorageServer to crossi;  subspace remains HOME)
-[:crossi/HOME>> put adir afile .
           (copies adir and afile into user's home on crossi; the "." is essential here!)
-[:crossi/HOME>>  prompt subspace
-[/HOME>>  and so forth

Appendix A

Files used with the FTM Command Line Interface
There are two files, kept by convention in the user's  home/.cli directory, which the user can use to set defaults:

The Properties file cli.props
This file contains definitions of properties, many of which specify default values. The user may wish to set the following values:

  • FileLimit: maximum size (in megabytes) of file which can be stored by a Get into the target in the client's environment. System default is "unlimited".
  • RequestLimit: maximum number of megabytes which can be stored by a Get into the target in the client's environment for all files in the request. System default is "unlimited".
  • NFilesLimit: maximum number of files which can be stored  by a Get into the target in the client's environment by a single request. System default is "unlimited".
  • OverwriteMode: the default mode for overwriting pre-existing files (see the -ow option above):
    •  OverwriteMode=0 specifies that a file may be overwritten freely, as in Unix.
       (this is the system default)
    •  OverwriteMode=1 specifies that a file will be overwritten only by a newer file (this mode is not yet supported).
    •  OverwriteMode=2 specifies that a file will be not overwritten, but that the request will continue without an error. 
    •  OverwriteMode=3 specifies that a file will be not overwritten, and the request will be aborted with an error mode.
  • BackupMode=true  specifies that by default, any overwritten file filename will first be backed up as filename~  (see the -bu option above) System default is "false".
  • KeystoreName specifies the name of the user's keystore within the .cli directory. 
  • KeystorePW specifies the password needed to access the keystore.
  • KeyPW specifies the password needed to access the private key within the keystore (typically the same as KeystorePW).
  • UserAlias specifies the alias of the user within the keystore (typically, this is not needed when the keystore contains only a single public/private keypair. 
  • VerboseMode=true specifies that verbose mode is the default for commands (see the -v option above). System default is "false".
  • Encrypt=true specifies that by default, files transferred over an SSL connection between the Safe server and the client will be encrypted. System default is "false".
  • CompressLevel specifies the default compression level (0-9) for the transfer of files (Get and Put); CompressLevel=0 specifies no compression.  CompressLevel=9 specifies maximum compression. See the -c option above. System default is the default for java jdk.
  • PartnerStorageServerName specifies the default StorageServer at the partner (the subjectof a file action, the source for a Get or the target for a Put). There is no system default.
  • PartnerSubspaceName specifies the default subspace on the partner StorageServer. System default is "HOME".
  • StorageServerDBpath specifies the name of the client's
  • OwnStorageServerName  (used only for Get or Put requests) specifies the name of the StorageServer in the client's environment (the target for a Get, or the source for a Put). If specified, it must be a StorageServer described in the client's storage server database (see below). If OwnStorageServerName is not specified, or is blank, OwnStorageServerName  is the client platform itself. 
  • OwnSubspaceName (used only for Get or Put requests) specifies the default subspace on the OwnStorageServer. 

The user will normally not have reason to set the following values except on installation:

  • SigningAlgorithm specifies the algorithm used for signing requests,
  • JsseProvider specifies the source of the SSL implementation used. 
  • StorageServerDBpath specifies the path of the StorageServer XML database within home/.cli (typically ss_db.xml
  • UserEmail specifies the user's email address. This is currently not used, but will be used to notify the user as to job execution results. 
  • ShPath specifies the default path to the protocol to be used to access StorageServers  within the client's environment (typically /usr/bin/rsh or /usr/bin/ssh). This specification will be overridden if ShPath is defined for the StorageServer in the database specified by StorageServerDBpath. 

The local StorageServer Database:     The local StorageServer database (typically ss_db.xml) within home/.cli specifies the properties of each StorageServer specified as OwnStorageServerName, and reachable "locally" from the client without the use of a Safe server and its SSL connection. Such StorageServers can be called "neighbors" of the user's client machine. The user's loginID on such a StorageServer must be the same as that on the user's client. Such StorageServers are accessed using the protocol specified in the default ShPath or the path specified for the StorageServer in the database. The format of the StorageServer database is defined by the following DTD: <?xml version="1.0" ?> <!DOCTYPE StorageServerDB [    <!ELEMENT StorageServerDB (StorageServer+)>      <!ELEMENT StorageServer (Subspace+)>        <!ATTLIST StorageServer           name           CDATA #REQUIRED           description    CDATA #IMPLIED           address        CDATA #REQUIRED           port           CDATA #REQUIRED           AccessMode    (local|rsh|ssh) "rsh"        >        <!ELEMENT Subspace EMPTY>          <!ATTLIST Subspace            name          CDATA #REQUIRED            description   CDATA #IMPLIED            path          CDATA #REQUIRED            fileLimit     CDATA #IMPLIED            requestLimit  CDATA #IMPLIED            nFilesLimit   CDATA #IMPLIED          > <!--path:          maps the SubspaceName to an absolute path--> <!--fileLimit:     max Megabytes for one file--> <!--requestLimit  max Megabytes for whole transaction--> <!--nFilesLimit:   max No. of files for whole transaction--> ]> The following hypothetical database defines two local StorageServers: <!--This is a DB for demonstrating the definition of local StorageServers --> <StorageServerDB> <!--This defines the client itself, resulting in its access using rsh: -->   <StorageServer name = "localhost" description = "(The client's local storage)"                  address="127.0.0.1" port="8888" ShPath="/usr/bin/rsh">        <!--HOME  is found automatically as the user's homespace on the platform -->        <Subspace name = "HOME"                  fileLimit="20" requestLimit="50" nFilesLimit="15"> </Subspace>        <Subspace name = "TMP" path="/tmp"                  fileLimit="30" requestLimit="100" nFilesLimit="15"> </Subspace>   </StorageServer> <!--This defines a neighbor's machine (named SuperEagle), accessible via ssh. Storage limits for this StorageServer will be the defaults from the  cli.props file. -->   <StorageServer name = "SuperEagle"                description = "Fritz's workstation via ssh (it has my id_rsa.pub in its .ssh/authorized_keys)"                address="188.204.30.30" port="8888" ShPath="/usr/bin/ssh">        <!--HOME  is found automatically as the user's homespace on the platform -->        <Subspace name = "HOME" </Subspace>   </StorageServer> </StorageServerDB> OwnStorageServerName (OSS) may be:    - blank,  undefined, or localhost,  indicating the local storage of the client itself    - the name of a "neighbor" StorageServer registered in the client's StorageServer DB A common case is: - OwnStorageServerName undefined, so that local storage is that of the client itself - PartnerVorgName = HLRS - PartnerStorageServerName defined as the name of a StorageServer in the HLRS Vorg Then  sources for a Put or target for a Get are on the local client, whereas a target for a Put or sources for a Get are on the PartnerStorageServer in HLRS.

 

Table I Escaping Special Characters in FTM paths
CLI
TO pa
  and
from nifti
pa.escape
SC_CHARS
@ &()|><!\"'$;
pa result
(the REALname
of the path)
remoteExec
SPECIAL_CHARS
@ &()|><!\"'$;*?[]\\
special chars which need to be escaped using "\" or protected within quotes
foo\ 9
foo\&
foo\>\ x
foo\>\ \$x
'foo> $x'
foo 9
foo&
foo> x
foo> $x
foo> $x
no escape
(take unchanged)
foo 9
foo&
foo> x
foo> $x
foo> $x
foo\ 9
foo\&
foo\>\ x
foo\>\ \$x
foo\>\ \$x
esc char as literal "\"  is a special case: (protect with ' '   - using " " or esc char not reliable)
foo\\\$x
'foo\$x'
'foo\\\?'
foo\$x
foo\$x
foo\\\?
unchanged
unchanged
 foo\\\?
foo\$x
foo\$x
foo\?
foo\\\$x
foo\\\$x
foo\\\?
Wildcard chars as wildcards (expanded matches may need escaping)
:foo?
'foo?'
"foo?"
foo\ \$?
foo?


foo $?
foo?


foo\ \$?
foo1
foo?
foo\
foo $9  et. al.
foo1
foo\?
foo\\
foo\ \$9    et. al.
Wildcard chars escaped as literal
foo\\\?
'foo\?'
"foo\?"
foo\?
foo\?
foo\?
foo\?
foo\?
foo\?
foo?
foo?
foo?
foo\?
foo\?
foo\?
NOTES:
  • only '   and    `   are illegal chars
  • CLI must provide PathArray elements in a form which yields the form in column 2 of the table;
  • NIFTI GUI  provides the elements already in the form of column 2
  • Most special characters (e.g. blanks) can be escaped using the escape character "\"
    It is often safer (especially for paths containing "\" and "$") to enclose the path in single quotes.
  • pa.expand expands any elements containing unescaped wildcard chars:
    • for elements containing wildcard chars,
      • all chars SC_CHARS (all special chars EXCEPT WCs and "\") are escaped e.g. element foo $? must be escaped to foo\ \$? so that it can be correctly listed.
      • the element is listed via remoteExec in order to expand the wildcard chars.
      • when pa.expand of wildcards results in matches containing SCs or WCs, these must be appropriately escaped so that they will be interpreted as literals when used by remoteExec etc.
    • other elements are simply taken verbatim.
  • for remoteExec, execute, etc., escapes are done with SPECIAL_CHARS (all Special chars including WCs and "\") e.g. for existing file foo\?   we must use escaped form foo\\\? (both the \ and the ? are escaped)