EDpCloud

Sync, Replicate, Protect Data

EDpCloud™: File Transfer Command Line Interface For Sending Files and Directories.

• NAME
• SYNOPSIS
• DESCRIPTION
• FILE SYNC QUEUING OPTIONS
• EXAMPLES Queuing a directory for replication and transfer
• FILES
• SEE ALSO
• RETURN VALUE
• ENVIRONMENT
• SUPPORT
• AUTHOR

NAME

edq - Initiate/Queue Folders And Files For Synchronization, Transfer, Replication and Distribution

SYNOPSIS

edq [-l linkname] [-r receiver] [-s source_node ] [-p port ] [-w password] -n file_or_dir_name|-f filelist [-R]

DESCRIPTION

edq: The command line interface for initiating File Transfer, content distribution or data replication. File and Folder synchronization starts immediately after edq is called (unless your replication link is paused).

edq allows you to queue files or directories and their content for transfer and distribution to remote sites. Only portions of the files that have changed will be sent to the remote site(s).

FILE SYNC QUEUING OPTIONS

-n file_or_dir_name

The file or directory name argument is mandatory. EDpCloud will queue the file for transfer and synchronization if the file matches the includes regular expression and it does not match the excludes regular expression. Directories and subdirectories and their entire content are also processed. edpcloud will rebuild only the files that match your file transfer criteria and that have changed since last time you initiated replication and content synchronization. This option is mutually exclusive with -f option.

-f filelist

This parameter allows you to queue the list of files in filelist for transfer. The format is one path name per line. This option is mutually exclusive with -n option.

if -R is specified along with -f then the filelist will be treated like a real time. This means that deletes, renames, etc will be accepted.

-w password

This is the password for the source node or the Graphical User Interface. This will be verified against what is in $ED_BASE_DIR/etc/edpasswd on the source node.

The format of the password file is "hostnamepattern:password". The password file is a text file and the password is in clear text. Therefore the password file must be read only by the owner or system administrator.

If no password is specified and the edpasswd file is not found, the wild card "*" will be used as a password.

-s source_node

This is the source node that synchronizes the content. The default is the localhost. The source node is also referred to as the manager node. Data synchronization will be initiated from the specified source node for the specified link and receiver.

-l linkname

This is the name of the link we are controlling.

The receiver/destination host and directories are identified by a linkname and a hostname. Link names are unique across a configuration. The wild card ".*" indicates that file and folder synchronization will be initiated for all links. You can use regular expressions to initiate file sync for only the links that match the link expressions/patterns. For instance, you can use -l "casa.*" to match all links that start with the word casa. If no link is specified, the default is to initiate replication for the specified receivers for all links (.*) in a configuration.

-r receiver

receiver is the hostname or IP of the remote receiver. It must match what is in the receiver section of eddist.cfg This is identified by the hname="hostname" or hostname="nodename" in the receiver section of the configuration. The wild card ".*" indicates that the command will be applied to all receivers. This is the default. You can also use regular expressions for the receiver(s) name(s). For example, you can use -r *.noaa.gov to match all receivers on the noaa network or 192.168.* to match nodes with IP's in the range 192.168.0.1 to 192.168.254.254

The parameter specified by -r must match what you have in the hname="parameter" or hostname="parameter" entry of your receiver entry in the configuration file eddist.cfg.

[-p port]

This is the port on which the manager node (source node) is listening for control commands.

[-Z]

Use ssl certificates

EXAMPLES Queuing a directory for replication and transfer

edq -n c:\data

This will queue c:\data and all of its content for synchronization to all links and all receivers. Since we did not specify a source, the local host will be used and thus we will distribute data in c:\data from the node where you executed edq -n c:\data to all links and all receivers in your eddist.cfg.

edq -l tokyo -r backups.enduradata.com -w secretword -n /data/japan

This causes the local host to initiate distribution of /data/japan and its content for link tokyo to receiver backups.enduradata.com.

edq -l link1 -r bridge2 -s www.enduradata.com -w mypassword -n /home/web/collaboration

This will distribute data in directory /home/web/collaboration on node www.enduradata.com to link1 and receiver bridge2, using password mypassword.

edq -r *.nasa.gov -s nasa1.nws.gov -n /nws1/sensors

This will replicate and distribute data from folder /nws/sensors, node nasa1.nws.gov to all links and registered receivers in the nasa.gov domain.

Example of a crontab entry for Linux/Mac/Unix users

Although edpcloud comes with its own scheduler, users can schedule tasks using Linux/Unix cron facility. This entry in crontab shows how replication for link casablanca and receiver paris.enduradata.com will be initiated at 06:00 every day of the year. Windows users can also use the at command or the job scheduler console to schedule data distribution. The same cron entry also shows how to pause and resume distribution. File synchronization will be paused at 9:30 am and resumed at 11:25 am.

Example of back up of a mysql database

mysqldump -u root --password=xxxx database > database.out edq -l remotelink -n database.out

This can be run from a schedule or from a command line.

0 06 * * * edq -l casablanca -r paris.enduradata.com -n /plant1/inputs

30 09 * * * edpause -l casablanca -r paris.enduradata.com

25 11 * * * edresume -l casablanca -r paris.enduradata.com

FILES

$ED_BASE_DIR/etc/eddist.cfg

$ED_BASE_DIR/etc/edpasswd

SEE ALSO

edintro(8)

edresume(8)

edpause(8)

edjob(8)

eddist(8)

edstat(8)

edmfq(8)

eddist.cfg(5)

RETURN VALUE

The status is 0 on success.

ENVIRONMENT

ED_BASE_DIR: This is the name of the top directory where the file replication and filesync software is installed. This directory will also contain various configuration, journals and log files.

ED_COLLECTOR_PORT: This environment variable is set to the port on which EnduraData eddist server listens to management commands. Set this environment variable to the port on which to listen for commands.

ED_DISTRIBUTION_PORT: This environment variable is set to the port on which EnduraData eddist server listens to data synchronization services. Set this environment variable to the port on which to listen for data distribution connections.

SUPPORT

For more information contact <support@enduradata.com>

AUTHOR

A. A. El Haddi, elhaddi@ieee.org