Linux Real Time File Replication Installation and Configuration
In this article, I will discuss how to configure EDpCloud Linux Real Time File Replication and Synchronization to move, transfer and sync files automatically from one or multiple Linux servers, laptops, or desktops to a remote site for data protection or other workflow purposes (The remote site can be Linux, Windows, Solaris, etc.).
Download A free Demo Copy of Linux, UNIX or Windows Replication Software
Download EDpCloud real time data replication software and a free demo license
First, run uname -a. If you have a 32-bit OS, download the 32-bit package. Otherwise, download the 64-bit package
Extract the tar file and run the real time file replication software install script:
First, install the Linux real time file replication:
tar xvf packagename.tar.gz cd enduradata_edpcloud ./install.sh
You will be prompted for some parameters, such as the install directory. Use the defaults for now, but don’t start the file replication services/daemons yet. We will do that after we configure Linux, real-time file replication software.
cd to the edpcloud bin directory (under the install directory/edpcloud/bin) and source in the script to set up the environment (this will be done automatically later)
. enduradata_env Please note that once you source in the env, your paths are set and some aliases are created for you. IMPORTANT: INSTALL FUSE ONLY IF YOU DO NOT have it Install one of the Linux real file replication components: ./edinstallfuse.sh
If your linux-utils is older than 2.18, then touch the file below in the bin directory to make sure the correct mount is used (To find out the version of your linux-utils: run mount –version to find out the version):
. enduradata_env touch ${ED_BASE_DIR}/bin/has_old_mount_version If you had to do that, you must update your operating system. Set auto start after reboot: $edpcloud/bin/autostart/sys5debian_autostart
Install the real time file replication license by copying it to:
${ED_BASE_DIR}/etc/edlicense
Configure
- cd to the edpcloud/etc directory and add the directories you want to replicate in real-time to edfsmonitor.cfg
Example: /data1 /data2 /data1 and /data2 entries must exist in the file system since these will be mount points.
- cd to the edpcloud etc directory and add the top directories( you want to replicate in realtime or in scheduled mode) to the includes file (use regular expressions)
Example of $edpcloud/etc/includes ^/data1/.* ^/data2/.*
You will need to add other directories if you want to replicate them (in real-time, on-demand, or in a scheduled manner).
- cd to edpcloud/etc directory and create eddist.cfg
Example: To replicate data in real-time from any localhost to 192.168.32.50 and put all data in /media/bob/backup/%ip%”. The “%ip%” macro will be converted at run time by the receiver to use the sender’s IP. Make sure the directory exists and that the “.” in the IP is an underscore and not a dot. You will see the receiver log complaining if the directory does not exist (see example below)
mkdir -p /media/bob/backup/192_168_32_2 #if the sender is 192.168.32.2
<?xml version="1.0" encoding="UTF-8"?> <config name="enduradata" password="zooo" > <link name="or" > <sender hostname="localhost" alias="*" /> <receiver hostname="192.168.32.50" storepath="/media/bob/backup/%ip%" /> </link> </config>
Your server’s Linux real time file replication configuration is done.
- Start edpcloud services now using the following command:
cd $installdir/edpcloud/bin . edpcloud.sh startall
Verify that edpcloud file replication software is running
edstat df
edstat will show you the status, the number of files in the journal, the bytes on the move, etc.
Examine the output of df, and you will see that edpcloud mounted both /data1 and /data2.
- If for any reason, edstat is showing any errors, then you need to
examine eddist.log under the edpcloud logs directory (other useful logs are edfsmonitor*log, ed_receiver*log, ed_sender*log). - Test that file replication is taking place in real time and that your local data under /data1 and /data2 is being synchronized as it changes on the localhost to the remote server:
- Now go to /data1 and touch a file or write to it, rename it, etc., and you will see your changes replicated to the remote host. Please note that deletes are not propagated by default.
Propagating deletes to the remote host
By default, to protect against misuse, deletes are not propagated.
If you want deletes to be propagated, then add the deletes=”1″ to the configuration as shown below.
<?xml version="1.0" encoding="UTF-8"?> <config name="enduradata" password="zooo" deletes="1" > <link name="or" > <sender hostname="localhost" alias="*" /> <receiver hostname="192.168.32.50" storepath="/media/bob/backup/%ip%" /> </link> </config> By default the top directory as indicated by storepath="/media/bob/backup/%ip%" must exist unless you edited enduradata_env to add: ED_CREATE_STOREPATH=1 export ED_CREATE_STOREPATH Another directive to create the top directory for storepath is to create file createpath under $edpcloud/etc: touch $edpcloud/etc/createpath
You will need to perform the initial synchronization to copy the local files and directories first:
edq -l or -r 192.168.32.50 -n /data1
edq -l or -r 192.168.32.50 -n /data2
You may also use edmfq for better control of the file queueing for replication and synchronization.
You will need to perform the initial sync only once, and after that, all file changes on /data1 and /data2 on the localhost will be sent to the remote host i in real time.
If you want to synchronize and replicate files from multiple servers to the same remote host, install and drop the files you created in the edpcloud/etc directory on each of the servers, run edpcloud.sh restartall, and you are on your way to consolidating your backups, aggregating your content or whatever you want to achieve (Don’t forget to get the right package and to run the following command (IF and only if your mount version is older than 2.18)
touch ${ED_BASE_DIR}/bin/has_old_mount_version
Your Linux real-time file replication and synchronization software are installed and configured. EDpCloud keeps your many servers in sync to automate your business processes and data protection using file replication for online backup. Make sure you visit again and read about creating file archives.
Next time, we will talk about archiving file changes. Until then, be safe, but above all, protect your data.
If you like this article, please share it. We would also love to hear from you.
–a
Additional articles you may find interesting:
https://www.enduradata.com/file-replication-software-a-cross-platform-solution
- Data Management Challenges in HealthCare
- The Future of Data Replication: Trends and Innovations
- How to troubleshoot Linux file replication connectivity issues
- EnduraData Releases EDpCloud Version 6.0.9: Enhanced Linux Data Replication and File Transfer Solutions
- Factors That Impact File Replication and Synchronization Performance
- Enhancing SSA’s Data Systems with EnduraData: Mastering Linux File Replication, File Synchronization, and Web Farm Data Replication
- Future-Proofing Your Data Strategy: The Essential Role of File Replication in Navigating Cloud Complexity
- Linux File Replication Installation and Configuration
- EnduraData: Your Comprehensive Data Protection Solution for Regulatory Compliance
- Debunking 5 Common Myths About Data Management: What Every Business Needs to Know
- EnduraData EDpCloud on SUSE SLES-15-SP4-64bit Linux (SUSE.COM) Certification Overview
- How System Administrators Can Utilize EnduraData EDpCloud for Robust Data Backup
- Risks of Not Replacing RepliWeb file replication: Impact of Discontinued Support and Unavailability of Software Updates
- Leveraging File Replication for Efficient Data Migration: Best Practices and Strategies
- Best Practices for Implementing Enduradata File Replication in Enterprise Environments
Share this Post