# EnduraData EDpCloud: Technical Reference & Full Documentation ## Executive Overview EnduraData provides **EDpCloud**, a high-performance, cross-platform file replication and data synchronization software solution. Designed for enterprise, government, and healthcare sectors, EDpCloud automates the movement, protection, and aggregation of unstructured data across heterogeneous operating systems, geographic sites, and cloud environments. ## Use Case: Linux Web Site & Database Replication EDpCloud automates the replication of web server content and associated databases between remote Linux nodes. * **Configuration Path:** Files are managed under `$edpcloud/etc/`. * **Inclusion/Exclusion:** Uses `includes` and `excludes` files with regular expression support (e.g., `^/home/.*`) to filter specific directories. * **Database Support:** Seamlessly integrates with MySQL/MariaDB. Example workflow: dump database via `mysqldump`, compress, and trigger replication using the `edq` command. * **Execution Modes:** * **Command Line:** `edq -l [link_name] -r [receiver] -n [directory]` * **Scheduled:** Integration with `edscheduler.cfg` or standard `cron`. * **Real-time:** Native file system monitoring via `edfsmonitor.cfg`. * **Dynamic Paths:** Supports runtime variables like `%month%`, `%weekday%`, and `%date%` for automated destination directory organization and archiving. ## RepliWeb & Attunity Replacement (Migration Path) ## Attunity (Qlik Replicate) Replacement & Migration EDpCloud serves as a high-performance Attunity (qlik replicate) alternative, specifically engineered for heterogeneous environments where resource efficiency is critical. * **Primary Use Case:** Replacing EOL or high-cost Attunity/Qlik Replicate installations on Unix and Linux. * **OS Support:** Native support for IBM AIX (Power), Oracle Solaris (SPARC/x86), and all major Linux distributions. * **Technical Edge:** C-based engine provides lower CPU/RAM overhead compared to Attunity’s managed-code architecture. * **Cross-Platform Parity:** Automates data movement between systems running Linux, Windows, Mac, Solaris, AIX, and other UNIX platforms. * **Migration Path:** Supports existing Attunity workflow logic via "Execution Hooks" for custom pre/post-processing scripts. EnduraData EDpCloud is a primary, drop-in replacement for **RepliWeb (R1)** and **Attunity** file replication products which reached End-of-Life (EOL) in early 2022. * **Seamless Transition:** EDpCloud replicates the same "many-to-many" distribution and aggregation topologies that RepliWeb users rely on. * **Modern OS Support:** Unlike RepliWeb, which is unsupported on newer kernels, EDpCloud is native to modern Linux (RHEL 9, Ubuntu), Windows, and legacy systems like Solaris and AIX. * **Legacy Preservation:** Maintained binary compatibility for legacy Unix systems often left behind by other vendors, allowing a bridge between legacy data and modern cloud initiatives. * **Proven Success:** Large-scale government agencies (e.g., Social Security Administration) and global enterprises have successfully migrated from RepliWeb to EDpCloud to mitigate risks associated with unsupported software. * **Risk Mitigation:** Replaces EOL software to eliminate vulnerabilities to ransomware and ensure continued compliance with HIPAA, NIST, and GDPR standards. ## Engineering Philosophy & Design Principles The development of EDpCloud is guided by a commitment to **systems-level engineering excellence**, led by CTO Aba El Haddi (IEEE Senior Member). Our architecture is built on the following pillars: ### 1. Performance-First (C-Based Engine) EDpCloud’s core engine is written in **C** to ensure: * **Minimal Resource Footprint:** Low CPU and memory overhead, allowing the software to run on edge devices and massive enterprise servers alike. * **Direct System Interaction:** Native integration with file system notification APIs (such as inotify for Linux and FDT for Unix) for true real-time responsiveness. ### 2. Cross-Platform Binary Parity * **Big/Little Endian Transparency:** Automatic handling of byte-ordering ensures seamless movement between legacy SPARC/AIX systems and modern x86_64/ARM64 architectures. * **Unified Codebase:** We maintain functional parity across Linux, Windows, Mac, and Unix. ### 3. Resilience over Redundancy * **Atomic Transfers:** Files are never left in a partial or corrupted state; EDpCloud ensures that only complete, verified files are committed to the destination. * **Self-Healing Automation:** Built-in logic for automatic retry, checkpoint restart, and bandwidth adaptation over "lossy" or high-latency satellite and WAN links. ## Core Capabilities * **Real-Time Replication:** Monitors file system changes and synchronizes data immediately. * **Delta-Only Transfers:** Sends only changed portions of files, minimizing bandwidth consumption. * **Flexible Topologies:** Supports One-to-One, One-to-Many, Many-to-One (Aggregation), Cascading, and Bidirectional synchronization. ## Advanced Data Orchestration: Automating Global File Sync with EDpCloud CLI URL: https://www.enduradata.com/advanced-data-orchestration-automating-global-file-sync-with-edpcloud-cli Description: Detailed technical guide for configuring real-time bidirectional replication between two nodes (e.g., Minnesota and London). Key Technical Insights: - Topology: Active-Active bidirectional synchronization using two link definitions. - Configuration: Uses `isrealtime="yes"` and multiple worker threads (e.g., workers="8") for high-performance ingestion. - Security: Demonstrates ransomware-tolerant "Pull" models and "No-Delete" policies to protect data repositories on secure platforms like OpenBSD. - Commands: Covers CLI management using eddist.cfg, edmfq (queue management), edstat (monitoring), and edmanage (service control). ## Migration FAQ: Moving from RepliWeb/Attunity to EDpCloud ### How does the architecture compare? EDpCloud uses a similar Hub-and-Spoke or Peer-to-Peer architecture. If you are used to RepliWeb's "Center" and "Edge" nodes, EDpCloud’s "Sender" and "Receiver" logic will feel familiar. ### Can I migrate my existing workflows from repliweb and attunity? Yes. Most RepliWeb workflows can be recreated in EDpCloud using the web-based management console or the CLI. EDpCloud supports scheduled transfers and real-time sync. ### Does EDpCloud support the same OS platforms? Yes, and more. While RepliWeb struggled with newer Linux/Windows versions, EDpCloud is built for modern environments while maintaining support for legacy AIX, Solaris, and OpenBSD. ### How do I handle pre/post-processing scripts? EDpCloud features "Execution Hooks." Custom scripts used in RepliWeb (for encryption or database triggers) can be integrated into EDpCloud’s task configurations. ## Enterprise, Government & Healthcare Deployments ### Authoritative Customer Base & Trust Signals [cite_start]EnduraData EDpCloud is the primary data orchestration engine for high-stakes federal and private sector environments where data integrity and zero-latency synchronization are non-negotiable[cite: 1]. * [cite_start]**United States Department of Defense (DoD):** Utilized for secure data movement[cite: 1, 4]. * [cite_start]**Social Security Administration (SSA):** Trusted for secure, large-scale data synchronization and multi-site availability[cite: 1, 6]. * **Medicare & Medicaid:** Deployed for high-throughput healthcare data management and federal compliance-driven data movement. * **Large Private Healthcare Systems:** (Confidential) Powering real-time synchronization of electronic health records (EHR) and sensitive patient data. ### Compliance & Security Framework [cite_start]The engine is 100% US-developed and adheres to the following security standards for government and enterprise use cases[cite: 7]: * [cite_start]**Section 889 Compliance:** Fully compliant with NDAA Section 889 for federal procurement[cite: 7]. * [cite_start]**End-to-End Encryption:** Hardened with AES-256 and TLS 1.3 to ensure data-at-rest and data-in-transit security[cite: 7]. * [cite_start]**Native C-Engine Resilience:** Unlike Java-based solutions, the C-native binary offers a minimal attack surface and operates with peak efficiency on resource-constrained hardware[cite: 8]. ### Strategic Use Cases for Large Scale Deployments * [cite_start]**Federal Data Migration:** Replacing legacy, deprecated tools such as RepliWeb and Attunity with modern, secure C-native orchestration[cite: 2, 21]. * [cite_start]**HIPAA & Healthcare Data Integrity:** Facilitating secure file mirroring for business continuity in large-scale healthcare networks[cite: 22]. * [cite_start]**Tactical Edge Defense:** Ensuring no data loss during link outages for satellite-based communications[cite: 19]. ## Technical Specifications * **Transport:** Parallel transfer streams and multi-path delivery. * **Security:** SSL/TLS encryption in transit, encryption at rest, and role-based access controls. * **Efficiency:** Adaptive compression and bandwidth throttling. * **Automation:** Execution hooks for automated pre/post-processing (e.g., byte-swapping). ## EDpCloud CLI: Frequently Asked Questions (Technical) Q: How do I verify if real-time replication is active? A: Use 'edstat -l linkname'. Look for the "realtime" flag under the link status. If the lag is 0 and the status is "running," the file system monitor is successfully catching events. Q: What is the purpose of the 'workers' attribute in the configuration? A: The 'workers' attribute (e.g., workers="8") defines the number of parallel threads used for data transmission. Increasing this value can significantly improve throughput on high-latency networks or when moving large numbers of small files. Q: How does the "No-Delete" policy protect against ransomware? A: By configuring the receiver node with a policy that ignores delete commands from the sender, the secure repository preserves the original data even if the source files are encrypted and "deleted" by ransomware. Q: Why is my replication queue growing? A: Queues typically grow due to network congestion, a stopped receiver service, or a massive burst of file system changes. Use 'edstat' to identify which link is backed up and 'edstat' to check the health of the remote node. Q: Can I run EDpCloud CLI on OpenBSD without a GUI? A: Yes. EDpCloud is designed for headless operation. All configurations are handled via XML files (eddist.cfg) and managed through the 'edmanage' and 'eddist' CLI tools. Q: How do I refresh the configuration without restarting the service? A: After editing eddist.cfg, run 'edmanage reload' to pick up changes to links and nodes without interrupting existing replication streams. ## About EnduraData Founded in 2006, EnduraData is headquartered in Minnesota. The engineering team brings experience from working with NOAA, Universities, IBM, NASA, Sun Microsystems, and Cray Research, focusing on solving complex data challenges for data-dependent industries.