DC ⇄ DR WAN Optimizer

The next full backup crossed the link in 8 seconds, not 89.

A software WAN optimizer for replication between a data centre and its disaster-recovery site. It sends only what the other side does not already have, compressed, over the link you already pay for. Backup jobs, SMB copies and SQL Server backups keep working unchanged.

Data centre tonight's full backup DC optimizer one authenticated tunnel over your WAN outlined: a reference to data DR already holds · solid: new data, compressed Recovery site rebuilt from its own store DR optimizer · verified
1 GiB VM image, next full backup with 3% changed100 Mbit/s link, 20 ms RTT
Direct SMB copyevery byte crosses
89.0 s
Through the optimizer5.1% of the bytes cross
8.0 s
Drawn to scale. Lab measurement on an emulated WAN, SHA-256 verified.11.19× faster
Request an evaluation See it running

Everything it does, live, on one screen.

The operations dashboard reads both optimizers once a second. This is a real recording: 9.7 GiB carried through the tunnel, with the optimizer itself holding the WAN to 200 Mbit/s.

WAN Optimizer Operations
Operations dashboard: 68.8% of WAN traffic avoided, 9.7 GiB of application data and 3.0 GiB on the wire, a 200 Mbit/s cap held at 199, a 100% chunk store hit rate, zero reconnects, authentication failures, stalls, watchdog resets, unrecoverable chunks and errors, and a list of recent streams. 1 2 3 4 5 6
  1. 68.8% never crossed the link6.7 GiB of 9.7 GiB, split into deduplicated, compressed, sent and passed-through bytes.
  2. A hard cap on WAN bandwidthOne total for every stream and lane, so replication never takes the whole link.
  3. The pipeline, second by secondChunking, hashing, compression and store rates, with CPU and event-loop delay.
  4. The DR store at a glanceSize, 100% hit rate, write and read p99 in microseconds, index memory.
  5. Health you can auditReconnects, authentication failures, stalls and watchdog resets: all zero here.
  6. Every stream explainedWhich streams were optimized and which took the fast path, and why each one closed.

Recorded with both optimizers on one machine: the savings are real, the link speed is the optimizer's own 200 Mbit/s cap rather than a physical WAN.

The gain depends on the data. Here is how much.

The same 1 GiB SMB copy, direct and through the optimizer, over the same 100 Mbit/s link. Bar length is transfer time.

Direct copyThrough the optimizer
VM image, next full3% changed since the last backup
11.19×
Logs and textcompression only
3.70×
VM image, first fullempty store on the DR side
2.32×
Encrypted or compressednothing left to remove
1.00×

On a repeat transfer

99.7%

less WAN traffic at best, 97.1% at worst, across the lab's repeat runs: last week's backup is already at DR, so the next one is mostly references.

On a long, lossy link

4.5–5.9×

faster first transfers with BBR congestion control at 80 ms RTT and 0.1% loss, and a disk image at 10.7 MiB/s over a link that tops out at 5.96 MiB/s.

Never slower on data it cannot help

1.00×

Every 32 MiB is measured. Video, archives and encrypted data go through raw, skipping chunking, hashing and storage.

Built for SQL Server backups, not just file copies.

A file transfer mode picks up .bak and .trn files from the DBA's backup folder and delivers them to the DR receive folder, keeping full and log backups in order.

SQL Serverbackup folder.bak .trn Readiness markerbound to this exactbackup generation DC senderreads read-only,1 MiB at a time DR receiverevery block durable,root hash verified Receive folderpublished atomicallyby rename tunnel
  • Never touches the source. Backup files are opened read-only, and a test proves no write reaches the DBA's folder.
  • Only finished backups move. A marker tied to the exact backup generation releases a file; a file that changes afterwards fails closed.
  • Log chains stay intact. Full and log backups are classified, log backups get priority, and a gap in the chain is reported, not hidden.
  • Nothing half-written appears at DR. A file is published only after every block is durable and the root hash matches.
  • Resumes after a break. Resume identity is bound to the source generation, so a newer backup never resumes an older one.
  • Survives a share that disappears. An unreadable source is retried with backoff instead of being read as "all backups deleted".

Everything a server room needs from it.

Deduplication by content

FastCDC-style chunking with a 16 KiB average, identified by SHA-256: a 3% change stays a 3% change on the wire.

Asynchronous compression

zstd, brotli or deflate on worker threads with adaptive levels; data that does not shrink is sent as is.

Adaptive fast path

Each 32 MiB window is judged; incompressible data skips the pipeline, so it costs neither time nor CPU.

Multiplexed, striped tunnel

Many streams with their own flow control on one tunnel, striped over up to 16 TCP lanes for lossy links.

WAN bandwidth cap

One token bucket for the whole optimizer, with bounded bursts; control and health frames are never delayed.

Peers that prove themselves

HMAC-SHA256 challenge-response binding both nonces, roles and node ids; nothing is touched before it passes.

Multi-terabyte store

A disk-backed chunk log with a compact index at about 54 bytes per chunk, rebuilt at 1.6 million records a second.

Self-healing streams

Stalls are warned at 60 s and reset at 120 s, the DC reconnects with backoff, and a torn store tail is repaired at start.

Full telemetry API

Per-stream state, store hit rates and latency histograms, a per-second pipeline profile, and optional PostgreSQL history.

Deployment that repairs itself

A Windows deployment tool with ten actions, dry run and a catalog of 32 known errors, each with its cause, repair and rollback.

Firewall by the book

Exactly four flows, each to one address and one port; SQL's 1433 and SMB across the WAN are refused outright.

Installs offline

Pure Node.js with no native modules: an offline bundle, a SHA-256-checked runtime installer, nothing downloaded on the server.

Large gains

  • Nightly and weekly full backups to a DR site
  • SQL Server full and log backups
  • VM images (VHDX, VMDK) and database dumps
  • Narrow or distant links where a full copy takes days

Little gain, no harm

  • Encrypted or already compressed archives
  • Video, ISO and installer images
  • SMB3-encrypted sessions

Runs on

  • Windows Server, as a supervised service
  • Linux and Docker
  • Any WAN, under IPsec or WireGuard

Prove it on your data before you pay.

The analyzer measures a sample of your data and says plainly whether it is suitable, including when it is not. Every transfer it reports on is checked byte for byte.

Data analyzer output: 768 MiB sample, 54.5% compressible, 13.6% duplicate, first transfer 2.53 times faster, 10 TiB in 4 days instead of 10, classified partially suitable and below the 4x target.
The data analyzer on a sample: 10 TiB in 4 days instead of 10, and an honest "below the 4× target".
End-to-end test: seven scenarios, every SHA-256 ok, ratios from 1.00x for random data to 23.26x for the next full VM backup and 54.61x for a repeated restore.
The end-to-end test through a real tunnel: seven scenarios, every one SHA-256 verified.

  1. Lab

    Windows SMB gate at 100 MB, 1 GB and 10 GB, on three different days.

  2. Staging

    100 GB and 1 TB direct, cold and warm, then a 2 TB soak over three days.

  3. Pilot

    Your data, your link: analyzer, direct baseline, optimized cold and warm.

  4. Production

    Store sized from the pilot numbers, with monitoring from day one.

Version 2.4.0, 263 automated tests. Validated in a Docker WAN lab and on Windows SMB; measured transfers so far go up to 1 GiB per file. The gates above are how a deployment earns larger numbers.

Buy it, license it, or run a pilot on your own sites.

The source code is not published. It comes with the dashboard, documentation, test lab, sizing tools and Windows deployment scripts.

PurchaseFull source code and rights.
LicenceUse it inside your organization or ship it in your product.
PilotA supervised evaluation on your DC and DR links.
Email Mohamed Mohsen Shehata LinkedIn

mo7sen.cto@gmail.com

مُسرّع شبكات بين مركز البيانات وموقع التعافي من الكوارث

يُثبَّت على طرفي الخط، فيرسل فقط ما لا يملكه الطرف الآخر، مضغوطًا. ونسخ SMB والنسخ الاحتياطي لقواعد بيانات SQL Server تعمل كما هي دون تعديل.

  • في اختبارات المعمل على خط 100 ميجابت: النسخة الكاملة التالية لصورة جهاز افتراضي أسرع 11 مرة، والسجلات والنصوص أسرع 3.7 مرة.
  • لوحة تشغيل حية، ونقل ملفات ‎.bak و‎.trn مع الحفاظ على تسلسل السجلات، وحد أقصى لاستهلاك الخط.
  • كل عملية نقل يُتحقق منها بـ SHA-256، مع أداة تقدّر المكسب على بياناتك قبل الشراء.

الكود المصدري متاح للبيع أو الترخيص أو التجربة على بيئتك: mo7sen.cto@gmail.com