Key findings and conclusions:
- In side-by-side testing with comparable product, the SG-4 shows superior network performance as services are enabled
- SG-4 demonstrated the ability to perform packet forwarding, firewall, and NAT functions with comparably less overhead than the ISR under similar network load conditions
- Maintained a highly serviceable level of LAN throughput simultaneously with a fixed 45Mbps VPN connection across a WAN
- SG-4 demonstrated a live service upgrade, without reboot, on a single module
NetDevices of Sunnyvale, CA engaged Miercom as an independent agent to verify the operational features of its NetDevices SG-4 Services Gateway, and a similar offering The Miercom performance testing found the NetDevices SG-4 Services Gateway distinguished itself in several aspects: In the key result, the SG-4 shows comparably better network throughput, accomplished with successively increased overhead of active network services, and over increasing packet sizes range.
The SG-4 showed sustained balancing of branch office-level traffic, achieving 97.5% forwarding and firewalling of local Gigabit Ethernet Ciscoâ„¢ LAN traffic while maintaining a fixed 45 Mbps VPN WAN link.
The SG-4 demonstrated a "hot-swappable"?, no-reboot software upgrade. In addition, network service modules can be upgraded
independently, for increased control of upgrade scheduling.
How We Tested
Service Gateway Performance. The test bed was designed to compare key performance aspects of the NetDevices SG-4 Services Gateway and the Cisco ISR 3845. The throughput test traffic was generated by a SmartBits 2000 placed in a loop with the SUT via fiber optic GigE connections. The ingress and egress connections from the SUT originated on copper, through a NetGear Media Converter, to fiber optic GigE. Test traffic was UDP protocol packets over statically configured ports, with successive packet sizes of 64B, 512B, and 1500B. Each packet size was tested with the successive addition of network services, including combinations ACLs from 100 to 5000, NAT, QoS, and DoS detection. Test measurements were reported by Spirent SmartApplications v3.00. The VPN WAN test was standalone, accomplished with the SG-4 connected to a comparable NetD SmartBits sent 1 flow per interface, using two interfaces. Four slots total were used, 2 for each bidirectional flow, also copper
converted to fiber optic, with one flow for unencrypted 1500B "local"? traffic and another for encrypted 1400B IPSec WAN traffic maintaining a fixed 45Mbps bandwidth. Results were captured with the SmartWindows feature of Spirent SmartApplication.
Results
The series of performance tests, with their successive combinations shows a trend that the SG-4 maintained a stable, undegraded level of route forwarding as the packet size and number of network services increased. In the benchmark tests, the SG-4 outperformed the ISR on raw packet forwarding (no services) by 30+% (64B packets), 892K to 600K. This trend continues, with the ISR showing less comparative throughput at 512B and 1500B packet sizes. The SG-4 exhibited a throughput rate change from 20% to 16% in the 64B test over the successive activation of the three significant services measured (100 ACLs, NAT, then QoS). The Cisco ISR displayed a rate change starting at 9% throughput down to 3%. In the 512B test cycle, the SG-4 maintained a rate at or above 90% with each service enablement (90%, 96%, to 91%) comparing to the 3845’s measured rates at less than 60% over the same sequence (56%, 27%, 18%). The favorable comparison of NetDevices continued at the higher packet size of 1500B with the SG-4 pinning at 100% throughput over the increased service enablements with the 3845 dropping to almost 52% of full throughput with all three services active (refer to chart p.1).
The VPN IPSec standalone test showed significant favorable performance in the forwarding rate on the LAN traffic, 160Kps (97.5% full throughput) while maintaining a hefty 45Mbps VPN connection across a WAN. A version change of the QoS module of the SG-4 was accomplished while maintaining a ping conversation between attached PCs, demonstrating no indication of an outage, and requiring no reboot for the module to activate. In comparison to the zero-time "hitless"? upgrade, the ISR shows a time of well over two minutes for a required upgrade reboot and access to the interface. The SG-4 base system is priced at $7,990; compare $14,395 for the 3845 ISR with the advanced security package.
About Miercom’s Product Testing Services…
With hundreds of its product-comparison analyses published over the years in such leading network trade periodicals as Business Communications Review and Network World, Miercom’s reputation as the leading, independent product test center is unquestioned. Founded in 1988, the company has pioneered the comparative assessment of networking hardware and software, having developed methodologies for testing products from SAN switches to VoIP gateways and IP PBX’s. Miercom’s private test services include competitive product analyses, as well as individual product evaluations. Products submitted for review are typically evaluated under the "NetWORKS As Advertised™"? program, in which networkingrelated products must endure a comprehensive, independent assessment of the products’ usability and performance. Products that meet the appropriate criteria and performance levels receive the "NetWORKS As Advertised™"? award and Miercom Labs’ testimonial endorsement.
