Borrego Health Foundation is a private, non-profit healthcare provider across Southern California. Borrego Health offers patients at satellite clinics direct access to visiting specialists in such fields as dermatology, mental health, pediatrics, podiatry, prenatal care, reproductive health and women’s health. Borrego Community Health owns 24 clinics and will reach over 30 sites in 2017. Borrego Health is supporting their interconnected clinics based on Mushroom Networks’ Software-Defined Wide Area Network (SD-WAN) solution, to harmonize their network performance and reliability.

Streamlining the WAN at their clinics by migrating away from T1 to add performance and reliability

Borrego Health’s clinics connect to the servers at the co-located data center for various centralized private cloud services. The phone systems were all independent and were not centralized. In majority of the clinics, the aging copper infrastructure of rural telcos could not support high speeds and high availability – as it was common to have ISP failures a few times a month. They had tried Cradlepoint wireless routers for failover, however, the manual switch-over process was not a solution that worked. The remote locations didn’t have dedicated IT personnel, so making any changes to the network, as the ISP lines went down or came back up, meant a long trip to the location.

When Dave Baldwin, the CIO of Borrego Health, started planning the WAN upgrade, his team had various requirements and constraints: the solution needed to be cost effective, needed to provide fail-safe in real time and have the capability to enable seamless and highly redundant connectivity between their clinics and their EHR data center, as well as connectivity to their cloud-based services, such as their hosted PBX.

Additionally, Borrego Health’s accelerated growth meant that the same small IT team now had to manage the IT of an organization four times the size. This initiated a search for a solution that can address both voice and data in a single platform. Up until recently, both their carrier voice and data connectivity did not have redundancy.

Alongside the planned migration to hosted-PBX in their clinics, Borrego also heavily relies on a Citrix environment to access various application servers over the Internet, via Citrix Remote Access.

One clear motivation was the patient experience, which is tracked via a quality score by Medicare that affects the funding support. Downtime meant Borrego’s access to their Electronic Health System (EHS) was not available, and therefore the doctors couldn’t access patient records. This not only caused poorer patient experience, but since they had to revert back to paper based records, this meant increased potential mistakes that could put lives at risk – a risk that needed to be resolved.

Baldwin’s team partnered with James Munro, an experienced IT consultant, to evaluate the options and design the network solution that would meet all the requirements that the project presented. Mushroom Networks’ SD-WAN solution was picked to provide Broadband Bonding capabilities at the clinics and combine various private and broadband ISP connections from Centurylink, Skyriver and Telepacific. Truffle, the Broadband Bonding appliance from Mushroom Networks, manages the WAN connections at the locations by intelligently combining the Internet resources into a faster and more reliable connection, whereby automated failover is accomplished to optimize application uptime.

The Software-Defined nature of the Mushroom Networks solution also enabled hosted VoIP services to be provided without being interrupted by WAN problems, as Truffle routes VoIP packets around network problems without losing the session. This meant VOIP calls were uninterrupted even during times when some of the WAN links were having performance issues. “Mushroom’s VOIP Armor technology is a true enabler for hosted VOIP services, as it takes the problems related to cloud connectivity out of the equation”, said Munro, and added, “Being able to optimize both real-time and non-real-time applications in a simple manner was one of the key factors in us choosing Mushroom Networks solutions for our WAN upgrade project”.

The Truffle devices at the clinics create a Broadband Bonding tunnel that creates an overlay tunnel to the Truffle at the data center. This enables high-performance connectivity and sustained application level performance. Broadband Bonding also meant the secondary and tertiary ISP links were not kept idle but could be utilized for the added bandwidth they provided. This also lowered the bandwidth cost for the clinics. The design also was engineered to avoid single points of ISP failure.

“Electronic Health Records is a fundamental cloud application for our clinics. Our EHR services can’t go down. It literally can be a matter of life and death. That is one of the primary reasons that attracted us to Mushroom Networks”, said Baldwin.

Truffle devices create bonded tunnels between the devices, either in a mesh or hub-and-spoke star topology to create tunnels with various paths between locations. The various WAN resources can be utilized for Internet access and cloud applications to optimize application performance – which can translate as higher throughput, lower latency, lower jitter and/or avoiding packet loss, depending on the application going through the bonded tunnel.

Using Truffle, the Software-Defined WAN Orchestration devices capable of Broadband Bonding to aggregate 2 or more broadband lines, provided Borrego Community Health with the following:

Boost in WAN Performance:

Truffle appliances were used to bond two or three WAN circuits per site from carriers such as Centurylink, Skyriver and Telepacific simultaneously. Borrego increased capacity 400%. The overall WAN metrics improved both for mission-critical applications such as Citrix, hosted VOIP, EHS, and Telehealth as well as best-effort traffic. As an example, the VPN connecting branch offices to the HQ office now has the full bonded speed of the broadband lines, as opposed to an individual broadband line speed. Bonded circuits per site range from 23 Mbps to 400 Mbps of aggregate bandwidth.

Boost in WAN Reliability:

Truffle provided self-healing WAN and dynamic failover capabilities for Borrego’s network. As a result, on-going Internet sessions are now shielded from any WAN outages or performance fluctuations, as long as at least one of the WAN connections stay up.

Reduced WAN Cost:

Compared to a single MPLS-based WAN network, the cost savings achieved by Borrego were very significant, reaching up to 70% cost savings, due to combining MPLS, Internet and Ethernet networks into one network seamlessly, reducing circuit and downtime costs and greatly simplifying management.

Reduced Emergency Calls & Related Cost:

Implementing Truffle significantly reduced emergency calls for IT problem tickets as most WAN-related outages and emergencies are now shielded from the users and can now be addressed after hours in scheduled maintenance windows.

Specialized SDN tunnels:

Truffle’s SD-WAN Orchestration meant specialized tunnels could be fitted to Borrego’s specific application requirements. One example was the use of Mushroom’s VoIP Armor. Borrego has a Cloud PBX at its 1 Wilshire data center in LA serving 1000 users. The WAN connects the PBX to all users at all the clinics. VoIP Armor tunnels shield the VoIP/SIP sessions between users and the PBX against any network fluctuations or outages, delivering high MOS scores and toll-quality calls.

Borrego Health is a private, non-profit healthcare provider operating in Southern California. Borrego Health offers patients at satellite clinics direct access to visiting specialists in such fields as dermatology, mental health, pediatrics, podiatry, prenatal care, reproductive health and women’s health. Borrego Health owns 24 clinics and will reach over 30 clinics in 2017. Borrego Health decided to support their interconnected clinics based on Mushroom Networks’ Software-Defined WAN (SD-WAN) solution, to boost their network performance and reliability. Borrego Health searched for alternative solutions as they migrated from their single carrier network. Mushroom Networks’ Truffle appliances in their clinics translated into network performance and reliability, providing the network uptime and performance needed to support their services and applications such as VoIP, EHS, Citrix and others. All this was accomplished with up to 70% cost savings over legacy MPLS.