What was Behind the Massive E911 Outages in the Pacific Northwest?
As we move towards IP-enabled emergency services networks in the United States, we supposedly leave behind an archaic, TDM-based, hardwired point-to-point infrastructure with a new, resilient architecture that is easily rerouted around a failed segment.
But exactly what happened in the Pacific Northwest when a supposedly planned maintenance event took out communications across a two-state area?
The immediate reaction by many was disbelief. After all, haven’t we learned our lesson in diversity and single points of failure with the massive outages in the Northeast from hurricane Sandy?
Nonetheless, the incident did happen, and for nearly 6 hours most — if not all — of 911 was off the air, stranding a reported 4,500 callers who needed assistance, according to a report from KIRO-TV.
As a professional in the public safety communications business, what I find slightly insulting are statements that make it into the press such as: “caused by an isolated system issue that was promptly resolved after root cause identification.”
While the term “promptly resolved” may be reassuring to the general public, the qualifier of, “after root cause identification” is the more troubling piece of information.
I can only imagine my daughter pulling into the driveway telling me that she had a flat tire, but she “promptly changed it, as soon as she realized it was flat” only then to realize that it took her six hours to realize she had a flat tire, hence the smoking, mangled wheel in the trunk of the car.
One of the areas of caution for public safety communications officials is going to be the monitoring of systems, and their interaction with the overall network. This is a significant point that needs to be taken under serious consideration when building and designing next-generation emergency services networks.
Public safety has always operated behind a closed curtain, taking the stance that important life-safety work was happening, therefore they should not be hampered by rules and policies that could affect their important mission.
But one area that has to be acknowledged is that commercial networks have come of age and have learned from previous events that include natural disasters, DDoS attacks, and direct attacks by some of the best hackers on the planet. The very resiliency and reliability that they demand to perform their mission critical life safety function is the identical resiliency and reliability that the world’s largest financial institutions demand to protect their data, as well as their customers.
When we sit back and look at the functionality that “Next Generation 911” will bring into the public safety workspace, it is the world-class multi-modal, multi-channel functionality we have been delivering to commercial entities for decades.
Communications is an app on the network. Inbound and outbound communications events are easily coordinated and managed based on specific criteria. The age-old adage of routing a caller to a resource with close proximity to them only makes sense when there isn’t a large scale massive event taking place.
At the recent FCC workshop discussing the transition of 911, NENA’s Trey Forgety brought up the very relevant point that during a massive emergency event, the best place for the Public Safety Answering Point (PSAP) may be in the back of a hardened vehicle that is driving away from the emergency to an area where they will be safe and can establish a new fixed operational base.
While that’s an excellent plan, it requires a whole new method of thinking and engineering.
I often remind the engineers I work with that an Internet Protocol (IP) session does not mean we’re putting information on the “Internet”. It means that were using the Internet protocol — a network of network — to route information from node to node.
Under normal sunny-day conditions, that IP-based network may very well remain within the walled garden of an agency’s network. But when an emergent condition exists at the time of a disaster, we need to be able to dynamically open gateways to specific paths within the garden that are accessible from the outside. Of course in doing so, the highest level of security needs to be considered and implemented.
I believe that what has yet to be developed, or has not been developed enough, is the technical acumen of networking technology by public safety specialists. As I sit back and think, I can almost count on a single hand the number of people that I would feel comfortable with designing modern IP-based critical communications environments.
When I’m asked what value Avaya brings to the public safety table, I talk about our decades of resilient, reliable, redundant communications, and keeping the proverbial “lights on” in some of the most dire events of the past few decades. As examples, I again look to our large financial and commercial contact center customers who have worked their way through 9/11, massive blackouts across the Northeast, and as the newly coined Derecho and Blizzards on the planet.
Be sure to join me on April 22 for a free webinar sponsored by our good friends at Smart 911 and Rave Mobile Safety. I’ll be joined by Todd Piatt from Rave, and Bill Schrier (@BillSchrier) who is with the State of Washington office of the CTO, where will be discussing NG9-1-1 / First Net in the changing role of 911.
I will also be kicking off the 2014 series of the Public Safety Best Practices Forum at the Embassy Suites Hotel at the Seattle Tacoma International Airport in Washington on Thursday, April 24, where I will be joined by Patrick Botz from Voice Print International, Inc., and other industry colleagues. If you are in the Seattle-Tacoma area and want more information you can go to http://psapforum.com/Seattle.
Want more technology, news and information from Avaya? Be sure to check out the Avaya Podcast Network landing page at http://avaya.com/APN. There, you will find additional podcasts from industry events, such as Avaya Evolutions and INTEROP, as well as other informative series by the APN staff.
Thanks for stopping by and reading the Avaya Connected blog on E911. I value your opinions, so please feel free to comment below or, if you prefer, you can email me privately.
Public comments, suggestions, corrections and loose change is all graciously accepted 😉 Until next week. . . dial carefully.
Be sure to follow me on Twitter at @Fletch911
E911’s Fatal Flaw is Lack of Location Data—How Avaya Breeze Can Solve
The night of her husband’s death, Alison Vroome did everything she knew to be right. She grabbed her phone, called 911 and told the operator her address. Then she repeated her address a second, third and fourth time.
The call went to a different North Carolina county; the operator couldn’t understand her address. It was more than 10 minutes into the 911 call before paramedics arrived. Like anyone calling 911 in an emergency, Vroome expected her call to go quickly and smoothly, but it didn’t. Vroome’s call was one of 5.7 million 911 calls that come from wireless phones in NC—about 74% of all 911 calls in the state according to data from 2015. Yet 911 call centers rely on the cellular carrier to provide a cell phone’s location data. The legacy 911 network is voice only and cannot pass any data from the device. Instead, they can only receive the location data from the tower pinged by the call, something not nearly as accurate.
No one can say for certain if Vroome’s husband would be alive today had paramedics arrived sooner, but there isn’t any doubt that the current technology used in E911 emergency situations fails citizens. And this isn’t an issue isolated to the U.S. With the rise of mobile devices, countries and communities around the globe face the same technological flaw—the lack of location information.
As Avaya’s Jean Turgeon addressed in his recent blog on the current state of public safety and E911, accurate location information is one of, if not the most important piece of information that an emergency responder needs; and resolving this fatal flaw requires proactive urgency.
How Today’s #Tech Can Address E911’s Fatal Flaw
My Avaya colleague Mark Fletcher, ENP, recently wrote that when it comes to significantly improving public safety and E911 response times, tech is king. He’s right.
Case in point: In Europe, the introduction of EU eCall to become an integral element of the European emergency number 112 is solving the GPS precision challenge for new passenger vehicles sold in the EU after 2018. In an emergency, an eCall will relay a vehicle’s exact location, time of the incident, and direction of travel to emergency personnel, as sourced from the device, and very accurate. This is done automatically by the vehicle or can be triggered manually by the driver by pushing a button inside the car. That’s technology in action! While we have about two years to go before it becomes available large scale, we’re heading in the right direction.
In addition to eCall, there’s another remarkable solution called Advanced Mobile Location (AML). When a person in distress calls emergency services with a smartphone where AML is enabled, the phone automatically activates its location service to establish its position and then sends this info to emergency services via an SMS. The current downside to this is that AML is only compatible with Android mobile devices (R3.4 or greater). But still … it’s a huge step forward, and sets an excellent example for others.
The concept of AML was developed in the UK by BT’s John Medland in partnership with mobile service provider EE and handset manufacturer HTC initially. First tests were so promising that the European Emergency Number Association (EENA) began to promote AML, which sparked the interest of Google, ultimately getting AML introduced into Android natively. Talk about a ripple effect!
As the world’s leading software and services company, Avaya understands there are better ways to deliver public safety and emergency services, and we’ve been innovating these same capabilities in many commercial arenas for years. Our efforts there have set off their own ripple effect across the public safety industry, urging government agencies around the globe to harness the power of technology to enhance public safety services for citizens. What’s more, our teams are leveraging the Avaya Breeze™ Platform to intelligently link the location data to the incoming eCall or AML call and make it available to the E911 responder. Recently, in partnership with Engelbart Software and Oecon, we’ve developed a flexible and scalable solution for this type of enhanced emergency calling scenario and the results have been positive.
In fact, eCall is looking more and more like a potential game changer, and here’s why.
Let’s look at the technology side of the overall process:
- A car is involved in an accident.
- Sensors in the car trigger a sequence of events performed by the In-Vehicle System (IVS).
- The SIM card registers to the strongest mobile network to raise the emergency call to the EU E112.
- A modem kicks in, coding the GPS data and other car-related information as audio tones into the voice channel.
- Immediately following the data transmission, the IVS switches to the hands-free communications system allowing the people in the car to communicate with the E112 responder.
What does this mean for the emergency responder?
- The E112 responder picks up a call from a mobile device, immediately receiving precise location information. That’s new!
- The E112 responder can be sure that it’s a serious situation because the airbags have been deployed, which triggers the emergency call sequence to start. So no one is left to wonder the seriousness of the call.
- Most likely there’s no one for the E112 responder to speak with in the car. Why? Because this is an automatic call, not a call voluntarily initiated by a real person. And while the modem is beeping its data to the Public Safety Answering Point, the passengers might already have stepped out of the car and can’t hear the E112 responder’s “Are you OK?” Or they simply can’t respond because they’re unable due to the severity of the accident.
So are we still talking about a normal emergency call? From my point of view, this is the Internet of Things (IoT) plunging right into public safety and emergency services: sensors, data, processes and integrations. IoT under the disguise of a voice call … this IS a game changer!
At Avaya, we leverage our Breeze workflow engine to tie together voice calls and the IoT. Even though eCall is an initiative in the European Union, we see the concept of telematic calls being discussed around the globe, in public safety as well as in private businesses like the automotive industry. And, yes, we strongly believe that this approach of integration building on Avaya Breeze can also work to help overcome E911’s same fatal flaw, location.
I’ve delivered a series of Avaya Breeze webinars with my colleague, Andrew Maher, featuring Engelbart Software developers. Together, we demonstrate how to deal with eCall and AML. Have a look to learn more about the capabilities of Breeze and its impact on public safety. The demo starts at 00:19:30.
When Is Enough Actually Enough? Exploring the Lagging Face of Public Safety (Part 2)
In Part 1 of this series, Avaya Vice President and Chief Technologist for software-defined architecture Jean Turgeon opened up a much-needed conversation about the current state of public safety and E911 (which, for the record, doesn’t look good). Just consider that a 2014 study of 1,000 public safety answering points (PSAPs) found that only 18.7% are confident in the location data they receive from wireless callers.
It’s no surprise that technology is vital for improving public safety. The way I see it, this is like a three-legged stool. We need:
- Originating devices to support location accuracy
- 911 call center networks capable of receiving the information
- A Public Safety Emergency Services IP Network to connect them
PSAPs must ensure all three legs are sturdy and of equal length, otherwise fundamental capabilities will be severely limited or missing altogether.
Let’s take a look at the networking side of public safety for a moment. Today in the U.S., there are life-threatening complexities associated with dialing 911 for no other reason than the restrictive legacy networks that transport these calls.
That’s a terrifying thought.
Many times the system programming in hotels and office buildings has similar restrictions. This is why I fight tirelessly in support of Kari’s Law, a U.S. Senate bill introduced earlier this year designed to improve 911 services for multiline phone systems. The law is named in honor of Kari Hunt, who was killed by her estranged husband in late 2013 at a motel in Northeast Texas. One of Hunt’s children tried repeatedly to dial 911 from the motel room’s phone, but wasn’t able to get through because the motel required people to dial 9 to get an outside line. This is a fact I continue to repeat, as I still find people who have not heard of this tragedy, or gave it a second thought.
At the same time, the majority of the emergency call centers today have a serious problem with grade of service. It’s something that’s often in the news, constantly talked about, but rarely acted upon. Our public safety networks are something rarely thought about. Consider the fact that there are somewhere close to 6,000 911 call centers across the U.S. today. Given this, what do you think is the average number of positions staffed in those centers? You likely think dozens, and maybe even hundreds. In actuality, that number is a sparse four people.
So, what happens when all four employees at the average 911 center are tied up because 20 people are calling about the same car accident? Those calls will likely overflow to a neighboring town or city, which then also immediately becomes tied up. This cascading effect starts to immediately make sense how quickly several local governments can be taken out of service. This becomes a serious issue when a person is having a heart attack and dials 911 only to get a busy signal or to be put through to a city 10-20 miles away. A more nefarious problem is how easily it would be to disrupt the U.S. 911 network via Telephony Denial of Service (TDoS) attacks, something the FBI and Public Safety worry about daily.
Overcoming Today’s Greatest 911 Challenges
In Part 1 of this series, JT mentioned a few reasons why PSAPs may overlook infrastructure upgrades. In my opinion, there’s only one primary reason: it’s cost-prohibitive. Why? Because at one point, a handful of businesses in the industry decided they wanted to capitalize on the market by creating very specialized and expensive equipment. Because so few people understand 911, these cost-prohibitive solutions (which run on old technology with massive limitations) are widely believed to be the only options available in the market today.
It has never been more evident that almost every 911 center is currently grappling with technological, financial and operational challenges that seem difficult to overcome. As FCC Chairman Tom Wheeler said July 12 in a congressional testimony: “Unless we find a way to help the nation’s [911 centers] overcome the funding, planning and operational challenges they face as commercial communications networks evolve, NG911 will remain beyond reach for much of the nation. Let me be clear on this point: 911 service quality will not stay where it is today, it will degrade if we don’t invest in NG911.”
But remember the three-legged stool, and the originating network, or the enterprise customer. For example, we recently worked with a large customer based in New England that boasted more than 25,000 network endpoints across 700 locations. This included everything from small two-person offices to regional medical centers all the way to large teaching hospitals and universities. The 911 solution this customer was originally going to deploy was estimated at $650,000 in CAPEX, in addition to a monthly recurring operational cost of about $25,000.
Thankfully, this organization came to Avaya before signing the contract and asked if we could assess the situation. After consulting with them, and examining their workflows, we engineered a new operational model that only cost $130,000 in CAPEX, and would be less than $1,500 a month in recurring operational costs. With Avaya functionalities along with technologies delivered by our trusted Select DevConnect Partner Conveyant Systems, Inc., we were able to hand this customer a half a million dollars back in CAPEX, and decreased their OPEX by $282,000 annually. The result of building an efficient 911 solution was the organization now being able to allocate hard-earned dollars towards other top-priority initiatives that had previously gone unfunded. That’s the beauty of it all.
The lesson learned and the key to easily and cost-effectively upgrading your 911 infrastructure is to not accept the status quo, and partner with the right provider for your needs. At Avaya, we know there’s a better way to deliver 911. We take pride in our commitment to driving awareness around this need. It gives us great honor to be advocates for those whose voices must be heard or whose voices have been silenced, like Kari Hunt. We’re dedicated to teaching organizations and our customers that there is in fact a way to seamlessly overcome today’s greatest 911 challenges. We hope that you’ll join us in this very important mission.
When is Enough Actually Enough? A Hard Look at the Lagging Face of Public Safety (Part 1)
When we talk about the state of public safety today, we unfortunately have to recognize the devastating tragedies that have forever affected our communities, schools and businesses worldwide. Research shows that we’re currently experiencing four times as many terrorist attacks globally than in 1990. This month alone, there have been 120 confirmed or suspected attacks—an increase from around 95 in January.
People are being targeted based on their religious beliefs, ideologies and even identities. In France, for instance, we’re seeing new laws that ban certain cultural garbs for fear of terrorist-related threats. Meanwhile, in the U.S., we’re seeing a divide between law enforcement and the very citizens that officers have sworn to serve and protect. In the Middle East, we continue to see unthinkable devastation as violence escalates daily. I understand these aren’t things we want to talk or hear about, but it’s important that we do in order to improve communication infrastructure and transform the global state of public safety and emergency response.
To this end, we’re seeing technology rapidly evolving to a point where there are next-generation solutions available that can help get us to where we need to be. For example, consider the all-new, reopened Sandy Hook Elementary School. On Dec. 14, 2012, the Newtown, CT-based grade school suffered the deadliest mass school shooting in U.S. history. Last month, however, the school reopened its doors equipped with extraordinary technology that ensures next-generation protection for children and staff this school year.
The new design boasts advanced security features that are hidden in plain sight, improving natural surveillance of the grounds. The technology also offers increased situational awareness through a series of impact-resistant windows. Overall, the hope is that the rebuilt school will be the first within the state of Connecticut to be compliant with a new state school safety code, the School Safety Infrastructure Council guidelines.
The redesigned Sandy Hook Elementary School proves that technology can reimagine the possibilities of public safety, if only we allow it to. Examples like this make it really difficult for me to accept that our current state of public safety lags so much. At Avaya, we’re doing all we can to actively bridge this gap. One massive inadequacy we’re especially passionate about improving is the accuracy of E911, or Enhanced 911.
E911 was designed to allow emergency responders to determine the location of a caller based on the caller ID. Today, however, devices have become nomadic and the phone number to location correlation is no longer a valid assumption. Fortunately, there are alternative solutions available that can detect the exact location of a device, an IoT object, or an individual by leveraging smart devices, wearable technologies, and more.
This combination of advanced technology (i.e., Wi-Fi triangulations, GPS, wearables with NFC capabilities) is a key to overcoming 911’s greatest flaw: lack of location data. These advances in technology make it possible, for example, to detect a child that has left a secure area and then immediately send an alert to emergency response teams. These different mechanisms make it possible to save lives. Imagine if someone was suffering a heart attack in an office complex. In this case, standard 911 will enable first responders to locate the building the person is in, but how do they know if the person is on the fifth floor, the 40th floor or in the basement? This same scenario applies to any suspected or proven terrorist.
All of this sounds great, but there’s one problem: for many, deploying these technologies isn’t top of mind. Just consider findings from a 2015 national investigation conducted by USA Today. After sorting through hundreds of pages of local, state and federal documents, it was discovered that:
- The average chance of 911 getting a quick fix on location ranges from as low as 10% to as high as 95%.
- In California, 63% of cell phone calls to 911 didn’t share location in 2014.
- In Texas, two-thirds of cell phone calls reached 911 without an instant fix on location during 2010 to 2013.
No two ways about it: the reason why so many emergency calls today reach 911 without an accurate location is because there’s a severe technology issue at play. Public safety access points (PSAPs) still rely on technology that was designed to locate landlines, despite the fact that the number of 911 calls that come from cell phone networks is 70% to 80% and growing.
Users are evolving from land lines to wireless technologies, but PSAPs continue to remain behind, locked into technology designed in the 1960s. Despite technology being readily available, it isn’t being implemented. Why does this travesty exist? The reason for this is simple: because providers choose not to. Because it’s too costly. Because it’s too much of a hassle or inconvenience. Meanwhile, the reason for implementation is and always will be more important: because lives hang in the balance when archaic infrastructure remains in place.
The bottom line is this: there needs to be a greater movement towards next-generation methodologies of tracking one’s location. PSAPs need to effectively keep up with today’s pace of innovation in order to better serve the general public. It’s great to have a caller’s general location, but responders need richer and more relevant caller information to elevate public safety to where it needs to be today. We need to create proactive urgency around this issue—otherwise, we’re going to keep suffering preventable tragedies until someone finally decides that enough is enough.
Coming up: In Part II of this series, Avaya’s Chief Architect for Worldwide Public Safety Solutions Mark Fletcher will dig into specific technology deficiencies and how to overcome them by easily and cost-effectively upgrading your 911 infrastructure.