What is REALLY behind an E911 'Glitch'?

This Blog is also available as an MP3 Audio Podcast.
If your device does not support Flash you can DOWNLOAD THE MP3 HERE

From time to time, people report ‘glitches’ in the E911 system that cause a call to be miss-routed, and ultimately some less than desirable results. One of the more famous incidents was the Washington Post article that read, “Man Found Dead in Office 10 Hours After 911 Phone Glitch Confuses Rescuers”. But was it really a phone ‘Glitch’, or was there something else to blame?

What exactly is a ‘glitch’?
It appears that there is general consensus that the term ‘glitch’ is “A minor malfunction, mishap, or technical problem”, or a “A false or spurious electronic signal caused by a brief, unwanted surge of electric power” .

Astronaut John Glenn is often credited with having the first recorded appearance of the word in 1962 where he wrote “. . . . Another term we adopted to describe some of our problems was ‘glitch'” referring to the various technical problems experienced early on in the space program.

This past month in Muskegon County Oklahoma, there were reports of glitches affecting 911 calls in the area where callers were miss routed to 911 centers hundreds of miles away. The Muskogee Phoenix reported on January 28, 2013 that problems in the E 911 network had surfaced where some calls were going to incorrect towns. Allegedly, and incident took place where a store with IP phones had called 911, however their calls were miss routed costing precious minutes in response time. As in many of these stories, the details surrounding the incident were sketchy and unclear. Was it a “over-the-top” voice over IP telephone service such as Vonage? Or, was it an IP telephone connected back to a corporate PBX in another area? In either case, the blame for this incident was placed on the technology. Unfortunately, most likely it was not the technology that was at fault. Most likely, the culprit was the provisioning of that technology, and the lack of testing.

As corporations flatten consolidate and extend (FCE) there network topologies, processing 911 calls from remote locations can become problematic, especially if those remote locations reside in a remote 911 jurisdiction or service area. Today’s legacy 911 networks are very geographic in nature. Typically they service a very specific area such as a County or consortium of counties. In many cases, the geographic 911 service area boundaries follow the old LATA boundaries established by the telephone company in the 60s and 70s. As states upgrade their E 911 backbone networks, often 911 routing can be statewide.

One of the problems for enterprise users, is that there is no single database source or online reference where one can determine what their 911 service area is. Even if one existed, with the constant flux of network grooming, enterprise administrators don’t have the resources to dedicate keeping up with that changing data.

To solve this problem, Voice over IP Positioning Carriers, or VPC’s, have emerged to provide a more universal “umbrella coverage” in the form of a nationwide 911 network. Although these services do provide a solution for FCE environments, they are not by any means a “set it and forget it” solution. This is where understanding how a specific service works is critical for an IT administrator implementing a hosted E 911 solution.

With most VPC providers, calls are routed from your PBX via SIP trunking, or via a 10 digit number, also known as a PSTN push. Routing at the VPC is done by examining the caller ID, and comparing that to pre-provisioned routing tables in the VPC database. As a failsafe in the network, most VPC services will offer some form of manual emergency call routing service should the number fail translations. “That happens when the provider hasn’t programmed the location in when the phone system is set up,” said Darryl Maggard, Supervisor for Muskogee County E911.

Apparently, this is what happened to the caller in Haskell. The national call center, whose operators are charged with directing calls to the proper E911 center, redirected the call to Muscogee County in Georgia and not Oklahoma.

“That could have been bad,” Maggard said. “And it’s a problem anyone with Voice Over IP service could have and not know about.” He also said customers with Voice Over IP service should call their carrier or their vendor and have them check to ensure emergency calls are routed correctly “Just to be safe”.

Nicholas Maier, Senior Vice President at RedSky Technologies stated, “The incident in question is one where a VoIP service provider has provisioned a customer in their system and has not properly provisioned the location address of the end user. Standard operating procedure when using a VPC is that the location address of the end user is submitted and validated as an MSAG valid address. Once the address is MSAG validated, the call will be routed to the correct PSAP.”

So the ‘glitch’, in this case, was simply someone not following procedure and making sure database information was correct. Meyer added, “There are hundreds of small VoIP service providers in the USA. Not all of them follow best practices”.

This is where great deal of your 911 solution is wrapped around the procedural side of the effort that ensures not only is proper data provided to the network, but the data is checked to make sure it is valid and correct. For example, the address of Avaya in Basking Ridge is 211 Mount Airy Rd. That is a valid address, it is validated in the Master Street Address Guide (MSAG) database, but if I happen to be using that address when I’m on a phone located 47 miles away in my hometown of Ringwood New Jersey, it is completely incorrect and irrelevant to 911.

Karina Yandell, Vice President of Business Development at 911 ETC, Inc. stated “Seconds and minutes matter when a 911 call is placed. We trust that 911 will bring immediate help, but problems exist in our nation’s emergency response system that can cause serious delay in a life or death situation. Your voice provider may or may not have taken the steps necessary to provision location information for outgoing 911 calls. Your employer may or may not have ensured that a 911 call placed from behind the PBX won’t send responders on a wild goose chase.”

As more and more enterprises look at 911 provisioning in their networks, it is not always the technology that is the most critical piece. Proper planning, implementation, and regularly scheduled testing are all key components for an effective 911 solution.

“Tragedies have occurred due to these glitches so it’s imperative for organizations to be proactive about this. The solution is often so simple and affordable that there truly is no excuse for a business or Vo IP provider to not take preventative measures.” Yandell added.

Just like any problem, the first place to start in solving your E911 remediation is a definition of the problem. Immediately follow that with a design session that involves specific use cases around your employees. Once this is established you can easily categorize your issues, define a remediation plan for each scenario or type of user, and establish an implementation schedule. Each group of users in your network have specific needs for their E 911 reporting. When you define what the solution looks like, don’t be afraid to deploy different solutions for different classes of users. E 911 is not difficult, but it requires proper planning. Many times that planning is very unique for your exact environment. Don’t be afraid to hire a consultant, but if you do, be sure to check the references and credentials. It could save your life.

Want more on E9-1-1?  E9-1-1 Talk Podcast
Subscribe to my weekly E9-1-1 Talk Podcast here

Thanks for stopping by and reading the Avaya CONNECTED Blog on E9-1-1, I value your opinions, so please feel free to comment below or if you prefer, you can email me privately.

Public comments, suggestions, corrections and loose change is all graciously accepted 😉
Until next week. . . dial carefully.

Be sure to follow me on Twitter @Fletch911


Related Articles:

Four Reasons to Focus on Mobility (and Three Ways to Get Started)

Mobility is a no-brainer in today’s smart, digital world … or is it? As recently as 2013, only 22% of leaders had made a strong business case for bring your own device (BYOD). We’ve made some inroads since (as of last year, 59% allowed employees to use personal devices for work), but still have a ways to go.

Mobility is an absolute must. But don’t just take our word for it. Here are four proven benefits:

  1. You’ll make a lot more money
    Research from the IES and The UK Institute for Work Studies found that increasing engagement investments by just 10% can increase profits by up to $2,400 per employee, per year. Consider a large enterprise with 500 employees—investments in mobility could lead to as much as $1.2 million in additional revenue for the company per year.
  2. You’ll drive employee wellness/satisfaction
    One survey shows that 97% of companies agree that employee well-being positively influences employee engagement. And wouldn’t you know, mobility helps ease areas of wellness with the greatest impact on engagement: work/personal stress, physical health, personal relationships and work productivity. Allow more flexibility with mobile team collaboration and you’ll see major improvements.
  3. You’ll get better results, period
    Mobility improves engagement, and engaged teams drive outstanding results. In fact, research shows that companies with better engaged teams outperform others by over 200%. Mobility also now seems to be at the heart of corporate culture (this Forbes article excellently argues why), and 95% of executives believe culture is important in driving business outcomes.
  4. You’ll narrow your talent gap
    You need the right people with the right skills to differentiate and compete. An average 34% of global organizations have difficulty filling jobs due to a lack of available talent, with this number significantly higher in regions like Asia-Pacific (45%), India (67%) and Brazil (42%). The right employees can be seamlessly on-boarded—regardless of location—with the ability to engage anytime, anywhere via advanced mobility.

Get Started

Ready to improve mobile team collaboration? Consider these ways to take charge:

  • Get with the softphone revolution. Softphones are now widely deployed in the enterprise and advanced versions go well beyond the phone to provide calendar integration, instant messaging, contacts and call logs, as well as one touch access to meetings and collaboration capabilities.
  • Embed real-time communications (i.e., voice, video, chat) directly into the applications employees use each day to work. Eliminate the need for them to hop in and out of multiple disparate apps.
  • Flexibly integrate applications using an open communications platform. These apps can be ready-made from a trusted provider or existing within the Avaya Snapp Store. Even better, challenge employees to build custom apps that can further improve team collaboration.
  • Invest in a desk phone solution that fuses mobile simplicity with advanced enterprise software tools. Personalize user experience and eliminate the need for employees to use multiple mobile devices to engage.

Avaya can help you successfully take these next steps, as well as create an effective mobile strategy for your business. Read the IDC Technology Spotlight and get started today.

Avaya Named a Leader in Gartner’s Magic Quadrant for Contact Center Infrastructure

Avaya is honored to be recognized as a leader in Gartner’s Magic Quadrant for Contact Center Infrastructure worldwide. Avaya has been the only vendor having the distinction of being named a Leader for 16 consecutive years. Each year the research organization creates a market view of key players for business users, reflecting business goals, needs, and priorities.

Contact centers have gone beyond phone calls with customers now expecting to communicate on their terms via text, IM, email, chat or video. For the past 16 years Avaya has created seamless and highly personalized experiences, building brand loyalty for companies all around the world.

According to Deloitte, 85% of organizations view customer experience provided through contact centers as a competitive differentiator. Todays companies must remain relevant by creating a single interface to connect customers with the correct resource each time, supporting their preferences. Supervisors and managers need real-time performance information to adapt immediately to situations to ensure optimized customer experience.

Avaya has focused its efforts on creating next-generation contact center solutions, creating communication strategies enabling a continuous transition between channels during customer interactions.

Please visit Gartner’s page to read the full report and see how Avaya’s Contact Center infrastructure continues to deliver best-of-breed Contact Center applications. We look forward to continuing innovation and leading business communications for the digital age.


Seeing into the Office of the Future

Dubai is heavily focused on delivering on its Smart City goals, with the goal of being among the smartest—and happiest—cities in the world. The drive toward smart cities is part of a wider shift, with countries around the globe seeing a migration from rural areas to urban. With more than half the world’s population now living in cities, organizations in the Middle East are facing increasingly difficult decisions about how they allocate resources and manage their workforce.

For a city like Dubai, that can be challenging. Finding the right real-estate location for office space, managing energy usage and providing physical workspaces for employees working different shifts in a modern, 24X7 city creates its fair share of headaches. Enterprises also have to cope with an increasing Bring-Your-Own-Device (BYOD) culture, with smartphone and device penetration especially high in the United Arab Emirates and other Gulf countries.

We have reached an inflection point where the number of devices connected and generating data is accelerating at an exponential level. Our work environments are beginning to blur, as workspaces are no longer physical but virtual. Organizations need to deliver a consistent work experience whether in an office or on the road, or at home. The key driver is to intuitively connect not only communications devices, but interact with the ambient technologies that surround us, like lighting, BMS and signage.

At Avaya, we believe that the Smart Enterprise is one of the key building blocks for smart cities—and one of the foundations of true Smart Enterprise development will be what we term the Office of the Future. This is about more than just technology; the Office of the Future involves automating work processes within the enterprise to deliver a more intuitive employee experience. As with any “smart” solution, the Office of the Future is only smart if it resolves the challenges it aims to address.

So what do we mean by Office of the Future? Imagine walking into a new office you’ve never visited before and your work station is ready before you sit down, configured to your preferences, right down to the air conditioning settings and digital signage displayed with your identity. When a client arrives to meet you at the office, they can be sent to a meeting room automatically, and you can talk to them on their mobile as you walk to greet them.

With Avaya solutions, the act of an employee booking a workspace would kick start a workflow that immediately sets up that space with all communication devices enabled and connected securely, while interacting with the building management system to ensure the environment was set to the employee’s requirements, everything from lighting to temperature to digital signage. The employee could be identified by their phone or a card ID. When they leave, the workspace can be reset for the next employee that reserves it.

Such Smart workspaces will help enterprises manage their resources more effectively, better leverage their real estate investments, and improve employee well-being and productivity. Here in Dubai, the Office of the Future starts now.