Marin presses AT&T over 911 disruption during storm

The third in a series of five stories examining the major tidal flooding this month in Marin.

In the aftermath of destructive flooding recently in Marin, officials are demanding answers about a widespread 911 disruption during the intense weather.

Events leading up to the five-hour outage began to unfold around 7:30 p.m. Jan. 4, when firefighters were called to an AT&T substation on Shaver Street in San Rafael for reports of flooding. They encountered floodwater and helped move standing water from a server room that housed critical equipment.

Within an hour, telephone lines, cellular communications, online and 911 services were disrupted across the county. AT&T provides 911 service in Marin.

During a typical 911 service outage, people can still make phone calls, but dispatchers can’t answer them.

“This was past that,” said Mike Marcucci, chief of the Marin County Fire Department. “This was, nobody could make phone calls.”

The department met with AT&T management on Monday to understand what led to the communications failure that night. They said they want to secure a commitment from the company that it will provide backup service to the system in the event of another outage.

“We remain deeply concerned about vulnerabilities in AT&T’s network and the need for meaningful system improvements,” Assistant County Executive Jason Weber said after the meeting.

“The county will continue to elevate these concerns until we are confident that AT&T fully understands the risks, appreciates the impact, and takes decisive action to eliminate single points of failure,” he said. “AT&T’s infrastructure must be managed with the same rigor as the public safety systems it supports, including 911 call-handling services.”

Weber said residents who tried to call 911 during the outage and could not reach a dispatch center should contact the county at 415-473-6717.

Marin County Supervisor Mary Sackett stressed the county’s interest in working with AT&T.

“The county of Marin recognizes the seriousness of this outage and the impact it had on public safety,” said Sackett, who is president of the Board of Supervisors. “We are taking steps to ensure AT&T fully understands the magnitude of this incident.”

AT&T spokesperson Kyle Loomis did not respond to a request for comment.

“The concerning issue is that we had a single point of failure and then we had a complete failure in the communications system, not only for emergency services, but day-to-day calls,” Marcucci said.

Emergency calls are generally directed to a law enforcement dispatch center first before being transferred to a fire agency if there is a fire or medical emergency.

The county fire department and other emergency response agencies have backup plans to move their communication lines in case of emergencies, Marcucci said.

“The problem was that none of those plans were going to work because the phone company had quit,” he said. “It’s all dependent on the phones still working.”

Marcucci said that while police and fire communications were affected across much of the county, Novato police’s dispatch center, which is not connected to AT&T’s San Rafael substation, remained operational.

Novato dispatchers took over and began fielding 911 calls on behalf of police and fire agencies across Marin. The city found itself in a unique position — the sheriff’s communications center aided Novato police last year after the city’s 911 system went down.

“We were happy to return the favor,” the police department said in a post on social media.

The county sent wireless emergency alerts that notified the public in Marin County about the communications outage. Residents were advised to go to a fire station if they could not connect with 911.

San Rafael police’s landlines, wireless service and intercoms also were impacted by the flood, police Lt. Scott Eberle said.

“Wireless service was unavailable for several hours, and while some cellular calls were able to connect to California Highway Patrol communications, many residents experienced an inability to place calls altogether,” he said.

San Rafael officers responded to a freeway crash after someone went directly to police to notify them about the incident, Eberle said. People also visited a Central Marin fire station in Greenbrae and were assisted by the on-duty crew with a medical emergency, Marcucci said.

The sheriff’s office communications system experienced intermittent service, sheriff’s Lt. Brennan Collins said.

“There was no indication that nobody needed our immediate assistance and they couldn’t get through to us,” he said.

In Fairfax, the community was impacted by the 911 service issues, Fairfax police Chief Rico Tabaranza said. He said the department’s 911 dispatch center also serves Ross and the College of Marin campus.

Fairfax has a backup plan in case its 911 phone system goes down. If the town and San Rafael police dispatch centers go down, then calls would be routed to the sheriff’s office. The sheriff’s staff would then route calls to Novato police if necessary.

“If all emergency lines across Marin County were to fail, we would request mutual aid from neighboring jurisdictions, including Sonoma, Alameda and San Francisco counties,” Tabaranza said.

In Stinson Beach, the fire district staff experienced communication failures that were mainly internet and cellular interruptions, Stinson Beach fire Chief Jesse Peri said.

“Even with those challenges, we stayed in steady contact with neighboring fire chiefs through our backup communications systems, which worked just as they were designed to,” he said.

Peri said that his staff is confident the county’s emergency operations management staff used wireless alerts to reach most households and gave clear direction to the public on what to do during the outage.

Service was restored at 1:30 a.m. Jan. 5.

Marcucci said that while the agency’s communications system was disrupted, first responders provided service on 22 calls from 8:30 p.m. to 1 a.m. Agencies communicated with each other via radio channels while on calls.

“We’re just happy that services were restored,” Marcucci said. “It was a testimony to the business we do every day that we were as successful as we were.”