Stanford researchers use AI to map racial covenants within Santa Clara County

Stanford researchers are using artificial intelligence (AI) to map and identify racial covenants — clauses written into property deeds that prevented non-White individuals from buying or occupying land — in partnership with Santa Clara County and researchers from Princeton University. 

This research comes as a response to a 2021 California bill mandating the redaction of racial covenants. Assembly Bill No. 1466, which was passed in September 2021, requires escrow companies and other associations to notify the county through a Restrictive Covenant Modification form if they come across a property that still subject to a racial covenant.

After the bill’s onset in July 2022, Louis Chiaramonte, the assistant clerk county recorder, said Santa Clara County chose to take a more proactive approach to the bill and began looking through the county’s historical records to begin identifying the racial covenants.

However, this task proved to be virtually impossible during the early months of this project, Chiaramonte said. With over 24 million documents accumulating in their records, going through them by hand was slow and ineffective. 

“It was literally eyes on paper on over hundreds of thousands of pages that our team was going through,” Chiaramonte said. “The books, the historical deed books, turning pages, reviewing every single word, trying to figure it out.”

The county then connected with the Regulation, Evaluation, and Governance Lab (RegLab) at Stanford. The lab introduced an AI language learning model that significantly increased the efficiency and speed of the work while maintaining reliability. 

Faiz Surani, a research fellow at RegLab involved in this project, said this task had two goals in mind: to “help them comply with their legal obligations in a…more efficient manner, using advances in machine learning” and use “something that can review these vast swaths of record for racial covenants to really illuminate the history and say something new, empirically, about the history of this county,” Surani said.

Surani said a 1948 Supreme Court ruling, Shelley v. Kraemer, ruled that racial covenants were unconstitutional and that enforcing these covenants would go against the Fourteenth Amendment. To this day, however, many racial covenants remain embedded in property deeds. 

The RegLab became involved with this project in early 2023. By March 2024, the lab had produced a list of 7,500 potential racial covenants between 1902 and 1980 found in the county’s records. 

“In many cases, these covenants didn’t just apply to you buying one house. They would apply to entire communities, entire developments,” Surani said. 

The research group discovered that in 1951, one in four houses was covered by a racial covenant.

Based on RegLab’s estimates, the open large language AI model “reduces manual efforts by 86,500 person-hours and costs less than 2% of the cost for a comparable off-the-shelf closed model,” saving the county both time and money.

By mapping the locations of these racial covenants, the research unveiled that the two largest racially restricted areas were in Palo Alto: the Stanford University Villa Tract and the Stanford University Park Development. 

“Palo Alto and especially the communities around this university were at the forefront in the worst way possible of this wave of racial segregation and the attempt to enforce it privately,” Surani said. 

The Daily has reached out to the University for comment.

This research project also dug up shocking cases of racial covenants and segregation. One instance occured with the Oak Hill Cemetery in San Jose, which sold 50 burial spots between 1925 and 1950 on the condition that they would solely be used for the “Caucasian dead,” Surani said, quoting the deed.

Of the 7,500 potential racial covenants identified, Chiaramonte said that the county has documented and recorded 4,500 of them and are currently working on confirming the rest. 

Since RegLab’s AI model only identified racial covenants present in deeds between 1902 and 1980, the county still needs to comb through documents from 1981 to the present and the handwritten documents before 1902. 

While Santa Clara County carefully reviews any AI use through their Generative AI Review Committee, the county’s Chief Operating Officer Great Hansen highlighted the role this project plays within the community. 

The county’s partnership with RegLab is a “great example of the circumstances under which AI can responsibly and productively be used to advance our work on behalf of the community,” Hansen said.

Surani and the rest of the research group hope to use the results to also analyze the modern impacts of these racial covenants.

“There have been a lot of really wonderful academic accounts of racial covenants, but I think none that have been able to say in true, precise empirical detail, exactly the impact that they had,” Surani said. “There’s a real opportunity in the gap there that we hope to be able to address.”

Hansen expressed optimism that AI can be applied for county-level work beyond Santa Clara.

“We hope this work can serve as a model for other counties and we are regularly identifying and pursuing opportunities to partner with Stanford and other universities to better serve our community,” Hansen said.

The post Stanford researchers use AI to map racial covenants within Santa Clara County appeared first on The Stanford Daily.