Who is your grandma and why should AI Care?

By Davar Ardalan and Robert Malesky
Many of us are familiar with our cultural heritage, but many of us are not. Walk down any street and look at the passersby. What is their ancestry? Nigerian, South African, Swedish? Does it matter whether they know or not? Or if they are aware of their own cultural history? How do they imagine themselves in the dominant culture? Is that even important? 

​Yes, it is important. Understanding one’s own culture and heritage not only develops our sense of pride, but allows one to interact with people of other cultures as an equal, and evolve a deeper, truer understanding of our world and the peoples who inhabit it.

Culture has traditionally been handed down through stories told by elders. In today’s interconnected world, that kind of storytelling seems to be losing its appeal to younger generations who are more interested in flashier, higher-tech outlets. But it is those same flashy, high-tech advancements that could help solve the problem.
Artificial Intelligence and Machine Learning, if designed to be deeply inclusive and historically accurate, can usher in a new era of excitement and interest in society’s past, and by extension, its future.

Not surprisingly, museums, given their focus on cultural heritage, are trying to take full advantage of the tools that AI and ML provide. But technology is advancing so quickly that museums are in an experimental phase to determine what works and what doesn’t. A recent blog by a young business tech consultant pointed out the challenge: there are so many possibilities open to museums now, that a close look at cost vs. relevance is essential. 

​With all the possibilities now available, museums must prioritize what services they can provide that best serve their visitors and their storytelling function. The Baseball Hall of Fame has made some interesting use of AI. With a huge collection of uncategorized baseball stories with accompanying pictures, they used ML to read through and appropriately tag the content so it can be fully utilized in exhibitions, enhanced visitor experiences, and online research.

Can AI and ML be taught to deeply understand and appropriately label ethnic and cultural heritage stories as well? That is one of the areas that researcher Wolfgang Victor Yarlott has explored. A member of the Crow nation, Yarlott wanted to see if Genesis, the story-understanding system developed by MIT's Computer Science & Artificial Intelligence Lab, could understand stories from Crow folklore. According to founder Patrick Winston, the Genesis system was built to model and explore aspects of story understanding using simply expressed, 20-100 sentence stories drawn from sources ranging from fairy tales to Shakespeare’s plays.  

Over the course of his work, Yarlott analyzed three collections of Crow literature, created a list of cultural features present in the stories, identified four as particularly important (unknowable events, medicine, differences as strengths, and uniform treatment of entities), and developed a set of five Genesis-readable stories in which those features were prominent.

“The key focus behind this system is: stories are an essential component of what makes human intelligence so remarkably different from that of other animals," Yarlott says."I believe that if Winston and the Genesis group are correct and stories are a key part of human intelligence, then it is necessary that Genesis, the system that serves to demonstrate this point, be capable of handling stories from all cultures, including less well-known cultures such as that of the Crow Indians, a tribe from the northern plains of the United States.” His research led to new elements in the story-understanding model, and showed that Genesis could indeed be adapted to understand stories from different cultures.

Genesis is one approach. Are other machine learning approaches relevant to cultures and traditions? Yarlott doesn't think that is the right question. "Specific machine learning approaches aren’t necessarily better or worse for approaching tasks that involve culture and tradition. While specific machine learning approaches are better suited to different types of tasks (e.g. clustering samples into groups or assigning labels to samples), what is more important to tasks involving culture and tradition are data, annotations, and features."

Collecting cultural data, whether it’s stories or music or photographs, is the first step. Annotating, or analyzing that data for cultural/historical information is next. Features refers to how that data and analysis are used by a machine learning algorithm. Data is central to everything ML can accomplish, and there has to be a sizable amount of it to be truly effective. But successful big data demands that proper care be taken to include multiple social and cultural perspectives. If it is not, bias might be inherent from the beginning, and that can then be difficult to eliminate. "A system to extract information trained only on articles from the Economist is going to be less capable when used out-of-domain on articles from a student newspaper," says Yarlott.

To achieve this, we must begin now to ensure that cultural data is properly tagged and vetted. That means charging people with those tasks who are not just knowledgeable in technology, but knowledgeable about, and preferably part of, the specific culture being collected and analyzed. We must recruit people from these communities, the people on the ground who live the culture every day and know it best, to become involved with the work of AI as well.

The future integration of cultural heritage with artificial intelligence is not only possible, it is essential if we want a diverse and inclusive society. Now is the time to ensure that we proceed correctly and give voice not just to mainstream culture but to our ancestry. At IVOW, we are at the early R&D stages of introducing identity and culture into AI-driven products.

We look forward to discussing these critical concepts on October 17 at Stanford at the Forum on AI for Culturally Relevant Interactions, sponsored by mediaX at Stanford University, in conjunction with IVOW, Baidu and Flybits.

About IVOW: We are a team of journalists, educators, technologists, and app developers with extensive experience in combining timeless principles of storytelling with new emerging technologies. We are affiliates of mediaX at Stanford University.