MCN 2024

The CDC reports that approximately 7M Americans suffer from blindness or vision loss. The Health Policy Institute at Georgetown University estimates that 20M people—8% of the American population—have some form of vision impairment that glasses, contacts, or surgeries cannot correct. Many museums, including those featured in this panel, have little, if any, of their digital collection augmented by visual descriptions, or alternative text (alt text) for this large population.

As a group of US-based art museums, we’ve been exploring the very real future of using multimodal AI to begin to bridge this gap: using AI “vision” to describe, tag, and provide image descriptions for artwork. After an initial convening in November 2023, we’ve developed a collaborative, inter-museum working group to explore this idea along with the technical, practical, ethical, and conceptual challenges inherent in this arena. This group includes the National Gallery of Art, The Metropolitan Museum of Art, the Cleveland Museum of Art, the Art Institute of Chicago, the Getty, the Harvard Art Museum, the Georgia O’Keeffe Museum, the Smithsonian Institute, and other partners.

This multi-presenter session will focus on our initial findings, insights, and the next steps we hope to explore. Specifically, the talk will roughly be divided into the “why”, the “how”, and the “what now.” The “how” section is our core and will feature comparisons of results across different models, insights into prompt engineering, strategies for systems architecture, and explorations around evaluation and human feedback. The “what now” section will introduce where we intend to keep exploring, with each presenting museum sharing current and upcoming efforts. Finally, we will invite attendees to join us elsewhere at the conference (location TBC!) to explore individual use cases and, when possible, dig into working examples.