Learning exactly how crappy AI vision is:
text2image experiments with pulp magazine covers
2025
For this project, I am creating an interactive website that allows users to explore how different AI vision models classify visual information. The site features data visualizations of generated phrases from various image-classification or image-captioning models.
project website
Project Description
This project explores how different AI vision models classify visual information using pulp magazine cover art from speculative fiction pulp magazines (mostly 1920s–1950s). The interactive website of this project allows users to view data visualizations of generated tags from image-classification or captioning models such as google/vit-base-patch16-224, SmilingWolf/wd-v1-4-vit-tagger-v2. By comparing these tags between models and to the actual cover art content, users can observe the models’ inaccuracies, biases, and limitations.
Additionally, the copy-writing explainations on the website also provides some insights into the contextual influences shaping each model. I paid attention to explaining how training data and creators’ positionality shape the model outputs.
Rationale Statement
My three objectives:
- Critique the interpretive limitations and biases in AI vision models, making the argument that characteristics of the training data and the positionality of its creators are amplified in every aspect of the model’s output. Using AI as a tool for visual analysis can reveal surprisingly little about the data while exposing much more about the models themselves.
- Provide a tool for anybody to learn more about how AI vision models 'understand' visual content in a direct, intuitive, and interactive way. Through comparison and specific examples of phrase-image pairings, a potential user can get a feel of the relationship between a given model's classification data and visual attributes through exploration.
- An experimentation to combine cultural heritage data, web-scrapting, AI, information experience design and data visualization in practice to help us better understand a topic.
A word on the data
This project uses a specific dataset with intention: pulp magazine covers in association with early women writers and cover artists in science fiction, as indexed by Lisa Yaszek's book, Sisters of Tomorrow. The genre of pulp fiction is also known for being overtly sexual and sexist, many featuring sex, violence, exploitative and racially problematic themes. However, among these artifacts lies a hidden movement of women artists and writers who subtly pushed against traditional norms through their contributions to pulp magazines. Female writers and artists in speculative fiction also used the creative freedom from pulp to create work that incorporated feminist perspectives deemed subversive at the time.
The pulp magazine cover art of speculative fiction from this dataset offers a fascinating snapshot of this mix of constraint and rebellion, which makes it an interesting subject to test the AI vision models for my project purpose. The images are suggestive and 'problematic' enough to reveal the models' different treatment and flaws when dealing with complex and questionable themes in visual content. The lurid nature of pulp also allows me to do so without having to run problematic tagger models on photographs of real people or sourcing out-right explicit images.
On the other hand, the history and contributions of women in sci-fi pulp magazines offer an optimistic counterpoint to the grim critique of AI. The popularization of pulp was a product of technological advancement: the availability of cheap and easy printing on pulp paper. Just as these women used Sci-Fi to imagine better futures and used the contradictory genere of pulp fiction to push forward these narratives, we can draw inspiration from their legacy to push for transformative narratives through AI. Despite the flaws and unsettling aspects of how AI is built, the works of women in early Sci-Fi and pulp remind me of the potential to repurpose imperfect systems as tools for creativity and progress.
By-product: a dataset for Sisters of Tomorrow
Building on the research of Lisa Yaszek in Sisters of Tomorrow, a broader dataset was also created as part of this project. It includes over 2364 entries of publication information from 27 women artists and authors in the golden age of pulp science fiction, filtered specifically for works with documented cover images.
References
- This project was created as part of the course Programming Cultural Heritage By Filipa Calado @ Pratt Institute School of Information
- Gallo, M. A., Lefanu, S., & Yaszek, L. (Eds.). Sisters of Tomorrow: The First Women of Science Fiction. Wesleyan University Press. LINK
- University of Connecticut. "Feminism and the Golden Age of Science Fiction Pulp." LINK
- Brundage, M., & Yaszek, L. (Ed.). The Heads of Cerberus and Other Stories. MIT Press. LINK
- AI Vision Models:
- Internet Speculative Fiction Database.