Conceived by Stamen Design-one of the leading computational design agencies for digital cartography-Aman created the “brain” of the system at the Frank-Ratchye STUDIO for Creative Inquiry during his sophomore year. DigitalGlobe, which provided satellite imagery, also provided financial support that allowed Stamen to host Aman’s work on Penny.

To create the model, Aman overlaid income census data on satellite imagery to train a neural network to predict what visual patterns best predict high- and low-income levels. For example, the network discovered that swimming pools and solar panels are signs of higher income, while brown lawns and industry tend to indicate poorer communities. Once these correlations were made, the model became able to predict income levels solely based on a satellite image.

“Aman’s work was essential to Penny’s successes. He took it further than our best people could have,” says Eric Rodenbeck, CEO & Creative Director of Stamen Design.

Penny also allows users to change aspects of a neighborhood—adding a parking lot or high-end hotel, for example—to see how AI predicts the value of the neighborhood would change.

“This playful and thoughtful UI lets you drop Trump Tower into housing projects, to see what effect that has on what an artificial intelligence thinks about income levels there,” Rodenbeck explains.

Penny’s predictions are sometimes logical, while at other times the results may be counterintuitive.

“For me it raises questions of what is meaningfully predictable. Say you modify a place and Penny responds in an unexpected way, the prediction is effectively unverifiable but also quite compelling,” says Aman.

What on the surface appears to be a tool to predict wealth, Penny is actually a system to examine artificial intelligence and machine learning. Penny enables users to interrogate what the system has “learned” about the world and explore the structure of its biases, failures, quirks and successes.

“The real point of Penny isn’t to predict wealth, but to be able to directly poke and prod at systems humans can be inclined to blindly trust – algorithms aren’t objective, and it’s important for us to be able to explore their non-objectivity,” says Aman. “As systems built using machine learning increase their influence on our lives it is important to be able to question them in a direct way.”

Penny is a continuation of the work Aman has done at CMU. As a freshman, he worked as an undergraduate research assistant at the STUDIO and helped develop the Terrapattern project led by Professor Golan Levin.

Penny has been written about in many publications including Wired, Fast Company, Hyperallergic, CityLab, and Vice.