‘Painting’ with Data: How Media Artist Refik Anadol Creates Art Using Generative AI

The Turkish-US media artist and designer is a pioneer in the aesthetics of data and machine intelligence. Examining creativity at the intersection of humans and machines, he uses data-driven machine learning algorithms to digitize memories and create abstract, colorful environmental art. His work has been exhibited at New York’s Museum of Modern Art, the Centre Pompidou-Metz, Australia’s National Gallery of Victoria and the Venice Architecture Biennale.

Anadol talks to WIPO Magazine about the convergence of art, science and technology, the potential of generative artificial intelligence and how he’s reconstructing nature using images of flora and fauna.

How do you describe your work?

First, I’m a media artist and director. I started programming computers and learned that when you create software and use it to expand your imagination, you can make the invisible visible. The elements machines use to communicate with each other can be turned into digital pigments to create a work of art.

I love computers. I started with computer games and since high school I’ve been building an art practice using machines.

I started using my own software to create art in 2008. I coined the term “data painting” when I started imagining the pigmentation of the data around us – the sound, the vision, the internet. I believe anything in life that is quantifiable can become an artwork. Data is not just numbers; it’s a form of memory and can take any shape or form.

“What happens if a machine can dream and, if it can, who defines what is real and what is not?”

Tell us about your creative process.

In general, the data that surrounds us is the inspiration for my work. Using generative artificial intelligence, we train machine-learning algorithms by harnessing large, focused and publicly available datasets to visualize nature, urban life and culture. Since the COVID-19 pandemic, my focus has been on compiling the largest possible dataset to artificially preserve nature.

You’ve been working with artificial intelligence (AI) and generative AI for some time.

I started my journey with artificial intelligence in 2016 when I became an artist in residence at Google. During an incredibly inspiring year, my team and I learnt how to use algorithms. In our first project, we worked with Salt, an open-source library in Turkey, to create our Archive Dreaming project. It was the first AI artwork to use public data to create art.

The question was: what happens if a machine can dream and, if it can, who defines what is real and what is not? The idea that machines can dream was a fundamental starting point. Over the past few years, computational power has improved so much and has given us a whole new perspective on how the datasets we leave behind can become new experiences.

“AI does not yet have consciousness. However, we expect it will acquire some level of consciousness in the future.”

We created a live AI exhibition called “Unsupervised” at the Museum of Modern Art (MoMA) in New York, which ran until the end of October 2023. It allowed visitors to experience AI that is infinite and constantly dreaming. The software we created for the installation uses data related to vision, sound and the climate. It was our most ambitious display so far and in developing it we created a living painting or artwork.

Is AI capable of independent creativity?

AI does not yet have consciousness. People fear that AI has decision-making power but, at present, it can only predict what may happen in the future based on past trends. However, we expect it will acquire some level of consciousness in the future. That’s one of the reasons we display the names of the AI, the algorithm and the data used in our works. At the MoMA exhibition, we dedicated a screen to our data sources and how the AI used them, to help visitors get a better sense of the processes involved.

What is the balance between human input and generative AI in your works?

The balance is about 50–50. My creations are true human-machine collaborations. I’m very optimistic about generative AI because of its potential to enhance our memories. As artists, we can utilize this potential, for example, to represent nature in the digital age, so those who see our works remember how it feels to be immersed in the natural world. Generative AI can train algorithms with image, sound, text or even scent data.

When creating an AI imagination, it is not as simple as putting something in and getting something out. With each project, we start from scratch and it takes months to complete the data collection and training processes. First, we curate data and use it to train the AI. Then, ironically, we teach the AI not to learn too much but to dream intentionally to create a unique imagination. We don’t use existing models. This is an important distinction between our work and other AI works. Imagine a camera in the mind of a machine. We program it and, as the AI learns, it stores the information in 10 to 24 dimensions.

We have been working since 2014 on the software to create this “brush”, which we dip into data. I call this “AI data painting”. We take the information from the machine’s “mind” and transform it into a digital canvas, which could take the form of a three-dimensional sculpture, like an AI data sculpture, or an immersive room or public building.

Many things we experiment with don’t work or just don’t feel right, whatever our artistic intention. We have many more failures than successes but we are constantly learning how to use this medium for artistic expression.

“We want to find new ways to use data to enrich our cultural heritage.”

What other projects stand out for you in this regard?

When doing research for our upcoming museum in Los Angeles, DATALAND, I was fortunate enough to be able to spend time with the wonderful leaders of the Yawanawà people, who live in the Amazon rainforest in the state of Acre in Brazil.

I was deeply inspired by their ways of learning and their knowledge of the natural world. Together we developed an open-source AI rainforest model built around respectful co-creation and to preserve their language. With this model, generative AI can even “reconstruct” extinct flora and fauna based on people’s collective knowledge.

We are now taking a closer look at new, more creative ways to use sources of data to preserve humanity’s memories. So much data is being generated today, it would be a real pity to simply dump it. We want to find new ways to use it to enrich our cultural heritage.

Is this where your open-source Large Nature Model, which was behind the “Echoes of the Earth: Living Archive” exhibition at London’s Serpentine Gallery, comes in?

Yes, we are developing a Large Nature Model, the first open-source generative AI model focused on nature images and sounds. For this research, we have ethically harnessed more than half a billion open-source data images from organizations such as the Smithsonian Institution, National Geographic and London’s Natural History Museum, as well as from various rainforests around the world.

“We never use personal human data in our creations. We’re very careful about that.”

My interest in nature took a turn about eight years ago, when my life partner and studio co-founder, Efsun, introduced me to Amazonian culture. It was then that I grasped how incredible and important Amazonia is. This revelation sparked a lasting interest in the dynamics of natural movement. I’ve been exploring the movement found in nature since 2010, creating simulations – or what I call “data pigmentations” – based on data related to water, wind and weather patterns. The pandemic posed a unique challenge. We couldn’t go to nature so the question was: could we instead bring nature indoors? We started archiving nature data and have gathered more than four billion images of flora and fauna to date.

This must have required a huge team.

We are a team of 15 people with diverse backgrounds. We have architects, designers, AI engineers, data scientists, musicians, philosophers, neuroscientists and more. Our shared dream is to translate the new language of humanity, as expressed by data, into art forms.

What level of control do you have in the creative process?

I’m hands-on. I’m involved in setting every single parameter. That is how I maintain a connection with my work and make it my own.

The provenance of the data used to train generative AIs is one of the big debates. Where do you source the original data you mentioned?

For some works, we collaborate with institutions such as the NASA Jet Propulsion Lab (JPL) or the MoMA and use their incredible datasets. Zaha Hadid Architects and Casa Batlló have also shared their data with us. In other cases, we use public data. Archives are also a huge inspiration. But nature is our ultimate inspiration.

We never use personal human data in our creations. Rather, we focus on things that belong to humanity. We’re very careful about that. We generally take open-source software and develop and improve it. Then we share it with neuroscientists or botanists. That way we are also supporting their research. It’s not just about shiny pixels.

Do you mind other people sharing, copying or using your work without authorization?

Yes, I certainly do. While I’m an educator – I lecture at UCLA’s Department of Design Media Arts – and share my knowledge to help others advance, I find that with open sharing, many more people copy my work without referencing it, even critics. The more people use my work without permission, the more I feel I have a responsibility to protect it, especially as it rises in value among collectors around the world. Without protection it becomes a free-for-all and nobody moves forward.

We spend years working on projects, so when I see someone mimicking an idea, that hurts, because it completely negates the time and energy my team and I put into it. Just as researchers are credited for their scientific work, artists need to be credited for their creativity. That’s why copyright is so important.

“Sharing knowledge and best practices can help all of us navigate the evolving world of generative AI and maintain the integrity and originality of our art.”

Do you have tips for other artists when working with generative AI?

It’s crucial to understand and manage the data inputs. By carefully curating the datasets you use, you can ensure that the output aligns with your artistic vision while respecting the intellectual property of others. Transparency and ethical considerations are also crucial. I am clear about the sources of my data and obtain the necessary permissions. This practice not only protects my work but also builds trust with my audience and collaborators.

Lastly, I encourage artists to engage with the broader community. Sharing knowledge and best practices can help all of us navigate the evolving world of generative AI and maintain the integrity and originality of our art.

What projects are you currently working on?

Dataland Los Angeles is our next journey and an important milestone that marks our studio’s 10th anniversary. It will unite innovators in visual worldmaking, science, technology and academia under the artistic leadership of Refik Anadol Studio. It will showcase artwork but also focus on research, education and embracing open-source practices, and serve as an example for future generations navigating AI.