We in Telegram
Add news
March 2010 April 2010 May 2010 June 2010 July 2010
August 2010
September 2010 October 2010
November 2010
December 2010
January 2011
February 2011 March 2011 April 2011 May 2011 June 2011 July 2011 August 2011 September 2011 October 2011 November 2011 December 2011 January 2012 February 2012 March 2012 April 2012 May 2012 June 2012 July 2012 August 2012 September 2012 October 2012 November 2012 December 2012 January 2013 February 2013 March 2013 April 2013 May 2013 June 2013 July 2013 August 2013 September 2013 October 2013 November 2013 December 2013 January 2014 February 2014 March 2014 April 2014 May 2014 June 2014 July 2014 August 2014 September 2014 October 2014 November 2014 December 2014 January 2015 February 2015 March 2015 April 2015 May 2015 June 2015 July 2015 August 2015 September 2015 October 2015 November 2015 December 2015 January 2016 February 2016 March 2016 April 2016 May 2016 June 2016 July 2016 August 2016 September 2016 October 2016 November 2016 December 2016 January 2017 February 2017 March 2017 April 2017 May 2017 June 2017 July 2017 August 2017 September 2017 October 2017 November 2017 December 2017 January 2018 February 2018 March 2018 April 2018 May 2018 June 2018 July 2018 August 2018 September 2018 October 2018 November 2018 December 2018 January 2019 February 2019 March 2019 April 2019 May 2019 June 2019 July 2019 August 2019 September 2019 October 2019 November 2019 December 2019 January 2020 February 2020 March 2020 April 2020 May 2020 June 2020 July 2020 August 2020 September 2020 October 2020 November 2020 December 2020 January 2021 February 2021 March 2021 April 2021 May 2021 June 2021 July 2021 August 2021 September 2021 October 2021 November 2021 December 2021 January 2022 February 2022 March 2022 April 2022 May 2022 June 2022 July 2022 August 2022 September 2022 October 2022 November 2022 December 2022 January 2023 February 2023 March 2023 April 2023 May 2023 June 2023 July 2023 August 2023 September 2023 October 2023 November 2023 December 2023 January 2024 February 2024 March 2024 April 2024 May 2024
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
News Every Day |

I'm using AI to 'read' my paintings and help me compose music. Here's how I do it.

Shane Guffogg
Shane Guffogg said AI helped him "unlock the musicality" in his paintings.
  • Shane Guffogg is a multi-media abstract artist with synesthesia, meaning he "hears color." 
  • Guffogg worked with AI experts and musicians to compose music that corresponds to his paintings. 
  • He believes AI is still a tool that "needs oversight" but it's enhanced his creative process. 

This is an as-told-to conversation with Shane Guffogg, an American artist who launched "At the Still Point of the Turning World – Strangers of Time," an exhibition of 21 paintings at the Venice Biennale earlier this month. This conversation has been condensed and edited for clarity.

I have synesthesia, which means I hear color.

So, what I'm listening to when I paint is important. I listen to Indian classical music, Gregorian chants, and some obscure composers such as György Ligeti, Leo Ornstein, and Terry Riley. The music sparks my creativity and allows me to be completely present and in that moment.

For years, I've been preoccupied with what my paintings might sound like. The AI revolution pushed me to search for experts who could help me. My first point of contact was Radhika Dirks, an AI and quantum computing expert. We had a couple of Zoom sessions, and she told me — to the best of her knowledge — that no AI program could help me. Instead, she suggested I create a visual alphabet that matched the musical chords I heard in my mind to colors.

I thought it could be a way to propel my creativity. It also built upon the idea of an unconscious alphabet that has informed my art throughout my career.

I met with musicians and AI experts to create a visual alphabet

I started by looking for musicians to collaborate with and met Anthony Cardella, a young, incredibly gifted pianist in Los Angeles. He's a Ph.D. student at USC and happened to know — and even play — many obscure composers I listen to when I paint.

We started collaborating. We would sit down and examine my paintings together. I would zoom in on a color in Photoshop, look at it, and sensorially feel the musical note. Then I would tell Anthony. I'd say, for example, I think that's the color of the note B. He'd hit the B, and I'd say, "No, that's not it; try a B sharp?" After a few trials, he'd suddenly hit the right notes. I would know because the colors would begin to vibrate for me. Together, we've charted chords that correspond to 40 colors.

Soon after, I met an AI researcher named Jonah Lynch through mutual contacts. He works at the intersection of the digital humanities and machine learning. I invited him over to my ranch in central California and explained the work I had been doing and how I created my paintings. We had long discussions about art, poetry, and creating an AI algorithm that could be fed the chords.

He developed a program to "read'' my paintings and convert them into music. I gave him the main colors I used in each painting and the chords I hear when I see those colors. Jonah watched videos of me painting, studied the movement of my hands, and wrote software that sampled images of the paintings, following my hand movements, and assigned each color sampled from the paintings to its corresponding chord. Then, he fed this sequence of chords into a neural network that has memorized most of the last 500 years of keyboard music. He prompted the network to "dream" of new sequences based on the color-chord sequences and the history of Western music to create pages of sheet music.

When I heard that music played back to me, it brought tears to my eyes. It was just a rough version of what I heard while painting, but I thought, "There it is."

I took the music back to Anthony, the pianist. Amazingly, I could point to the sheet music and tell him what compositions I was listening to while painting, and he'd say, "Yes, I can see it in the chords." The Indian ragas, the Gregorian chants, the Ligeti, and Ornstein — they were all there.

Still, the music was largely a series of chords at that stage. Anthony said we could have melodies if we rearranged it a bit.

AI is still a tool that needs human oversight.

Only Through Time Time is Conquered by Shane Guffogg
Guffogg's piece, Only Through Time Time is Conquered, was the basis for the sonata Cardella played for guests at the Venice Biennale.

We composed music for several paintings and have played it for audiences worldwide. We held a concert last month at the Forest Lawn Museum in Los Angeles, where I also had a few paintings in a show. The audience could look at the paintings while Anthony played, which was a profound experience. A couple of people cried.

At the launch of my latest exhibition during the opening week of the Venice Biennale, Anthony played the world premiere of a sonata he composed inspired by my painting, Only Through Time Time is Conquered, to a live audience. After the performance, I talked to several people, and they said they could see where the colors and the notes met on the painting. It was something they had never experienced.

I know many people are very afraid of AI, and I, too, see it as a tool that needs human oversight. It's not a means to an end. Still, it opened up many possibilities and enhanced my creative process. I don't know if I could have unlocked the musicality in my paintings in a real way without it.

Hear the sonata below:

Read the original article on Business Insider

Собянин рассказал о развитии инновационной экосистемы Москвы

Ballroom culture coming to the Long Beach Pride Festival

AML check crypto

Glen Powell’s parents crash Texas movie screening to troll him

Precision in Motion: Exploring Baltic Bearing Company-Riga


Read also

Skepta Criticizes Previous Nike Deal, Says He Was Treated Like an ‘Influencer’ for Sneaker Designs

2024 NBA Eastern Conference Finals predictions: The Pacers' cute little run will end against the stacked Celtics

‘It’s summer’s must-have dress’: Primark worker reveals the £12 frock ‘everyone needs’ that flatters all shapes

News, articles, comments, with a minute-by-minute update, now on Today24.pro

News Every Day

Glen Powell’s parents crash Texas movie screening to troll him

Today24.pro — latest news 24/7. You can add your news instantly now — here

News Every Day

Glen Powell’s parents crash Texas movie screening to troll him

Sports today

Новости тенниса
Дарья Касаткина

Касаткина, Андреева и Кудерметова — в тройке лидеров в борьбе за звание лучшей теннисистки

Спорт в России и мире

Курсант Пермского военного института Росгвардии принял участие в финале интеллектуальной олимпиады Приволжского федерального округа

All sports news today

Sports in Russia today


Олимпийский чемпион Роман Костомаров поздравил победителей номинации «Экскурсия без границ» программы «Лига Экскурсоводов»

Новости России

Game News

Шапки женские вязаные на Wildberries, 2024 — новый цвет от 392 руб. (модель 466)



Архангельск впервые принял Всероссийские соревнования по танцевальному спорту

Губернаторы России

Проводы Джикии, дубль Сулейманова и парады голов в Нижнем Новгороде и Екатеринбурге: чем запомнился 29-й тур РПЛ

Бонгонда: стадион и база «Спартака» — мои любимые места в Москве

Шапки женские вязаные на Wildberries, 2024 — новый цвет от 392 руб. (модель 466)

До конца года в Москве проведут почти в 2 раза больше операций, чем в 2023 году

Путин наградил председателя Общественной палаты Ставрополья Орденом дружбы

Певица Анна Плетнева чуть не задохнулась на сцене из-за конфетти в виде сердца

«Просто не афиширую»: куда пропала Волочкова с театральной сцены

«Считают каждую копейку»: Юрий Лоза рассказал о трудностях Максима Галкина* и Аллы Пугачевой в Израиле

Цой жил: съемки звезды «Кино» и эксперименты Билли Айлиш

Рыбакина узнала свое место в новом мировом рейтинге

Рахимова прошла во второй круг турнира WTA в Рабате на отказе Таунсенд

Сумасшедший матч «Реала», Медведев опустился в рейтинге ATP. Главное к утру

Шнайдер вышла в финал турнира WTA-125 в Париже, обыграв Грачёву


"Возрождение интереса к народному искусству и ремеслам в современном мире"

«СВЯТОЙ ЛЕНИН» УЛУЧШАЕТ ЗАКОНЫ, управляет патентами и улучшает командное планирование в целях учёта интереса всего народа.

Олимпийский чемпион Роман Костомаров поздравил победителей номинации «Экскурсия без границ» программы «Лига Экскурсоводов»

Тревел-тату как искусство: Яндекс Плюс запустил проект «Из путешествия – с татуировкой»

Выездной Фотограф для всех желающих, ну и конечно Артистов и Музыкантов.

Шапки женские вязаные на Wildberries, 2024 — новый цвет от 392 руб. (модель 466)

Июньские дни духовности: Православные праздники месяца от ясновидящей Галины Янко

Полпред РТ в РФ и ректор Финансового университета обсудили вопросы сотрудничества

Очевидец рассказал, что участники перестрелки в Москве покинули место ЧП сами

Сергунина: в Москве появилась собственная платформа по разработке моделей ИИ

В Москве после драки в Бирюлево возбуждено уголовное дело

Путин в России и мире

Персональные новости Russian.city

MK.RU: Певец Андрей Данилко удалил пост, посвященный смерти президента Ирана

News Every Day

AML check crypto

Friends of Today24

Музыкальные новости

Персональные новости