ArtSciLab Paper Accepted for Understanding Visual Music 2016, Brazil

An ArtSciLab paper by Andrew Blanton, Connectome Data Dramatisation: The Human Brain as Visual Music, has been accepted for Understanding Visual Music to be held June 10, 2015 in Brazil.

Below is an abstract:


 

Connectome Data Dramatisation: The human brain as visual music.

Authors: Andrew Blanton, MFA; Sruthi Ayloo, MS; Micaela Chan, MS; Scot David GreshamLancaster, MA, MFA; Roger Malina, PhD; Tim Perkis; Neil Savalia, BA; Maximilian Schich, PhD; Anvit Srivastav, MS; Gagan Wig, PhD

Abstract

We, as a collaboration of scientists and artists, have built a visual and sonic representation of highly connected areas in the human brain. This model was developed to not only be a tool of scientific research but also as a tool for art creation. In the process of developing the software, the tool was built to interface with musical instruments for real time visualization and sonification. Working conceptually with the idea that scientific data can be repurposed for art creation, the Connectome is performed as both a sonic and visual representation of fMRI data, manipulating the model in real time as a form of multimodal data dramatisation.

Introduction

Partnerships between artist and scientist allow for creative forms of collaboration that can push both scientific and artistic research. With the Connectome Data Dramatisation project, our principal interest was in the creation of a hybridized tool, one that could work as both scientific instrument as well as artistic work. Beginning with a dataset that consisted of 441 neural bundles or nodes systematically differentiated into 21 areas or systems of interest in the human brain based on fMRI data collected by one of us (Gagan Wing) as part of the work of the UTDallas Cognitive Neuroimaging Lab.[1]

Area Centers Coded by System Membership

Our team was able to extract visual and sonic representations of the connections between those areas using custom software. We then developed that representation further in the form of an interactive three dimensional node edge graph and sonification of the 421 highly connected areas of the brain (in the case of the visualization, the width of the edges).

This would form the basis of the representation. With the addition of the ability to activate nodes from external data feeds via Open Sound Control[2] different nodes could be excited at will creating a virtual, three dimensional instrument that could be used for visual and sonic performance. Using four small drums, the visual and sonic representation of connections between areas of the brain can be played in real time. Custom software receives input in the form of audio signal from each drum and excites specific areas of the brain. Each section of the brain that is played will present a unique visual and sonic representation.

Historical Perspective

Building on previous explorations in bridging art and science through the development of new technology, we were actively looking to understand how this project is situated within the history of visual music. In looking at the work done at Bell Labs in the 60’s and 70’s[3] and with the work of artist such as James Whitney[4], the question emerges, what are the components of a successful art and science collaboration? How do separate practitioners collaborate while furthering each of their own research? Phill Mortin and Dan Sandin’s image processing units[5] also played a role in both the conceptual development as well as the technical development of the work. How is information shared and disseminated after it’s creation? Other contemporary artist were looked at as well including the work of Noisefold[6] in their sound extraction techniques form visual information, Ryoji Ikeda[7] in his visual and sonic representation of data as well as Semiconductor[8] in their blending of art and science amongst others working with visual music as a contemporary practice.

Visual music has been historically tied to the development of technology. This holds true now as much as it has in the past. Current rendering technologies are evolving rapidly within the gaming community and practitioners of visual music are greatly benefiting from real time rendering advancements within the gaming communities. Robust community support and the indie gaming movement have provided new tools for interfacing with gaming environments[9]. Two areas that are underdeveloped with regard to these environments and practitioners of visual music can provide insight are in the development of procedural animation, and the assimilation of data into these environments. With this project we have begun to build a framework that can both provide a series of procedural animations with regard to node edge graphs as well as interface a gaming environment with a dataset of approximately 77,000 connections. In doing so we have tried to maintain the work as both a piece of art and a scientific instrument.

Future Work

In the process of building this project, we have worked with many technologies to find the right combination of frameworks and development to allow for extensive flexibility in artistic representation of the data set. We have worked with Max/MSP Jitter[10], Unity[11], Syphon[12], Three.js[13], node.js[14] midi.js[15], coffee collider[16] and D3.js[17] in a exploration to find what technology would serve the representation of this dataset best. Beginning with a representation using three.js hosted on a node.js server we were able to bring in live data via OSC to trigger the model. We found ultimately that building everything in the web browser provided great accessibility for global use of the tool, however, confining the project to the web browser also creates limitations with regard to power for rendering and audio synthesis. We have built a framework that now uses the Unity game development environment specifically for it’s strength with regard to real time rendering and are working on integration of Pure Data[18] via the Kilimba Unity extension[19]. This process will allow us to build a platform addressing the two primary areas of dataset integration into gaming environments and procedural manipulation as well as sonification and visualization of said dataset.

Summation of Findings

The creation of the Connectome project has led to some interesting further work in collaborations between artist and scientist. Beginning with the fundamental question can scientific instruments be used as tools for art creation and can artist tools produce scientifically valid results, our team was working to further a dialogue between artist and scientist while creating real value for each party involved. In doing so we have opened up another path of exploration in the form of using game development platforms for data visualization and sonification as well as the reappropriation of these platforms for use in real time audio visual work. By creating a core representation, we were able to build a model that could be manipulated in real time using incoming Open Sound Control data and provide a scientifically accurate representation of the underlying dataset.


[1] Area of interest in this case were areas of concentration of neurons in the brain as identified by researchers at of the Center of Vital Longevity Cognitive Neuroimaging Lab at the University of Texas at Dallas. http://vitallongevity.utdallas.edu/cnl/ accessed march 7 2015.
[2] http://opensoundcontrol.org/ accessed March 7 2015
[3]http://www.ieeeghn.org/wiki/index.php/Archives:Bell_Labs_%26_The_Origins_of_the_Multimedia_Artist accessed March 7 2015
[4] William Moritz on James Whitney’s Yantra and Lapis http://www.centerforvisualmusic.org/WMyantra.htm accessed March 7 2015
[5] Museum of Modern Art https://www.moma.org/momaorg/shared/pdfs/docs/press_archives/5958/releases/MOMA_1982_0014_14.pdf?2010 accessed March 7 2015
[6] http://noisefold.com/
[7] http://press.web.cern.ch/press-releases/2014/01/japanese-artist-ryoji-ikeda-wins-third-prix-ars-electronica-collide-cern
[8] http://semiconductorfilms.com/
[9] http://pjim.newschool.edu/issues/2011/01/pdfs/ParsonsJournalForInformationMapping_Medler-Ben+Magerko-Brian.pdf
[10] https://cycling74.com/ accessed March 7 2015
[11] http://unity3d.com/5 accessed March 7 2015
[12] http://syphon.v002.info/ accessed March 7 2015
[13] http://threejs.org/ accessed March 7 2015
[14] https://nodejs.org/ accessed March 7 2015
[15] http://mudcu.be/midi-js/ accessed March 7 2015
[16] https://github.com/mohayonao/CoffeeCollider/wiki accessed March 7 2015
[17] http://d3js.org/ accessed March 7 2015
[18] http://puredata.info/ accessed March 7 2015
[19] https://github.com/hagish/kalimba accessed March 7 2015

Arts, Humanities, and Complex Networks — 6th Leonardo satellite symposium at NetSci2015

We are pleased to invite you to the

Arts, Humanities, and Complex Networks — 6th Leonardo satellite symposium at NetSci2015

taking place at the World Trade Center Zaragoza (WTCZ) in Spain,
on Tuesday, June 2, 2015.

Abstract:
For the sixth time, it is our pleasure to bring together pioneering work in the overlap of the arts, humanities, network research, data science, and information design. The 2015 symposium will again follow our established recipe and will leverage interaction between the areas by means of keynotes, a number of contributions, and a high-profile panel discussion.

In our call, we are looking for a diversity of research contributions revolving around networks in culture, networks in art, networks in the humanities, art about networks, and research in network visualization. Focussing on these five pillars that have crystallized out of our previous meetings, the 2015 symposium again strives to make further impact in the arts, humanities, and natural sciences.

Running parallel to the NetSci2015 conference, the symposium provides a unique opportunity to mingle with leading researchers in complex network science, potentially sparking fruitful collaborations.

As in previous years, selected papers will be published in print, both in a Special Section of Leonardo Journal MIT-Press and in a dedicated Leonardo eBook MIT-Press (see below).

Keynote:
Martin Krzywinski, Scientist, Genome Sciences Center, Vancouver, Canada
Make sure to check out his websites: http://mkweb.bcgsc.ca/ and http://circos.ca/

As in previous years, our high-profile keynote exemplifies the areas of cultural data science, network visualization, and/or network art.

Organizing committee:
Maximilian Schich, Associate Professor, ATEC, The University of Texas at Dallas, USA
Roger Malina, Executive Editor at Leonardo Publications, France/USA
Isabel Meirelles, Professor, Faculty of Design, OCAD University, Toronto, Canada

Submissions:
We invite you to submit a 300 word abstract including one descriptive figure by March 29, 2015 using our EasyChair submission link:

https://easychair.org/conferences/?conf=ahcn2015

Note: Our previous calls had an acceptance rate of 14 – 25%. Contributors were selected using a peer review process with three to four independent reviews per paper. Succesful submissions usually include an abstract not exceeding 300 words (plain text in the EasyChair submission form, no paper attachment), a striking figure (.jpg attachment, optimized to about 2000 x 1200 pixel in landscape aspect ratio), as well as an URL, all of which should express the relevance to our call. Previously selected paper topics cover a large territory, including networks in archaeology, art, film, history, music, literature, network visualization, and the culture of art-science. For previous examples see our companion website at ahcncompanion.info.

Important dates:
Deadline for submission: March 29, 2015.
Notifications of acceptance will be sent out by April 6, 2015.
Date of symposium: Tuesday, June 2, 2015, in Zaragoza, Spain.

Attendance:
Important note: The main NetSci2015 conference requires satellite attendees to pay at least a one day registration fee, in addition to registering to our symposium via EventBrite.

To attend our symposium,
1. Please get a free EventBrite ticket at http://ahcn2015.eventbrite.com/
2. Pay at least a one-day registration fee at http://www.netsci2015.net.

If you do not get an Eventbrite ticket (but you paid the NetSci2015 fee), there is still a chance to attend, as some ticket holders may not be able to show up. We will fill these spaces in Zaragoza, again on a first come, first serve basis. Priority will be given to those on our Eventbrite wait list and those registered for the main NetSci conference.

Program Schedule:

9:00 – Roger Malina: Opening Remarks

9:10 – Maximilian Schich: Introduction

9:30 – Martin Krzywinski: Keynote talk

10:30 – Coffee break

11:00-13:00 – Contributed talks

13:00 – Lunch

14:00-16:00 – Contributed talks

16:00 – Coffee break

16:30 – Panel discussion

17:30 – End


 

About NetSci:
NetSci is the flagship conference on Complex Networks promoted by the NetSci Society. It brings under one umbrella a wide variety of leading researchers, practitioners and stakeholders with direct interest in Network Science, from Physics to Computer Science, Biology, Social Sciences, Economics, Technological and Communication Networks, Big Data and so on.

Links:
eBook on Arts, Humanities, and Complex Networks [Kindle Edition]:
Companion website: http://ahcncompanion.info/
For purchase (US $ 7.99): http://www.amazon.com/dp/B007S0UA9Q

Special Section in Leonardo Journal:
Leonardo Journal 43:3, June 2010, pp. 212: http://www.mitpressjournals.org/toc/leon/43/3
Leonardo Journal 44:3, June 2011, pp. 239-267: http://www.mitpressjournals.org/toc/leon/44/3
Leonardo Journal 45:1, February 2012, pp. 77-89: http://www.mitpressjournals.org/toc/leon/45/1
Leonardo Journal 45:3, June 2012, pp. 275-286: http://www.mitpressjournals.org/toc/leon/45/3
Leonardo Journal 46:3, June 2013, pp. 267-279: http://www.mitpressjournals.org/toc/leon/46/3
Leonardo Journal 47:3, June 2014, pp. 265-278: http://www.mitpressjournals.org/toc/leon/47/3

Previous Symposia websites:
2010: http://artshumanities.netsci2010.net
2011: http://artshumanities.netsci2011.net
2012: http://artshumanities.netsci2012.net
2013: http://artshumanities.netsci2013.net
2014: http://artshumanities.netsci2014.net

Other relevant sites:
ATEC, The University of Texas at Dallas: http://www.utdallas.edu/atec/
BarabásiLab, Northeastern University, Boston: http://www.barabasilab.com
Faculty of Design, OCAD University, Toronto: http://www.ocadu.ca/academics/faculty-of-design.htm
Leonardo/ISAST: http://www.leonardo.info
NetSci2015: http://www.netsci2015.net


 

Contact:
If you would like to be added to the list of interested people, please drop us an e-mail with the subject [ Please add me to the Arts, Humanities, and Complex Networks list ] at artshumanities.netsci@gmail.com. Alternatively you can follow us on Twitter.

Creative Disturbance celebrates its first year by celebrating International Women’s Day and Women’s History Month

Through March, Creative Disturbance is featuring podcasts that showcase women voices in topics of art, science and technology. Some of the more popular podcasts include:

•  Art and Technology Pioneer Liliane Lijn

•  Music, Science and Technology pioneer Pauline Oliveros

•  Internet pioneer and innovator Christine Maxwell’s discussion with educator Rebecca Nix, Why Big Dreamers Need to Know About Big Data.

•  Artist Caroline Ometz‘s scientific and artistic collaborations with UT Southwestern researcher Dhru Deb in Cancer: Finding Beauty in the Beast.

•  Drs. Kathy Ellins and Susan Eriksson address the current status of art and earth science collaborations and new directions that could bolster their utility in geoscience research and education.

..and other topics of interest.

Visit creativedisturbance.org/womensday to hear the talented, powerful and diverse voices, or add your own podcast at the Voices of the Crowd channel to join these more than 25 voices.

Contact us at feedback.createdisturb@gmail.com if you would like to post a podcast!


 

What is Creative Disturbance

Creative Disturbance is an international, multi-lingual, online platform that (once complete) will provide a unique virtual environment for the intellectually curious across the globe to meet, network, collaborate, create, and socialize.

One means of both sharing and spurring such interactions is through a dynamic collection of podcasts crowdsourced and produced by Creative Disturbance members.

These ‘conversations’ help illuminate and inform others on matters of interest across the Creative Disturbance community.

S.T.E.A.M. Power: The Power To Change The World

If you’re looking for more STEAM listening, we recently discovered this podcast over at steampowerpodcast.com that you should check out! From their about page:

“We are a podcast dedicated to bringing you the latest and greatest news stories from across the entire spectrum of Science, Technology, Engineering, Art and Mathematics; as well as the Maker Movement and various Open Source Initiatives. We seek to inspire the public imagination in S.T.E.A.M. topics thus enabling a society that is technically and scientifically literate. We do this believing that a society that balances technological and scientific pursuit with humanity is the best hope for our collective future.

We also seek to serve as an advocate for the digital rights of individuals. We are strongly committed to ensuring technology plays the role of good in developing and safeguarding the democratic ideals of personal freedom, spirited debate, unfettered elections, and open conversation — free from fear, censorship, and harassment. We will ensure such ideals are passionately defended the world over.

We strive to be an advocate for four core initiatives, they are:

1) STEAM education in the United States and around the world. From elementary school to college and beyond. Not just in the classroom but also within youth groups and in the home.

2) Raising the public awareness and appreciation for the impacts science, technology, engineering, art, and mathematics has on our society both now and in the future

3) Evangelizing the Maker Community, Open Source Movement and the Do-It-Yourself (DIY) mentality so that we can become a more creative, self-reliant, and openly collaborative society.

4) Advocating for the digital rights of individuals by ensuring any policies put in place by governments and corporations balance privacy with security. Thus ensuring that the rights and freedoms of the individual is never trumped by corporate greed or government overreach.”

About The Hosts

Mike Parks is an engineer, Navy veteran, and small business owner.  You can follow along on Twitter or Google+, and he occasionally rants on his blog.

Lisa Parks is a sociologist and Mike’s better half.  She is passionate about understanding the impact technology has on people, society, and education.  She is also the creative genius of Super Chic Dollies.

ArtSciLab & Leonardo Initiatives Seeks Experimental Publishing Intern

The ideal candidate is someone who is detail-oriented, enjoys the publishing process, communicates well with others, and is familiar with E-books. The candidate will acquire valuable experience with e-publishing tools and methods. Possible continuation over the summer since most of the work can be done online remotely with coordination meetings held on campus. The work is in the ATEC ArtSciLab under the supervision of professor Roger Malina

Responsibilities:

• Formatting existing or new articles for the upcoming Leonardo E-book according to the current style guide

• Updating style guide and process as it becomes more efficient

• Collaboration with various editors at MIT Press to supplement the development of Leonardo E-books

• Like a liaison with MIT Press, you will communicate and collaborate with E-book editors internationally

• Will help publish the E-book to Amazon

• Support the E-book’s online presence by maintaining its social media

• Responsible for the E-book companion website maintenance

• Could be tasked to experiment with other scholarly publishing platforms (i.e. publishing professional podcasts for the Leonardo Creative Disturbance channel)

Recommended:

• Previous experience in editorial work, social media, and front-end web development

• If interested contact Roger Malina at: roger.malina@utdallas.edu

Carolyne Adhiambo Ojwang Joins ArtSciLab

Carolyne Adhiambo Ojwang was born and raised in Kenya before I relocated to the US to join her family. She is currently a University of Texas at Dallas student in the Masters program in ATEC graduating in Spring, 2015. As an immigrant, she is  interested in the African Diaspora, especially the East African community and how they can use cultural discussions and technological initiatives to provide a gateway to the rest of the world. Her Masters project will address how to enable East Africa to become visible in the global scene as part of the ArtScilab experimental publishing initiatives.

She will be carrying out her masters project in collaboration with the Art Sci Lab Virtual Africa project led by Yvan Tina and contributing to the ArtSciLab Creative Disturbance podcast and collaboration platform.

NEA Grant Awarded to collaboration between University of North Texas xREZ Lab,University of Texas at Dallas ArtSciLab and Texas A and M C Star project for One Antarctic Night

We are thrilled to announce that xREZ lab and collaborators at UTD ArtSciLab and Texas A and M CStar antarctic telescope have received a prestigious NEA art works grant for creation of the interactive artwork Instrument” One Antarctic Night.

http://oneantarcticnight.xrezlab.com/

INSTRUMENT: One Antarctic Night is an interactive artwork created from 287‚800 images of the universe captured recently by a robotic telescope in Antarctica. Using this data about how the universe works‚ we are creating electronic instruments that participants interact with to make digital image and sound remixes. The experience is like a video and music jam session taking place in the gallery‚ on large scale displays‚ mobile devices‚ and online simultaneously.
The project is art+science collaboration between artists and astrophysicist led  by UNT (http://www.xrezlab.com/ ) , with UT Dallas Art SciLab, and Texas A&M (http://mcba11.phys.unsw.edu.au/~plato/cstar.html) with participating artists from RISD and SJSU. Watch the project video and learn more about this exciting artwork at http://oneantarcticnight.xrezlab.com.

ArtScilab collaborators Scot Gresham Lancaster and Brian Merlo and Andrew Blanton at San Jose State University are participating.

Susan Eriksson Joins ArtSciLab as a Research Fellow!

Susan C. Eriksson is a geologist, educator, and artist, with a specialty in translating cutting-edge scientific research into programs that impact students and society. She has served as a research scientist for a major oil company; faculty, administrator, and museum director at Virginia Tech; and Education and Outreach Director for UNAVCO, an NSF facility for geodetic research and for the Earth Observatory of Singapore.

She supports an emerging community of scientists and artists working together in the subject of the Earth by founding  Bella Roca, a website with news and articles on people and events in the geoscience and art arena and by co-convening several sessions on geoscience and art at the American Geophysical Union annual meeting.

Susan has exhibited her Earth-inspired art work nationally at the American Association for the Advancement of Science and the National Science Foundation.

Dr. Eriksson is also an independent consultant in strategic planning and evaluation of STEM programs.

Dr Eriksson is producer of the podcast channel on Art and Earth Sciences on the ArtSciLab Creative Disturbance project ( www. Creativedisturbance.org )

The EODIAH – UTD Leonardo Initiatives “Generation 0: Chronicles of the Art-and-Technology Vanguard”

The University of Texas at Dallas’ new Edith O’Donnell Institute of Art History (EODIAH) and the Leonardo Initiatives of ATEC at UTD are pleased to announce the Leonardo Generation 0 Project, an open and expanding archive of the foundational forces in art-and-technology.

 

After the Second World War, two major developments transformed the cultural landscape: the digital computer and discoveries in the realm of genetics. The Leonardo Generation 0 Project chronicles the multi-perspectival merger between technology and art from the 1950s through the 1980s.  Generation 0 amplifies the voices of pioneering and influential artists, engineers, curators, and key organizations through a growing databank of written first-person accounts and podcast recordings. The project seeks to document the experiences of those most closely involved in the creation of digital art, the biological arts, new media art, and computer art using the tools of the digital humanities. Leonardo Generation 0 shares the unique perspectives in a style that is at once embracing and accessible, intellectually rigorous yet casual. This new project is aimed at uncovering a rich, if somewhat underappreciated, time in art history by recording the memories of the pioneers.

 

The Generation 0 Project is part of the Leonardo Pioneers and Pathbreakers project of the Observatoire Leonardo des Arts et Techno-Sciences in Paris, and in collaboration with its director, Annick Bureaud. Memoirs are being published both on the Pioneers and Pathbreakers website athttp://olats.org/pionniers/pionniers.php and in the Leonardo Journal art history section edited by Professor David Carrier: http://leonardo.info/leoinfo.html

 

The co-directors of the Generation 0 Project are Professor Roger Malina, an affiliate faculty member of the EODIAH and an ATEC Distinguished Chair at the University of Texas at Dallas, and Dr. Charissa Terranova, an Associate Professor of Art History in EODIAH. ATEC ArtSciLab Research Fellow, Poe Johnson, a University of Texas at Dallas PhD student, coordinates the project. Their varied perspectives and skill sets embody the hybridity of the memoirs project itself, and the philosophy of University of Texas at Dallas ATEC and Arts & Humanities programs.

 

Among the recent memoirs are an inside account of the workings of Bell Labs from A Michael Noll, Helen and Newton Harrison about their early work in art and environment, Frieder Nake views as a pioneering German computer artist, Trudy Reagan early days of the YLEM organization.

 

Submissions for the Pioneers and Pathbreakers memoir project are decided and peer reviewed by The Frieda Ackerman Committee: Marc Battier, Paul Brown, Annick Bureaud, David Carrier, Joel Chadabe, Anne Collins, Eduardo Kac, Roger Malina, Patrick McCray, Frieder Nake, Louise Poissant, Eddie Shanken, and Charissa Terranova. The call for memoirs is available at http://leonardo.info/isast/journal/calls/pioneers.html.

New Research Assistant in ArtSciLab – Anvit Srivastav

We are pleased to introduce the new Research Assistant in the ArtScilab.

Anvit Srivastav is a master’s student in Computer Science at the University of Texas at Dallas. He received his bachelor’s in Information technology from Jaypee Institute of Information technology in India. He has worked extensively with web technologies and is skilled in Ruby on rails, Javascript and, PHP. His areas of interest are Game development, Cloud computing, Web design and, Network security. He is currently specializing in the field of Information assurance. In his spare time, he likes to work on music production, digital art and play the guitar. He is a huge metal head and is always up for concerts if you’re looking for someone to go with.

Anvit will be working on our DataRemix project with Connectome data and on the Creative Disturbance project.