Staging Life: AI and Biotech in Art

This past March, Yvan Tina, the coordinator of the Creative Disturbance podcast project in the ArtSciLab, defended his dissertation at the School of Arts and Humanities of Aix-Marseille University in Aix-En-Provence. Yvan’s PhD program is very unique in the fact that it is a joint supervision thesis between Aix-Marseille University and UT Dallas under the direction of Professor Yannick Butel (Aix-Marseille University) and Roger Malina (UT Dallas).

Yvan presented his dissertation at Aix-Marseille University on March 16, 2018 about the use of artificial intelligence and biotechnology in art has led to a radical reformulation of theater as living performance. Below is his thesis abstract:

The use of artificial intelligence and biotechnology in art has led to a radical reformulation of theater as living performance. These technoscientific practices have displaced the subject of performance and produced various new discourses: In this study, I propose to make use of these discourses to expand the frame of theatricality to the realm of artificial life art. The displacements operated by means of theatricality in the artistic field are taking place both on the level of the artworks and the level of discourse.

In light of such operations, we see the potential of transformation relying on the use of these materials in theatrical aesthetics, as well as the obstacles found in them. Taking place between the arts and the technosciences, the study proves that the theatricality of technological works relies on the artifice of language.

The jury he presented to composed of Amos Fergombe (University d’Artois), Michael Osborne (Oregon State University), Roger Malina (UT Dallas) and Yannick Butel (Aix-Marseille University). He was awarded the title of Doctor in Performing Arts at the end of his defense.

The criteria and standards to start a PhD in France is much different than it is in the States and because of this there were a lot of administrative complexities dealing with Yvan’s PhD.

In France, it is a bottom-up system: You can only get accepted to the PhD program once you submit your proposal of your dissertation. In the US, you submit your proposal after taking doctoral exams and being admitted to the PhD program.

Because Yvan started his PhD in France then enrolled as a PhD student at the School of Arts, Technology and Emerging Communications in 2014, it made things a bit more complicated. His PhD program at UT Dallas was different than in France and going through what he needed to be qualified was essential to his completion of his PhD.

Yvan is currently still conducting research at both universities and he plans on staying in Dallas this upcoming semester to work on the translation of his dissertation from French to English, take his doctoral exams, and redevelop the Metalife website so that it can be used as a research tool.

Lab Alumna Wins Big Idea Competition

Student Veena Somareddy won the top $15k prize as well as the $2.5k Diversity and Inclusion Award with Neuro Rehab VR

ArtSciLab alumna and Arts and Technology PhD student Veena Somareddy is this year’s UTD Big Idea Competition winner. Veena is Neuro Rehab VR co-founder and software engineer at Fort Worth’s Neurological Recovery Center.

Neuro Rehab VR is a startup working to disrupt the field of physical therapy with virtual and augmented reality games leveraging the brain’s neuroplasticity.  Suffering a stroke or traumatic brain injury can leave a person with long term disabilities. Neuro Rehab VR has developed three games that focus on different areas of the body and brain. Patients at the Neurological Recovery Center have been using them, and there are plans to expand to five more clinics across the U.S. by the end of the year. Winning a $15,000 grand prize and a $2,500 Diversity and Inclusion Award, at the UTD Big Idea Competition, will help to hire more software developers.

The UTD Big Idea Competition put on by the Institute for Innovation and Entrepreneurship at UT Dallas was held in the ATEC Lecture Hall on Thursday, November 16. With a prize pool of $80,000 categories for this year included diversity and inclusion as well as social impact.

The 2017 Big Idea judges included Julie Nickols, partner at Haynes and Boone, Courtney Caldwell, co-founder of ShearShare, Jeff Williams, partner at Interlock Partners, and Bob Metcalfe, co-founder of Ethernet and director of innovation at the University of Texas at Austin. Guy Kawasaki, a former Apple executive and current chief evangelist at Canva, also served on the judging panel and gave a keynote speech about the art of innovation.

Roger Malina Receives Honorary Degree

Roger Malina has been awarded an honorary degree from the Technical University of Valencia in Spain for his work promoting and advancing research at the intersection of art, science and technology.

utdallas_94235487_medium-768x1024

The Spanish University cited his role as director of the ArtSciLab as a contributing factor. As a transdisciplinary research lab, the ArtSciLab focuses on innovative projects such as the podcast platform Creative Disturbance.

For 25 years, Malina has been involved with the Leonardo organizations, which his father founded in San Francisco and Paris. The organizations strive to promote work that explores the interactions between the arts and sciences, as well as between the arts and new technologies. Malina currently serves as the executive editor of the Leonardo journal, published by MIT Press.

Malina earned his bachelor’s degree in physics from Massachusetts Institute of Technology and his doctorate in astronomy from the University of California, Berkeley.

PhD Student Attends Fellowship Program Conference

Conference Chaz Lilly

Charles Lilly, a Phd Student in the School of Arts, Technology and Emerging Communication and member of ArtSciLab, received a Society for Scholarly Publishing Fellowship in March, 2016.

Lilly attended the 38th Annual Meeting of Society for Scholarly Publishing took place in Vancouver, Canada. This was the first year of the SSP Fellowship Program. While its previous incarnation (the Travel Grant Program) provided funds to attend the Annual Meeting, the Fellowship Program adds something much more valuable than money: mentorship, and immersion in everything that the SSP has to offer, through the opportunity to join committees and conversations.

The group of 12 Fellows consisted of seven early career professionals and five students – of whom three were “international Fellows” from outside the US. The different types of organizations, universities and backgrounds that we represented led to a thought-provoking number of perspectives.

Lilly talked about his role in the program:

To start, SSP has smartly invested in the future by providing resources for students and young professionals to attend the annual meeting. The fellowship program provided instant community: to walk through contemporary issues in scholarly communication with a diverse group of mentors and peers was energizing. As a student, my research revolves around the future of the monograph. ASU’s Center for Science and the Imagination offered discussion on potential transformations of the scholarly book. We “sprinted” down our thoughts using the collaborative authoring and publishing tool Overleaf. Some represented our conversation with multimedia; others penned poems. The end result was a multi-authored collection of essays, media and more.

In the panel “Transformative Publishing Platforms for Digital Scholarship in the Humanities,” university presses, largely funded by the Mellon Foundation, presented tools and platforms that aim to make open, networked, living (constantly editable, “always in beta”), digital monographs. I ran up to Susan Doerr from the University of Minnesota Press to discuss Manifold, which promises to think beyond static replications of print. I also found myself racing up to Dr. John Maxwell after he spoke about radical openness and networked books at the closing plenary, “Change is Already Here: Revolutionary Examples.” Dr. Maxwell, who is director of Simon Fraser University’s publishing program, promised to send materials on monographs.

Everyone I chased was accommodating and quick to give out their card. I expect many fruitful conversations to follow. So, the question may not be what did I learn, but what will I learn. Thanks to SSP’s generous fellowship program, I am certainly on a new, exciting path in my research.


Parts of this article originally appeared on the Scholarly Kitchen website on June 10, 2016.

ArtSciLab Paper and Performance Proposal Accepted for International Computer Music Conference

Scot Gresham-Lancaster,  a UT Dallas ATEC associate professor of sound design, has composed a piece and paper, “Culture of Fire” for Analog Neural Network Synthesizer, Geiger Muller Counters and Computer, which have been accepted for the  41st International Computer Music Conference hosted at the University of North Texas in Denton. The ICMC will take place September 25th to October 1st, 2015.

Below is an abstract:

“Culture of Fire” for Analog Neural Network Synthesizer, Geiger Muller Counters and Computer

Author:

Scot Gresham-Lancaster, UT Dallas

Abstract:

The “Culture of Fire” is an ongoing live performance piece constructed of residue from a project that was started by David Tudor and others to turn INTEL’s now defunct Electronically Trainable Analog Neural Net (ETANN) into a music synthesizer. A secondary layer of “control” in the live performance of the piece is that Geiger Muller Tube triggers are used as the source of actuation and location distribution of sonic events.

Roger Malina appointed to board of advisors for Sristhi Institute for Art, Design and Technology in India

Roger Malina was recently appointed to the international board of advisors for the Sristhi Institute for Art, Design and Technology in Bangalore, India, whose founding director is friend and colleague Geetha Narayanan. Geetha is an educator with over four decades of experience as a teacher, an educator, a curriculum and instruction designer. At all times a catalyst, Dr. Narayanan has tried over the years to evolve paradigms of learning that integrate the mind, body and consciousness and in the last few years has worked at creating collaborative pedagogical frameworks for the teaching of mathematics, science and languages within the Indian educational system at the informal and formal levels of schooling.

In particular, Dr. Malina will be helping with the new PhD program in Art and Design at Srishti, which is a recognized research center of Manipal University. Six scholars joined the PhD program in 2014, and nine more scholars from India and abroad are expected to join the program in 2015.

The advisors for the program are an interesting cross-section of the international art, design, and technology-practicing community. They include:

Joichi Ito, who is the Director of the MIT Media Lab and the Chairman of the Board of PureTech Health. He is also on the Board of Sony Corporation, The New York Times Company, The MacArthur Foundation, The Knight Foundation, and The Mozilla Foundation and the co-founder and board member of Digital Garage. He is a member of the Japanese Ministry of Education, Culture, Sports, Science and Technology, Center of Innovation (COI) STREAM governance committee.

Brandon Gien, who is the CEO of Good Design Australia and current President of the International Council of Societies of Industrial Design (ICSID), the world organization for Industrial Design. Through his work with ICSID, he co-founded the World Design Impact Prize to honor design-driven projects that make a positive impact on our social, economic, cultural and environmental quality of life.

Ezio Manzini, who has been working in the field of design for sustainability. Most recently, his interests have focused on social innovation, considered a major driver of sustainable changes. With this perspective, he started DESIS: an international network of schools of design specifically active in the field of design for social innovation and sustainability.

John Newbigin, who is Chairman of Creative England and of Cinema Arts Network. As Special Advisor to the Minister for Culture, Rt Hon Chris Smith MP, he was closely involved in developing the UK government’s first policies for the creative industries in the 1990s. He was Head of Corporate Relations for Channel 4 Television and executive assistant to Lord Puttnam as the Chairman of the film company Enigma Productions Ltd. As a policy advisor to the Leader of the Labour Party and Leader of the Opposition, Rt Hon Neil Kinnock, MP, he had responsibility for environmental and cultural issues, amongst others.

Soon-In Lee, who has been a graduate school lecturer in design management. He was also Dean of International Design School for Advanced Studies of Hong-ik University. Soon-In Lee was President (2011-2013) of the International Council of Societies of Industrial Design (ICSID), Chair of the Korea Design & Brand Management Society, and General director for Gwangju Design Biennale 2007. Since 2003, he has been the Chair of Asia Design Network and Korea 3D Printing Culture Forum. At present, Soon-In Lee is Executive Managing Director of Seoul Design Center.

Performed Data from One Antarctic Night to be featured in soundtrack for BBC’s Sound of Space

Sounds from One Antarctic Night, a series of interactive artworks created from 287,800 images of the night sky, are to be featured in the soundtrack for the BBC World Service series The Sound of Space.

One Antarctic Night, led by a creative team including Ruth West, Roger Malina, Lifan Wang, Scot Gresham-Lancaster, Alejandro Borsani, Andrew Blanton, and Brian Merlo, used data from the robotic telescope CSTAR in Antarctica and created electronic instruments that participants could interact with to make digital image and sound remixes. The resulting project sought to blend art and science in new ways, and has been featured in venues including the San Francisco Exploratorium, the New York Hall of Science, and the Rubin Center.

The Sound of Space, which features a tour of the Universe using audio that has either been recorded by probes such as Cassini-Huygens, or sonified from data gathered by spacecraft and telescopes, will have a soundtrack that includes artworks inspired by space. The soundtrack will include 5 sonifications from One Antarctic Night, including Deep Choir and White Noise Universe. Other audio in the soundtrack will include works from Caroline Devine, Louis Dandrel, Radioqualia, Sigur Ros and Semiconductor.

This project can be found under the ArtSciLab Projects page for more details.

ArtSciLab Paper by ATEC PhD Candidate Accepted by Union College Symposium on Engineering and Liberal Education

An ArtSciLab paper by Kathryn Evans, Senior Lecturer in Music and ATEC PhD Candidate at UT Dallas, has been accepted for the Union College “Engineering and the Liberal Education” symposium in Schenectedy, NY.

Below is an abstract:


 

“Does studying music enhance higher order learning skills in undergraduate non-music majors?”

Authors:

Kathryn Evans, Senior Lecturer in Music, School of Arts and Humanities, Frank Dufour, Associate Professor, Rosanna Guadagno, Associate Professor and Roger Malina, Professor, Arts and Technology, The University of Texas at Dallas

Abstract:

Many studies have looked at the correlation between music study and academic skills. A review of over 11,000 studies between 1950 and 1990 conducted by Harvard Project Zero tested the claim that studying the arts leads to some form of academic improvement. Only three areas were found that demonstrated a clear causal link between education in an art form and achievement in a non-arts, academic area. Two were in music: a medium-size causal relationship between listening to music and spatial-temporal reasoning and a large causal relationship between learning to make music and spatial-temporal reasoning. (Winner 2001). The majority of these studies have been conducted with students in primary and secondary education, but little research has been done on students at the undergraduate college level who study music, either as a minor or for general interest. Most pedagogical studies in music address the needs of music majors and not non-majors.

This pilot study looked at students at the University of Texas at Dallas who enrolled in music studies (either music performance, music theory or sound design) who are not majoring in music. Many are students in the STEM (Science, Technology, Engineering and Mathematics) areas. Through phenomenological research methods, we looked at the experiences of students who study music or sound design and how they perceived it affects their academic skills in other areas. Emails for students currently enrolled in music or sound design courses in AY2014-15 were obtained from registration records and they were solicited to take an on-line survey, with an option to volunteer for an in-depth interview. Over 800 students were solicited in February 2015 and a response rate of 20% has already been obtained. Additionally, over 30 students have volunteered for the interview. Initial data and a preliminary analysis will be presented.

DataRemix Paper Live on Leonardo Just Accepted Page

“DataRemix: Designing the Datamade,”is now on the Leonardo Just Accepted page, hosted by MIT Press. The paper was a part of the Special Section of Leonardo Transactions “Highlights from VISAP’13”, and was previously announced by the ArtSciLab as presented at the 2013 IEEE VIS Arts Program (VISAP) in Atlanta, Georgia. The authors of the paper were Ruth West, Roger Malina, John Lewis, Scot Gresham-Lancaster, Alejandro Borsani, Brian Merlo, and Lifan Wang.

The full paper can be downloaded here: DataRemix: Designing The Data made Through ArtScience Collaboration

The article is forthcoming in the Leonardo print publication, and can be cited with the DOI: 10.1162/LEON_a_01060.

ArtSciLab Paper Accepted for Understanding Visual Music 2016, Brazil

An ArtSciLab paper by Andrew Blanton, Connectome Data Dramatisation: The Human Brain as Visual Music, has been accepted for Understanding Visual Music to be held June 10, 2015 in Brazil.

Below is an abstract:


 

Connectome Data Dramatisation: The human brain as visual music.

Authors: Andrew Blanton, MFA; Sruthi Ayloo, MS; Micaela Chan, MS; Scot David GreshamLancaster, MA, MFA; Roger Malina, PhD; Tim Perkis; Neil Savalia, BA; Maximilian Schich, PhD; Anvit Srivastav, MS; Gagan Wig, PhD

Abstract

We, as a collaboration of scientists and artists, have built a visual and sonic representation of highly connected areas in the human brain. This model was developed to not only be a tool of scientific research but also as a tool for art creation. In the process of developing the software, the tool was built to interface with musical instruments for real time visualization and sonification. Working conceptually with the idea that scientific data can be repurposed for art creation, the Connectome is performed as both a sonic and visual representation of fMRI data, manipulating the model in real time as a form of multimodal data dramatisation.

Introduction

Partnerships between artist and scientist allow for creative forms of collaboration that can push both scientific and artistic research. With the Connectome Data Dramatisation project, our principal interest was in the creation of a hybridized tool, one that could work as both scientific instrument as well as artistic work. Beginning with a dataset that consisted of 441 neural bundles or nodes systematically differentiated into 21 areas or systems of interest in the human brain based on fMRI data collected by one of us (Gagan Wing) as part of the work of the UTDallas Cognitive Neuroimaging Lab.[1]

Area Centers Coded by System Membership

Our team was able to extract visual and sonic representations of the connections between those areas using custom software. We then developed that representation further in the form of an interactive three dimensional node edge graph and sonification of the 421 highly connected areas of the brain (in the case of the visualization, the width of the edges).

This would form the basis of the representation. With the addition of the ability to activate nodes from external data feeds via Open Sound Control[2] different nodes could be excited at will creating a virtual, three dimensional instrument that could be used for visual and sonic performance. Using four small drums, the visual and sonic representation of connections between areas of the brain can be played in real time. Custom software receives input in the form of audio signal from each drum and excites specific areas of the brain. Each section of the brain that is played will present a unique visual and sonic representation.

Historical Perspective

Building on previous explorations in bridging art and science through the development of new technology, we were actively looking to understand how this project is situated within the history of visual music. In looking at the work done at Bell Labs in the 60’s and 70’s[3] and with the work of artist such as James Whitney[4], the question emerges, what are the components of a successful art and science collaboration? How do separate practitioners collaborate while furthering each of their own research? Phill Mortin and Dan Sandin’s image processing units[5] also played a role in both the conceptual development as well as the technical development of the work. How is information shared and disseminated after it’s creation? Other contemporary artist were looked at as well including the work of Noisefold[6] in their sound extraction techniques form visual information, Ryoji Ikeda[7] in his visual and sonic representation of data as well as Semiconductor[8] in their blending of art and science amongst others working with visual music as a contemporary practice.

Visual music has been historically tied to the development of technology. This holds true now as much as it has in the past. Current rendering technologies are evolving rapidly within the gaming community and practitioners of visual music are greatly benefiting from real time rendering advancements within the gaming communities. Robust community support and the indie gaming movement have provided new tools for interfacing with gaming environments[9]. Two areas that are underdeveloped with regard to these environments and practitioners of visual music can provide insight are in the development of procedural animation, and the assimilation of data into these environments. With this project we have begun to build a framework that can both provide a series of procedural animations with regard to node edge graphs as well as interface a gaming environment with a dataset of approximately 77,000 connections. In doing so we have tried to maintain the work as both a piece of art and a scientific instrument.

Future Work

In the process of building this project, we have worked with many technologies to find the right combination of frameworks and development to allow for extensive flexibility in artistic representation of the data set. We have worked with Max/MSP Jitter[10], Unity[11], Syphon[12], Three.js[13], node.js[14] midi.js[15], coffee collider[16] and D3.js[17] in a exploration to find what technology would serve the representation of this dataset best. Beginning with a representation using three.js hosted on a node.js server we were able to bring in live data via OSC to trigger the model. We found ultimately that building everything in the web browser provided great accessibility for global use of the tool, however, confining the project to the web browser also creates limitations with regard to power for rendering and audio synthesis. We have built a framework that now uses the Unity game development environment specifically for it’s strength with regard to real time rendering and are working on integration of Pure Data[18] via the Kilimba Unity extension[19]. This process will allow us to build a platform addressing the two primary areas of dataset integration into gaming environments and procedural manipulation as well as sonification and visualization of said dataset.

Summation of Findings

The creation of the Connectome project has led to some interesting further work in collaborations between artist and scientist. Beginning with the fundamental question can scientific instruments be used as tools for art creation and can artist tools produce scientifically valid results, our team was working to further a dialogue between artist and scientist while creating real value for each party involved. In doing so we have opened up another path of exploration in the form of using game development platforms for data visualization and sonification as well as the reappropriation of these platforms for use in real time audio visual work. By creating a core representation, we were able to build a model that could be manipulated in real time using incoming Open Sound Control data and provide a scientifically accurate representation of the underlying dataset.


[1] Area of interest in this case were areas of concentration of neurons in the brain as identified by researchers at of the Center of Vital Longevity Cognitive Neuroimaging Lab at the University of Texas at Dallas. http://vitallongevity.utdallas.edu/cnl/ accessed march 7 2015.
[2] http://opensoundcontrol.org/ accessed March 7 2015
[3]http://www.ieeeghn.org/wiki/index.php/Archives:Bell_Labs_%26_The_Origins_of_the_Multimedia_Artist accessed March 7 2015
[4] William Moritz on James Whitney’s Yantra and Lapis http://www.centerforvisualmusic.org/WMyantra.htm accessed March 7 2015
[5] Museum of Modern Art https://www.moma.org/momaorg/shared/pdfs/docs/press_archives/5958/releases/MOMA_1982_0014_14.pdf?2010 accessed March 7 2015
[6] http://noisefold.com/
[7] http://press.web.cern.ch/press-releases/2014/01/japanese-artist-ryoji-ikeda-wins-third-prix-ars-electronica-collide-cern
[8] http://semiconductorfilms.com/
[9] http://pjim.newschool.edu/issues/2011/01/pdfs/ParsonsJournalForInformationMapping_Medler-Ben+Magerko-Brian.pdf
[10] https://cycling74.com/ accessed March 7 2015
[11] http://unity3d.com/5 accessed March 7 2015
[12] http://syphon.v002.info/ accessed March 7 2015
[13] http://threejs.org/ accessed March 7 2015
[14] https://nodejs.org/ accessed March 7 2015
[15] http://mudcu.be/midi-js/ accessed March 7 2015
[16] https://github.com/mohayonao/CoffeeCollider/wiki accessed March 7 2015
[17] http://d3js.org/ accessed March 7 2015
[18] http://puredata.info/ accessed March 7 2015
[19] https://github.com/hagish/kalimba accessed March 7 2015