Gray Literature

Publications

Research

machine listening

Director

Roger F. Malina is a space scientist and astronomer, with a specialty in extreme and ultraviolet astronomy, space instrumentation and optics. He served as director of the Observatoire Astronomique de Marseille Provence and was NASA Principal Investigator for the Extreme Ultraviolet Satellite project at the University of California, Berkeley.

He is also a publisher and editor in the new emerging research fields that connect the sciences and engineering to the arts, design and humanities. Since 1982, he has served as Executive Editor of the Leonardo Publications at MIT Press. He founded, and serves on the board of two nonprofits, ISAST in San Francisco and OLATS in Paris, which advocate and document the work of artists involved in contemporary science and technology.

He is currently a Distinguished Professor of Art and Technology and Professor of Physics, at the University of Texas at Dallas and Directeur de Recherche for the CNRS in France. He serves as the Associate Director of ATEC, and founded the ArtSciLab in the ATEC program fall 2013.

Roger Malina

Colleagues:



Under the leadership of ASL member Scot Gresham Lancaster, Sharath Chandra Ram and I have just submitted a proposal to the US National Science foundation: 



AstroSounds: Developing spectrally based multivariate time series data sonification player for Astrophysics 



Overview:



We are going to use auditory cognition techniques of speech perception for the listening that astrophysicist would be asked to do with a spectrally based multivariate time series data player.



LISTENING TO DATA (Sonification) is a new tool needed for analyzing the ever growing data cloud. Our work is perfecting the many ways to make better decisions by not just visualizing, but also LISTENING TO DATAs This has lead us to helping others in many fields, neuroscience, business, general physics etc. but now we are returning our attention back to astrophysics and the advantages that the integration of audio brings to the data science interfaces that are such an integral part of astrophysical research.



We have developed a new approach to classifying the techniques of data to audio conversion and the specific types of applications that can be offered using the advanced techniques available to top digital audio experts to realize those conversions. Collaborating with our colleagues at UC Berkeley, Stanford and other institutions, we have been sharing our ideas and approaches to the sonic realizations that highlight features blurred or misrepresented in visualizations. We have exposed artifacts occluded within only imaged representations of data. Also we can simultaneously represent many other dimensions not visually represented in the complex time varying graphs. Data visualization tools have successfully been augmented to allow more rapid and accurate decision making in the context of ever growing data sets.



We continue to create training modules to learn to listen, explore and pass testing of sonifed data to interpret data as information via ongoing cognitive testing. Working from our existing, tested and published "fuzzy" taxonomy of classifications and techniques we will use 2nd order sonification that remaps information systematically via spectral analysis and auditory spectrum resynthesis of time series data. Additionally, we will use our machine listening techniques to find spectral similarities in the overwhelmingly large amounts of available data in the existing open archives. Using new techniques of binaural audio augmented reality, we will allow researchers to move through the soundscape created by time varying multidimensional data sets. This is based on our existing and cognitively tested earlier work in this area. 



Broader Impacts:



The adoption of sound in the market of Smart Speakers (Alexa, Google Home, etc.) and Audio AR (Bose, Onset, etc. ) have shown the growing more dominant place for audio oriented design. For the first time we and leading with astrophysics, we will move to scale across all areas, that are becoming dependent on data science techniques and outcomes. The perception of multidimensional auditory information will lead to new discoveries that would not be made by visual means alone.

Do you want a "Data Sonification Player" for Astrophysical Data or your own data?

roger-malina

The CDASH (Curriculum Development in the Arts, Sciences and Humanities) website was established in 2012 by Kathryn Evans and Roger Malina as both a resource for faculty who engaged in or were interested in engaging in art-science-humanities curriculum; and as a data collection point where these types of curriculum could be surveyed for innovation and sustainability. Our new site will include graduate, undergraduate, primary/secondary and informal curriculum.

Our new website allows you to easily enter your course and all the relevant details. Please log in or register before you submit your course. You may also access the other areas of the site. Those who submit syllabi will be admitted to the Cloud Curriculum, where you can download other syllabi and resources.

For more information, contact <a href="mailto:cdash@utdallas.edu">cdash@utdallas.edu</a>.

CDASH

Experimental Publishing and Curation

This Esport Cyberathlete Development project launched by the University of Texas at Dallas’ ArtSciLab aims to contribute to design one of the first new sports of the 21 st century.

Esport Cyberathlete Development

Experimental Publishing and Curation Assistant

Maisha is an Applied Cognition and Neuroscience Graduate student at UT Dallas where she works for Dr. Basak’s Lifespan Neuroscience and Cognition lab. She’s passionate about writing and talking about behavior, cognition, and tech. She hopes to use her blended background in research and writing as well as her work at the ArtSciLab to help further science communication among diverse audiences.

Maisha Razzaque

Managing Editor

Chaz is interested in alternative modes of scholarship and publishing. As a PhD student in the ArtSciLab, he is designing experiments that tease out new ways of knowledge production, curation and dissemination.

Chaz Lilly

chaz-lily-300x300

<h1 style="text-align: center">Arteca</h1>
ARTECA is a digital space that supports scholarly materials that explore the intersection of arts, humanities, science, and technology. The portal serves as a virtual art-science community resource and critical content provider for scholarly and experimental publications, including gray literature.

The project is a collaboration of The MIT Press, Leonardo/ISAST, and the ArtSciLab at UT Dallas.

ARTECA acts as a valuable hub for academic librarians and researchers, and aids in raising the art, humanities, science, and technology community’s visibility when other monodisciplines have already established such services.

ARTECA

Deputy Co-Director of Transdisciplinary Strategy & Innovation

Thanks to more than a decade of experience working in the realm of film entertainment (ranging from film festival management to independent film production) and in the entrepreneurial world (in the tech, non-profit, and media sectors), Garcia Topete's research and practice cover and converge in three main areas under the broader scopes of Art-Science and Experimental Publishing of the ArtSciLab: transdisciplinary intelligence, innovation and collaboration theory, filmmaking research, and public engagement design.

Alex Garcia Topete

alex-garcia-topete-1-450x450

Project HERMES looks at how researchers and expert practitioners work together in interdisciplinary (across institutional departments) and transdisciplinary (across stakeholder communities) contexts through the framework of Transdisciplinary Intelligence. The framework includes knowledge and aptitude metrics, good practices, and transdisciplinary training modules to improve collaboration among artists, designers, engineers, scientists, and other experts in order to boost innovation through tool sharing and idea exchange. Project HERMES is currently underway internally at the University of Texas at Dallas among its 100+ research groups and will expand to industry partners in 2020. 

HERMES

A collection of multidisciplinary exemplars of science, engineering, art, design, and humanities.

The SEAD Exemplars exhibit will comprises a collection of exemplar works demonstrating effective intersections among the sciences and engineering with art, design, and the humanities. The works represent international diversity as well as diversity of practices and disciplines. The sections of the exhibit reflect the different purposes and impact of the exemplars:
<ul>
 	<li>Pioneering: Individuals and groups whose careers have transcended disciplines and paradigms.</li>
 	<li>Exploring: Projects that have pushed the boundaries of both arts and sciences.</li>
 	<li>Bridging: Projects that seamlessly connect and merge the arts and the sciences.</li>
 	<li>Engaging: Projects that seek to express and communicate to the public at large.</li>
 	<li>Questioning: Projects that ponder about the implications of arts and sciences themselves.</li>
 	<li>Educating: Projects meant to both teach and encourage learning.</li>
 	<li>Innovating: Projects that led to discoveries and inventions.</li>
</ul>

SEAD Exemplars

ArtSciLab Colleagues:



I would like to draw your attention to the Recent YASMIN WWWASP discussion (the Who, what where of our artscience community of practice)- Led by Guillermo Munoz with several colleagues across the world (see at end of this post). The YASMIN list is an international list of artscience professionals and amateurs and observers.



The discussion is now open to any colleague on the Tasmin list- we look forward to your thoughts: If you aren't on <a href="https://ntlab.gr/mailman/listinfo/yasmin_discussions_ntlab.gr">YASMIN</a>: 



I just posted this to YASMIN



"I would like to pick up on Salome Cuesta proposal for ‘micro-actions’ in our art-science communities of practice, and Diamond Berverly’s idea of a ‘continous conundrum (see below).



After 30 years as Executive Editor of Leonardo, for the first time I am now deeply engaged as a practitioner in art science collaborations. I like
 to joke that astrophysics was so easy ! We all agreed on the success 
criteria and used the same concepts, methods and terminology. Plus we 
were very well funded in comparison with artscience research.



 Thanks to Guillermo Munoz and colleagues at the University of Valencia I received a PhD in Art, so I am now an ArtScience ‘postdoc’ with a PhD in astrophysics and a PhD in art !



As James Leach pointed out in his Leonardo article <a href="https://cnrs.academia.edu/JamesLeach">(Extending Contexts, Making Possibilities: An Introduction to Evaluating the Projects - James Leach)</a> many many art science collaborations ‘fail’ in achieving their original objectives.



In our ArtSciLab we have been arguing that there is a very big step between inter/multi disciplinary collaborations and ‘trans’ disciplinary collaborations which bridge very very different disciplines with very very different personal and collective success criteria, and very very different often contradictory methods, concepts , terminologies-



These are in Diamond Berverly’s term ‘conundral;- see Merriam Webster



“The exact origin of conundrum isn’t known with certainty. What is known is that the word has been in use since the early 1600s, and that it had various spellings, such as conimbrum, quonundrum, conuncrum, and quadundrum, before the current spelling was finally established sometime in the mid-17th century. One theory of origin suggests that the word was coined as a parody of Latin by students at Oxford University, where it appears to have enjoyed particular popularity in its “word play” or “pun” sense. While the prevalent sense in this century is that of the seemingly unanswerable question or problem, frequently applied to heady dilemmas involving ethics, sociology, or economics, the word is sometimes so loosely applied to anything enigmatic as to be synonymous with puzzle or mystery.”



One of the strategies we have been 
trying is “micro-projects” in Cuesta’s terminology- we define them as 
short duration ( weeks), projects which require no money or cash, only 
gift exchange of time and access to facilities, expertise etc ( the gift
 exchange vocabulary comes from James Leach )



In the ArtSciLab we are approached continuously by ‘stem’ professionals who want to collaborate. Interestingly these range from physical sciences and engineering to the brain and Behavioural sciences to business and social sciences…



Often the stem professionals who reach out to us, “they” have antiquated ideas of what making art involves or results in today. Often they view art as less primary in the ‘tree of knowledge” (i.e. in a branch not in the trunk). 



Often they have very different methods , concepts, terminology (we are trying to develop transdisciplinary apprenticeships as part of the approach). Mauricio Mejia at ASU, Alex Topete, colleagues, and I are currently submitting a workshop proposal on applying translation studies methods to transdisciplinary collaborations.



If a micro-project doesn't succeed, its likely that a significant project won't in our experience.



I am delighted that a student research in our ArtSciLab , Diamond Beverly, has proposed: “how do we go about including diverse voices and fostering heterogeneous approaches instead unconsciously excluding people from the conversation and thus creating a continuous conundrum.” which highlights the implicit bias of the artscience community towards people in academia with its ethnic, gender, socio-economic and other implicit social biases.



Maybe the non -academic and younger members of Yasmin should weigh in, just as from Diamond Berverly: I was very intrigued with your post last week when you emphasized educational spaces and work shop methodologies. I would like to know how you define micro-actions. I also find such maker spaces and hackathons a good step into the future of collaborative educational space. A question that persists however is how these spaces find their audience?



And by this I mean how do we go about including diverse voices and fostering heterogeneous approaches instead unconsciously excluding people from the conversation and thus creating a continuous conundrum.



So all Latin students on Yasmin go at it : "One theory of origin of ‘conundrum” suggests that the word was coined as a parody of Latin by students at Oxford University, where it appears to have enjoyed particular popularity in its “word play” or “pun” sense. While the prevalent sense in this century is that of the seemingly unanswerable question or problem, frequently applied to heady dilemmas involving ethics, sociology, or economics, the word is sometimes so loosely applied to anything enigmatic as to be synonymous with puzzle or mystery."



Or maybe living in a continuous conundrum is a desirable state ? Certainly it is a state that drives our ArtScience work and is generative of desired outcomes.



Roger Malina



P.S These are the invited discussants:



Jadwiga
 Charzynska: I am the director of LAZNIA CCA in Gdansk since 2004 and in
 our program, one of the most important points is the Art + Science 
Meeting project, which we organize regularly from 2011. It’s really 
unique program in Poland presented Art / Sci projects. Links: <a href="http://www.laznia.pl/instytucja/" target="_blank" rel="noreferrer noopener">http://www.laznia.pl/instytucja/</a>



Joao
 Silveira: Brazilian entrepreneur, choreographer and pharmacist. Was a 
Harvard Fellow (Le Laboratoire/ 2017-2019) and currently is a research 
fellow at the ArtSciLab – UT Dallas, and CienciArte Lab – Fiocruz. Jing
 Chen: Associate professor of Arts and associate director of Art and 
Cultural Innovation and Creativity Lab of Nanjing University. Gustavo
 Ariel Schwartz.: Physicist and writer. Scientist of the Spanish 
National Research Council (CSIC) at the Materials Physics Center. 
Founder director of the Mestizajes Program, at the Donostia 
International Physics Center, whose purpose is to explore and cross the 
borders among Art, Science, Literature and Humanities.

art-science

artscience

Micro ArtScience Actions (MASA)

<figure class="wp-block-embed-youtube wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-4-3 wp-has-aspect-ratio"><div class="wp-block-embed__wrapper">
https://youtu.be/lJlBJ-NagLM
</div></figure>



<a href="https://medium.com/@looplogic/halloween-transmissions-machine-listening-from-the-diary-of-ana%C3%AFs-nin-276f67d71068?sk=336e1b3958336f3ee0952e0bbbb3d4f5">Full article @ Medium.</a>

Halloween Transmissions: Machine Listening from the Diary of Anaïs Nin