This week I haven't done much to show visually, but I've made the beginning preparations of my slides for the workshop and the beginning of my written portion of the thesis.
On the workshop side of things: For the slides I'm just using the same template I made at the beginning of the semester, and will use up until the oral defense, since I already have it and for the sake of time and consistency I'll just keep using it; no sense in making more work for myself than necessary at this point. In planning I've gone back to the outlined structure and will be mostly sticking to it with some minor edits: starting with an overview of AI and how its impacting the world and the field, showing some creative examples from this thesis, then getting into the setup of the LoRa and integrating it with TouchDesigner, then allowing for creative freedom and exploration, coming back at the end for reflection and presentation of each others work. From the committee meeting feedback and thinking on the workshop itself a little more, I think the importance here is to educate and demonstrate on methods for being creative and designing with AI.
It's important for a lot of reasons: not limited to, but a highlight being the ramifications it has on the field, and from my own experiences working with clients in this new age of AI, it's important for designers to be knowledgeable of its strengths and limits, and how to work with it better and more intentionally than non-designers prompting and pressing buttons. Especially for undergrads looking for jobs, the workshop can serve as a means to democratize AI tools to them, and to give them the ability to make it more of their own thing as opposed to, again, what a client or firm can expect or execute from standard and widespread models, which is where the LoRa's and TouchDesigner come into play.
On the side of the written portion, I've got an outline of it, and this week so far I've been gathering the papers for the literature review and references sections, and I've been reviewing the guidelines for formatting, and looking at last years graphic design graduate papers (such as Ernests and Ugos) for reference since (shocker!) I find it more helpful to see visual examples, and with the updated guidelines into 2024-25 and ISU's websites less than helpful descriptions of them, it can be kind of confusing without looking at exactly what's kind of expected. Not that formatting will be a huge lift but I'd rather know before I get into the grind of actually writing it.
What I’ll be Doing Over Winter Break
This won't be any new news from the final class presentations and my committee meeting, but during winter break, my focus is on laying the groundwork that everything else will build on in the spring. The two main priorities again being workshop planning and making meaningful progress on the written portion of the thesis. Rather than trying to execute major public-facing components during this time like the exhibition, I’m using the break to slow things down strategically and make sure the structure, goals, and logistics of the workshop are clear and well thought out. A key part of this is doing a test run of the workshop with a small, low-stakes audience (my family), which will allow me to refine pacing, clarify instructions, and troubleshoot any technical or conceptual issues before running the official session early in the semester. I’m also using this time to identify and order any remaining materials needed for the workshop and exhibition, and if I do end up having more time, to continue refining the installations so they’re more stable and resolved heading into spring.
Again on the written side of things, winter break marks the beginning of the final push on the written thesis. My goal here is to establish a strong foundation by solidifying the literature review and introduction, making sure my research questions, theoretical framing, and methodological approach are articulated well before workshops and exhibitions demand more attention. By the end of winter break, I want to enter the spring semester with a clear plan and slides for the workshop, a partially drafted written thesis, and fewer open questions so that the rest of the timeline can focus on execution, analysis, refinement, and defense preparation rather than foundational planning.
Semester Reflections
This semester flew by! As I'm sure it does for all 3rd year grad students. I'm really satisfied with my work and progress I've made over the semester. From the beginning presentation I was real hard on myself and worried about how it would turn out But over time with research, work, and planning, all the pieces became more and more clear, and I was able to put it together into a package that I believe makes sense and is important and meaningful. My opinions and thoughts on AI are always in flux, and will probably remain so for long after the final oral defense, but in the sense of research and academia as well as public knowledge it is important to inform people of the multifaceted aspects of AI as much as possible, so they themselves can decide what it's best used for and how to use it effectively and ethically, not through word of mouth of CEOs and demos, or social media rants. The hype/hate cycle is real, as is a potential bubble ready to pop. As I said in my presentations, the future is uncertain, but it's up to the people to decide if these tools should be adopted or rejected, and in what ways they should go about either, with only the informed mind being able to make the best decision.
On a process side of things, I also gained a much clearer sense of how making and research inform each other in this process. The installations, workshop planning, and technical experimentation aren't separate from the writing or theory, more so being actively shaping how I think about AI, interaction, and learning. Seeing ideas succeed, fail, or behave unexpectedly in practice reinforced to me why hands-on engagement is such a critical part of understanding AI systems, and why this work is important. That balance between thinking, building, and reflecting is something I’ve become much more comfortable with this semester, and it’s given me confidence moving forward. While there is still a lot to do, I feel like I’m entering the final stretch with a strong foundation, a clearer direction, and a project that aligns with both my academic goals and my personal values as a designer and researcher. Lastly, I just want to say thank you to the graphic design faculty, both in my committee and out, and my graduate student colleagues (even if they all won't see this) who gave feedback on my presentations and work to help make this thesis what it is and what it will be, and thank you to Alex, who is certainly a huge part in helping in all steps along the way as my Major Professor. It's not over, and there is still a lot of work to be done, but I am now more than ever confident and ready to finish strong.
This week I have a bit of progress to cover from last week due to missing the post deadline (oops!), as well as having a full committee meeting to discuss my thesis with the committee members. I'll start with the 2 projects I made a lot of leeway on, and what I still need to do for them, then close the post with summarizing and reflecting on the committee meeting.
How Original
On the how original project, a lot of progress was made in the logic and performance of the interaction itself. Every iteration is a wittling down of errors and increasing the fidelity of the matching system, making it more and more accurate to the live users poses. With this too there is a clear, pretty much causal relationship between the quality of the dataset with the quality of the match-up performance, where at this point in the project the thing that needs to be focused on more is the dataset: gathering a great deal more images (aiming for a couple thousand) to allow breathing room for all the possible poses, and accounting for the reality that lots of historical photos are low resolution, sometimes don't include a full body, and can be hard for machine vision to accurately find all the joints on a body from these low-resolution photos with variables like lighting and multiple people, among other things, that can confuse a computer. I've basically automated the process of finding joints, but I wonder if it would be necessary to make some sort of manual process for me to place joints on a photo when the automation goes wrong.
Malicious Sycophancy
In contrast to the sort of collective connection and experience with humanity as a whole in "How Original", this "Malicious Sycophancy" project takes the opposite direction in creating an experience that uses a users brainwaves to create an over-generalization in the guise of a solely individual experience.
This project centers a more intimate—and potentially harmful—dynamic: how AI can manipulate perception and mental health. Cases of “GPT psychosis” are growing, with AI gaslighting people into believing they’ve unlocked universe-altering equations or other delusions. This risk is amplified by AI sycophancy, where models over-agree to maintain engagement and positive experience, even at the expense of user well-being—now potentially affecting nearly half of adults and a majority of teens who report using AI for emotional support.
Drawing from things like rorshach tests and astrology, these things that are not valid and considered sort of pseudo science still impact peoples lives and decision making, with AI now being added to this list. In this instance, the user puts on an EEG monitor that in real time scans their brainwaves, which are mapped to what “emotions” they are feeling. That data and these emotions are fed into a custom chatGPT instance, which gives them a totally unreliable psychological analysis. Every aspect of this is initiated when a user makes the choice to put on the headband, with the visuals all being drawn from that data.
Visually speaking, this one has the most work to be done stylistically, where (like Alex said in the meeting), exploring things like color, motion, type, etc. will be important to consider as this takes more shape over time, specifically adding more distinction in the output / takeaway to give it some variability compared to the other projects. This could be highlighting the chat output? Adding an audio element? Things to consider, but importantly the backend interactions are solid and I have a good understanding of how the brainwaves appear on screen and how that data can be manipulated visually.
Committee Meeting
Rounding out this post, I had my first, very late, committee meeting today, that I think went very well. On a spectrum of "the entire thesis unravelling and falling apart" to "no changes see you on your oral defense", the sense I got is that the direction is good, it makes sense and is clear what I am doing next, and covered some of the lingering questions I still had. We covered some good ground on what research looks like in these circumstances with the workshop, the newly defined expert interviews, and beginning to plan out in more detail what the exhibition looks like and how to lay it out, in the sense of a physical space and the details about it.
On the interviews, I initially thought that I had to get them through IRB to be anonymized, but Alex made it clear that I can use them much more openly, being able to quote them and actually show their work to make some of my points and positions clearer to the thesis audience, which is definitely very helpful for me.
Second, the framing of the workshop, where the actual research of the data being generated by participants is taking less of a main-stay, while the focus on what I'll actually present for my thesis is going over more of how it went, an assessment of the approach, and covering what worked, what didn't, and how future iterations could be improved upon, which is much clearer to me in how I'll go about planning the workshop over winter break.
We also talked about some considerations with the exhibition, planning out the space, thinking about how certain things will be presented, where they will be placed, considering where backdrops can / should go in order to limit the amount of people that could potenitally interfere with the motion tracking, etc. This I'll cover more in the final reflections / next steps post as I'm writing this pretty much immediately after the meeting, so with a little more time to think and reflect I can be much more perscriptive of the next steps.
Next Steps
In this sort of transitioning period between design finals week and time before break, I want to have a gathering and reflection of everything I've done and what I will do. I have the workshop outline so I know at a high level what structure that will sort of look like, and I have an outline of my written portion, that while I haven't looked at it for a little while, is more of a starting point than from scratch. This will also look at all the materials I'll need for both the exhibition and the workshop, things like cameras, cables, mats / tape and monitors for the exhibition, and usb devices for the workshop for people to take their projects with them. Again, I'll expand on this more for next weeks post, but the synthesis is I have a game plan that I am ready to act on for winter break and into spring.
This week I made progress on 2 of the installations, and made some supporting images for the exhibition itself.
The first installation I made a lot of progress on was Non-Sentences, in which I got the thermal receipt printer all hooked up and talking with TouchDesigner, so that every sentence it generates pings to the Arduino so it gets printed. This took a little bit of head banging within TouchDesigner and with the Arduino IDE, but eventually I got it to work. The main thing was the right chip settings, which is obvious in hindsight but with the chip Alex giving me to borrow a Diecimila instead of an Uno which I was more familiar with, just a couple changing of settings was good to get it talking to TD and to get the sentences put out on the printer. The good thing was that within TouchDesigner I already had a module that spit out the full sentence in one go, which was left over from before I had the console type-out effect of each character appearing one by one. Having a little script that communicated with a Serial DAT in TD lets it talk to the printer, and with a slight modification of the print layout for installation purposes, it was good to go.
I think the printer adds a lot. As I said in a previous post it adds a more tangent physical aspect to the digital words that are output by this LLM, and with the huge line that I already managed to print that reached my floor from my apartments counter, I can see the vision of it piling up a lot more over time. As far as the words on the paper itself, in the instance I was running it the sentences were backing up on themselves a little bit, with words and characters being a little re-arranged in just a bit more of a non-sense random way. I think this can unintentionally add a bit to the "Non-Sense" idea of the record keeping of these AI outputs, where the sense it makes becomes even more diluted, with even smaller strings of a couple words making sense. Overtime this adds up to this wall of text with no meaning besides what we make, akin to a never-ending chat with an LLM that can keep going forever and ever, losing that meaning over time. Visually, this also connects to the image/latent space ideas I brought up earlier this semester. A comparable visual is the "library of babel", which is a thought experiment / actual website comprised of theoretically every piece of text that can ever and will ever be written. Of course, most of it is nonsense characters in chapters of books of meaningless-ness, but technically everything is in there, from the cure to cancer to the exact details of the day you die, to my entire written portion of my thesis, already written for me. Think infinite monkey's on infinite typewriters across all of time and space. In a way, "Non-Sentences" is a physical slice of the library, one that I am really satisfied with how it has progressed.
Secondly is the "How Original?" project, which again involves pose tracking to match a historical photo with a persons pose from a live feed. For class this week I was really focused on getting a good image database of historical poses to feed into the media pipe pose detection scripts in TouchDesigner. This was much easier said than done, with lots of ups and downs in finding good sources of images, navigating online databases, and downloading the images to my database file. The good thing is that the backend system for this is mostly working, so I have would only need to plug in my images into the system and re-run it to see how accurate I can get it to a persons pose. As far as the images, again I was looking at all different databases with Creative Commons licenses, from the Library of Congress, Internet Archive, New York Times archive, British Library, and Stanfords image archive. I found a lot of good images on there, but in the end I found I might have been overthinking it a little bit, and found that simply looking on Google Images or Pinterest was better than combing through these archives, because I could find more useable images faster and in higher quality/quantity.
In the process of downloading these images I was thinking that in essence I was acting like an AI model, sourcing images all across the internet with or without permission to create this experience. As the experience connects us with our collective past and the inability to separate ourselves from it, the process is a connection with it as well, and a connection to how an AI model sources these images, but from a distinctly human angle. When I see an image of 1920's child laborers having fun, I get an emotional reaction, but I'm also analyzing it for their poses and how viable it would be for the pose detection system and if it a person interacting with the installation would strike that pose. All that to say, on the topic of AI literacy, the connective tissue of our past as and future of a species are closely intertwined, with many gray areas. These real people from the past aren't consenting to having their images or likeness used by the models or the training sets of encoding "vintage" aesthetic vibes, just as they aren't consenting to have their images stored forever in a Creative Commons database for the entire world to see. In a way they will live forever through these methods, and through this installation, but in a more socially acceptable way than if you encode grandmas likeness to be deepfaked so you can talk to her after she's passed. Begging questions of how these past connections strengthen or weaken our present connections with technology, and how we can learn to make these connections more responsible for the future.
At the time of writing I've finished gathering 300 distinct images for the project. I'll include some samples below, but my plans for this break are to plug them into the system to see how similar I can get them to my poses (originally achieving 0.2 or less similarity with the previous ML database), and to begin more in-depth the brainwave project. I have it talking to TD through OSC data, it's just a matter of laying it out and fine tuning some of the finicky nature of the actual reading of a persons brainwaves into a more smooth experience.
Once I've made progress on that, I will need to fine tune the rest of the installation pieces, and work on moving them from computer to computer, as there can be some complications that come with this, ultimately to end the semester with a clear designation of which of my computers (beefy Nvidia laptop, my 2 Macs, or the mini-windows one's Alex is letting me borrow) will host which projects for the gallery.
What I’ve Read
This week I was able to read a little more of "Transcending Imagination", and Chapter 9 had some interesting things about the myth of the "creative genius"
Manu dives into the belief that creativity is an innate gift possessed only by a select few “geniuses.” He argues that this myth, which emerged during the Romantic era and is reinforced through figures like Mozart, Leonardo Da Vinci, and Einstein, misrepresents the true nature of creativity as a collective and iterative process. He goes through several misconceptions: that talent is purely innate, that creativity arises from sudden “Eureka” moments, that geniuses create in isolation, that creativity is chaotic rather than structured, and that madness is a necessary companion to creative brilliance. Manu reframes creativity as a skill honed through dedication, discipline, collaboration, and resilience, rather than divine inspiration or individual genius.
The second half of the chapter expands this critique on "the creative genius" in light of AI and generative technologies. Manu suggests that tools like AI reveal creativity to be a social process rather than an individual miracle, as algorithms themselves build upon collective human data and knowledge. Rather than threatening creativity, AI democratizes it, lowering barriers for participation and encouraging collaboration across backgrounds and disciplines. By enabling more people to create and contribute, AI disperses the myth of the solitary genius and replaces it with a model of collaborative intelligence, where innovation emerges from networks of humans and machines working together. I think this makes sense given the current creative climate with social media and AI. Over the years there have been less and less of these creative geniuses popping up. You don't see someone captivating the world like a Michael Jackson anymore, the closest maybe being Taylor Swift or Kanye, and even then they have decades long careers at this point. I think the combination of social media and AI in a way allows for a sort of individualized natural selection of what people deem creative, where novelty and value can be mapped to views and likes, although there are certainly ways to game the system, the merit of something creative and captivating is allowed to at the same time be made an easier process, and to reach all around the world much quicker. In this way the "creative genius" of our time can be anyone, but remains a human thing. AI slop posts typically don't get much traction apart from the "boom" of a new model coming out, or hate from a select viewpoint. One of the practitioners I interviewed has said he's gotten next to no hate for using AI in his works, and he postulated it was because of the uniqueness and meaning behind the work that allowed for this, making his work break the typical AI visual language and the commentary that follows it.
Where the Next Steps are Leading
As I said previously, over the break I'll primarily work on the "How Original" pose tracking project for maybe 1/4 of my time, and the rest focused on pushing forward with the brainwave scanner, with the ultimate goal of finalizing the installations for the end of the semester, to work on planning the workshop and written portion of the interviews for winter break.
This week I made a lot of good progress in a bunch of the different installation projects, and made moves to getting a space to have them actually installed in Gallery 181 in the design building next to Carly's work, in that I actually have the gallery space reserved! I added a great deal of interactivity to the Emergent Garden, and made solid work on two somewhat smaller works that I think could be included as an interactive and introspective experience with AI, working titles being "How Original?" and "The Single Greatest Piece of Art That Has Ever, Can Ever, or Will Ever be Made" (a bit pretentious of a title but I'll add more context to what that actually means and entails below).
Emergent Garden
Like I said I added much more interactivity to the Emergent Garden, combining aspects of the drawing tool I made a while back with a gesture system that allows the user to draw shapes that are then interpreted to different flowers. The main "brush" is a simple circle with a feedback that keeps the previous frames visible, and is situated in the midpoint between a users index finger and thumb. By moving their hand around they can draw shapes on the canvas, and by pinching their fingers together or apart, they can change the size of the brush. To change the color, the user gives a thumbs up to the camera, which cycles the color of the brush from red, purple, blue, green, orange, and white. Finally, the user can clear the entire canvas by pointing their finger up, so that they don't have to worry about erasing every single shape they've made. I added pictogram instructions that will be fixed on screen over the camera feed, and the shapes they draw are made slightly transparent so they can still see their hands and themselves as they draw.
Right now the motion tracking takes into account only one hand, but soon I'd like to add the ability for more than one hand to be tracked to add a more collaborative aspect of interaction. Another issue I have with it right now is that the camera feed is at a 16:9 resolution, while the AI is only capable of outputting a 1:1 square resolution. This means that the actual size of the drawing canvas, or camera feed, is currently hidden behind the AI canvas. It's not much of a usability issue, but this means that some of the drawing could be under the AI canvas, and make shapes that don't correspond to what a user is actually seeing. This is somewhat subverted by the catch all erase function, but it is an area of improvement I'd like to make. More so, this limits the potential of shape exploration by the user, with a wider canvas allowing for more variation over time and over users. That said, it is less of a priority than the collaborative functionalities I'd like to make. I also have begun setting up systems for the changing of the AI prompts, but I'm still undecided if I want a midi pad to control it, or if I want some other options such as having them cycle slowly through time between flowers, insects, and fauna, with the Emergent [title] changing to let a user know what their shapes are currently being interpreted to, and to make the small lines of code that are added in post over the AI canvas to be dynamic and changing, to maybe further emphasize the collaboration between the user and the system. Now that I have the gallery space too I can start thinking more about presentation options. Right now the two feeds are juxtaposed, which I think works well, but I can think about other options than having a TV monitor in the space, such as if I want the AI output to be projected onto the wall next to some posters I make, or if the entire feed is projected. Something to think about, and the projection can be incorporated with another new project I've made good progress on.
How Original?
This is a new project that stems from AI image databases. The structure of the database involves billions of images scraped from the internet (often without permission) to be used in training. In order for an image generator to be able to generate any given prompt, these images cover all sorts of ground from art to photography, from nature and industry, to animals and people. From all this coverage there is one thing that unites all of the images in the database, and that is the human behind them all. Even a picture of a tree deep in the amazon rain forest, or of a galaxy billions of light years away, the underlying factor that makes the image possible to be viewed and interpreted in the first place is the human ingenuity, from the person who took the picture, or from the years of collaboration and accumulated knowledge that allowed for the creation of the image-creating devices. This is true of AI as well, as the technology behind processes has taken years of hard human work to achieve, and the training images even more: spanning from the present day to the beginning of documented history.
The nature of the images collected and repurposed has left some to be perturbed by the process, and maybe rightfully so, since no permissions are often given for an image to be fed to the model. However, the taking from one source and repurposing for something new, is in fact not new. From renaissance ideas of antiquity and humanism inspired from ancient Greek philosophy, or the architecture of U.S. government buildings directly inspired from ancient roman architecture, our inventions and our technology are seldom, if ever, actually invented more so as they borrow ideas from the past. Insert Virgil Abloh's 3% rule: To make something culturally significant, you only need to modify 3% of something that already exists. Machines and AI have a unique way of interpreting the preexisting images and data fed into it, "remembering" the past differently than we do. In the instance of gesture and body tracking, a machine with inputs of millions of human bodies and gestures across history, is much better at measuring the distances between points and vectors, than it is at actually understanding historical significance or emotion. This is not necessarily a flaw, but more of a confrontation of the limits of AI and machine logic, and the irreducible complexity of human expression.
This preamble leads to the "How Original?" project. The idea is to match a live gesture from a viewer or user, with that of a historical image of a person doing a near similar gesture to that of the viewer. In this instance, AI, often framed as a tool of the future, is repurposed as a medium for engaging with our collective past. (An interaction with time, and a focus of the 3rd project discussed in this post).
Currently, the database being used in the pictures is from open source training sets used in machine vision, and don't really match quite well with the users live gestures. The current database was used more as a means to get the backend logic working, with the mapping of joints on a picture and somewhat matching that to the live stream. The good thing here is that the hard work of actually coding is mostly complete, what's left for me to do is gathering a database of open source historical photos (I'm targeting anything pre-1960 so that the time between the "now" and the "then" is more drastic). The bad thing is that this will likely be tedious and manual, having to find pictures that are of a decent resolution, and show a persons full body, and I'll have to gather enough to get a full mapping of a person's possible gestures. I'm not targeting complete similarity between poses, I think around .7 would do the trick in getting the idea across. The idea of viewing history as a living collage and peeling back layers of time, where our technologies and our gestures are both individual and collective, and repetitions of ideas are not coincidence or con-work, but a continuity of humanity and our stubborn refusal to let go of the past. The idea of working with AI through time is a strong basis for the last project I've put a little work on: "The Single Greatest Piece of Art That Has Ever, Can Ever, or Will Ever be Made".
The Single Greatest Piece of Art That Has Ever, Can Ever, or Will Ever be Made
Spoiler alert: it's not "The Single Greatest Piece of Art That Has Ever, Can Ever, or Will Ever be Made". However, if you tell an AI image model to create "The Single Greatest Piece of Art That Has Ever, Can Ever, or Will Ever be Made", it will generate an image "The Single Greatest Piece of Art That Has Ever, Can Ever, or Will Ever be Made". But as you would guess, what it would generate would more likely be "The Single Greatest Piece of Meh Confined to a Square Canvas".
Similarly, AI as a technology is often heralded in the hype cycle as this technology that will change everyone's lives for the better, make work easier, but has it actually? Or has it just increased CEO's profit margins and make funny slop memes on TikTok? In the hate cycle, people say AI will take the common-folks jobs, ruin the environment, or that the bubble will burst soon and the stock market will fall, a depression will commence, soon followed by the end of the world. But how much is AI impacting the environment compared to things like the oil leaked from massive shipping vessels or the already existing data centers that house the servers for things like Google and Instagram that people don't seem to have as much of a problem with? Will the bubble actually burst or will the progress of AI just lead to massive expansion? A common endpoint for both sides is the idea of a singularity that looms over our heads that in either case would cause massive upheaval of everyone's lives, maybe not good, maybe not bad, but certainly massive change. These areas of unknown and uncertainty can only be known after time allows it.
Time is a funny thing when it comes to AI. As in the previous section, AI connects us to our collective past and our potential future. In the case of image or video generation, AI generates near-instantly what could take a traditional artist weeks or months. What if we subverted this time-to-generate, lengthening the time it takes to generate from almost instantaneous, to hours, or weeks, or years, or even centuries?
Using time as this supplementary medium is not entirely new as a concept (repetition through time again). The pitch drop experiment is the worlds longest running "experiment" that sees pitch (a highly viscous liquid comparable to tar) being dropped through a beaker. Because of how viscous it is, the time between singular drops lasts anywhere between 8-14 years. This has sensationalized it in a way, with a constant live-stream of the beaker being filled up with viewers whenever a drop is about to...drop. With how much liquid is in it given how infrequently it drops, the beaker is expected to continue dripping for at least a hundred more years until it runs out, with only 9 drops so far. On the more artistic side, a John Malkovich movie called 100 years, will fittingly come out in 2115, 100 years after it was made and likely after the entirety of the makers will have passed. Lastly, the project I think is the most interesting and takes place on the longest time scale is the Zeitpyramide (time pyramid) is a sculptural art project that will see 120 concrete blocks shaped in a pyramid completed over the course of almost 1200 years, with only one new block being added every 10 years. Starting in 1993 and as of 2023, there are only 4 blocks in the structure, and with the current schedule the project will be complete in the year 3183. If you include evolution as a creative act, then we're talking about a time frame of tens of millions of years, or including the natural beauties of the world like the grand canyon, then billions of years of rock and magma and water crushing and grinding together. Beautiful spiral galaxies on the edges of the known universe even longer.
All of this is to say that our perceptions of time and unveiling of the unknown can often be disappointing, and our expectations should be kept in check. The pitch drop experiment will keep dropping, John Malkovich's movie was sponsored by a cognac brand that ages it's product for 100 years, we already have visualizations of the time pyramids completion in the form of diagrams and pitch concepts, and evolution and star formation act on such a long scale of time that they may as well be static to us (as some actually believe). In the case of AI and "The Single Greatest Piece of Art That Has Ever, Can Ever, or Will Ever be Made", it's important to keep our wits about and not let the hype/hate get to us. The promise of "The Single Greatest Piece of Art That Has Ever, Can Ever, or Will Ever be Made", and AI's benefits and consequences, are manipulated through time and language. Big words and high fidelity tech demo's only show a piece of the picture, just as in this project, the AI denoising of a timeless incontestable "masterpiece" slowly shows progress over time, but the final outcome is often disappointing compared to the buildup of hype we can be trapped in.
As it stands right now, I have this ComfyUI setup that I can naturally extend the steps on to an extent, which pushes the generation of an image from a couple seconds to a couple minutes. If I actually wanted to keep the AI going for longer than a day, my computer and GPU would probably burst into flames, so I need to think about ways I can work around this limitation to achieve the intended effect, and to introduce more design elements that can push the concept or prolonged process and the manipulation of time and hype more.
I think it would be a bit of a disservice to the concept if I knew at all what the AI would generate over a conceptual 100 years, but in setting up the system and getting a prompt going, I had to generate at least a couple images. With a massive prompt along the lines of "create a maximally creative piece of art that has no equivalent using any medium", the first image it generated I thought had a bit of poetic irony to it, although to be fair I made all the meaning myself.
Interesting that the seemingly maximally creative singular best piece of art ever created by an AI would be of a dark cube-like shape... a black box if you would, burst open to reveal the visage of a human head. As with all great art I'll leave it up to audience interpretation, since again with the AI image itself there is no inherent meaning other than what we make of it.
I think the concept is potentially strong as it is, but I think selling the execution as a Thesis project leaves maybe some more to be desired. I think if I can frame it correctly as an installation piece, as another interaction through time, this particular one focused through the lens of the hype/hate cycle, it could work a bit better.
Where the Next Steps are Leading
Evidently reading has been sidelined this week to crunch on these projects, which I will make a strong push to flesh out as much as possible both before the semester ends and before my committee meeting. These projects and making more progress on them aside, I have the Muse 2 brainwave scanner now that I have to build up a project on. The scanner is a little finicky and not the most consistent, so I will have to build accordingly to make it the most usable given its limitations. Combining it with more reliable face tracking and the emotive-reactive project makes more sense this way, as that can more drive the "dynamicism" of the project for a user to interact with, while the brainwave scanner can be more of something that activates the project from an idle state, as opposed to relying heavily on the brainwave data it produces. With as much progress as I've made on the installations, I've also had the chance to conduct 3 interviews with AI practitioners to get some insight on their outcomes in understanding AI, which I will have to lay out and decode some common themes on, and maybe reach out to more practitioners in the coming weeks, since the interviews are low-intensity I would be able to conduct much more depending on if I get responses from those I reach out to. Lastly, the workshop planning will take more precedent as I push forward with these projects and likely be a goal of mine through winter break to finalize.
This week I made more progress on the Non-Sentences project, and made more steps in the other two projects for my thesis: The Emergent Garden Interactive Edition and the Cognition project using the muse 2 brainwave scanner.
First for Non-Sentences, I've made more steps in solidifying the visual aesthetic of it, as it "vibes" towards an old computer monitor with a front end "terminal" on the right, and the network of words on the left. I added some feedback and gamma edits to make it more "glowy", and I'll explore some more effects like scanlines, some warping, and grain to pull it all together.
Another edition of this project to place it better in the installation space that Alex brought up which I think can tie it together is the addition of a receipt printer, to print each sentence and network physically. That would primarily take the form of a mono black and white version of the current setup landscape on the printer, and in physical space this achieves a couple of different things.
1) It will add a tactile layer of interactivity with audiences able to see physically what manifests out of a non-sentence, and adds a bit of small permanence to the sentences themselves, as they fade out of existence on the monitor, they would be preserved in their printed forms.
2) It serves as a physical representation of the scale of the "slop" the AI produces. While people would be free to take a piece of receipt with them as they please, I foresee many choosing not to, or no one being around to take them if the system runs autonomously, which would result in a pileup of the papers on the ground. This physical representation would show the speed of potential non-sense that is filling up the digital realm, translated into the physical, as well as an image of the environmental consequences of this non-sensical generation. As far as the content in the paper, as the system runs and runs the words would eventually start to homogenize, which is a potential concern as the internet is being filled more with AI content. AI is trained on human content, but as more AI generations fill the internet in writing and images, a potential feedback loop can occur where AI endlessly echoes and repeats itself based on what its already generated.
3) In the same spirit as number 1, the permanence also leads to the potential for something else to be created, as I can see a potential poster that lays out the receipts as something even longer lasting, and touches on calls for sustainability in AI. Turning nonsense into sense, and meaningless into meaning.
For the Emergent Garden interactive edition, I've taken steps to add this level of interactivity from my previously made TouchDesigner network. From what I've already done, I would say the project is at 1/3 done. The outputs of blob tracking, prompting, and visual code already exist in the project file, the next steps are to add the gestural interactivity of cultivating the garden with users hands: drawing shapes with their fingers and hands, and the physical hardware interactions of changing prompt parameters and the AI behavior with a midi pad. This dual interaction takes into account the hands that shape and the tools that build. The hands that shape the world around us as forming the backbone of the garden with the original primitives that can be physically placed and manipulated, drawn and erased, planted and uprooted. The tools that build, through the hardware and software, show the calculable effects the introduction of these tools can have on the garden, ecosystem, and world around us; but are ultimately driven by human intention and intervention. These interactions are the other 2/3 of the project that need to be done, but will take a primary focus in the coming weeks. As far as the physical space, I'll have the monitor setup with the interactions showing live the final evolving output, with the 3 printed posters next to it to both fill up the space of the exhibition and again to add some permanence to the outputs.
Finally, for the Cognitive Control project, I just got my Muse 2 brainwave monitor in the mail, so I'm excited to explore that and combine it with the face tracking - emotional control project I already have. I think what I'll want to touch on this one is the element of the known and unknown, how, since AI is such a new thing, it is not fully understood how it can affect our brain, but there is some research supporting it's affects on cognition, thinking, etc., both positively and negatively. For a visual I'd like to supplant some real images of brain scans in conjunction with a users real brainwaves, their face tracking, and the AI visual that ultimately comes out of it, drawing from highlighted neurons to create a pseudo-data visualization / visual commentary.
What I’ve Read
Reading this week focused more on play, interaction, and experience as they relate as defining terms for what I'm working on as a whole body of work: installations and workshops. The more I read into the following research papers and into Homo Ludens, it seems that play is naturally making it's way out of my thesis in favor of the cross section of interaction and experience, while some of the ideas of how I personally (and informally) would define play remain, but are more focused and intentional under these terms.
Starting with interaction, as I read more into it, it seems that it shares some similarities with play as a concept in being somewhat unambiguously defined in research, which is surprising to me with it's inclusion in the name of the field of HCI. There are just as many "towards a definition of interaction", "defining interaction", "in support of a definition of interaction" research papers as there are that substitute interaction with play. Some older but relevant papers: “Towards a Definition of the Term and Concept of Interaction” (Schwaber) and “In Support of a Functional Definition of Interaction” (Wagner) make more functional, operational, and rigid definitions of interaction.
Wagner grounds the concept in instructional and learning theory, defining basic interaction in learning contexts as "reciprocal events that require at least two objects and two actions", while also making distinct interactivity and interaction, with interaction as a learning-oriented process of this reciprocal influence between learner and environment, and interactivity as a technological affordance that enables such exchange.
Schwaber moves towards a more psychological approach to defining it, moving a little away from a strict learning environment, with interaction as an "intersubjective co-creation and reflection", again bringing up these ideas of reciprocal engagement between two or more parties but in a less rigid and pedagogical lens. Her definition emphasizes the co-creation of meaning through mutual influence and recognition between entities, in my thesis case this would be between a person and an AI: either through direct making in the case of the workshop, or through experience with the installations.
These definitions are well and good, but a bit old, and not directly related to technology or HCI, which is where Hornbæk & Oulasvirta's "What is Interaction?" fits in nicely, and makes up what these older definitions lack in their "folk notions". A direct HCI paper, they position interaction as "conceptual diversity and human-computer coupling", proposing not one but 7 different sub-definitions under the umbrella of interaction. These sub-definitions being:
Dialogue (turn-taking communication) Transmission (information exchange) Tool use (human action mediated through artifacts) Optimal behavior (adaptive goal pursuit within constraints) Embodiment (being and acting within a socio-material context) Experience (emotional and experiential flow) Control (continuous feedback systems minimizing error)
All of these relate in one way or another to aspects I'm covering in the workshop, or at least one / all of the installations, especially in experience, dialogue, tool use, and embodiment. I want to take note especially of experience and dive into it's own definition as well as it relates to my thesis, with one of my main source papers that I brought up in my midterm presentation that takes a look at experience as it relates to explaining AI, this being aptly named "Experiential AI: Between Arts & Explainable AI".
An overall definition of "experiential AI" in this paper is “An approach to the design, use, and evaluation of AI in cultural or other real-world settings that foregrounds human experience and context. It combines arts and engineering to support rich and intuitive modes of model interpretation and interaction, making AI tangible and explicit.” In other words, experiential AI makes the invisible operations of AI visible through human-centered experience by blending artistic methods with technical insight so people can feel, see, and manipulate how AI systems work.
The definition of experience in this paper is twofold. One is derived from experiential learning theory (Kolb, 2014), where experience becomes a medium for knowledge creation, as people learn through active involvement, reflection, and feedback (seen in what my workshop is trying to achieve).
The second definition is experience as aesthetic encounter (seen in my installations). This puts experience as directly graspable engagement, where audiences emotionally and cognitively interact with AI systems and artworks. These experiences foster understanding, critical reflection, and emotional connection.
Taking all of this together, I've gotten a working synthesized definition of the concepts as it relates to my thesis:
Interaction and experience are understood through the lens of experiential AI, where understanding AI arises through embodied engagement, creative co-creation, and affective reflection. Experience is both process and outcome, a space where literacy is gained through exploration, experimentation, and encounter. Experiential methods make AI systems legible, tangible, and emotionally resonant, transforming technical clarity into situated learning. Interaction functions as the medium through which this learning unfolds, a feedback loop that connects human curiosity, aesthetic interpretation, and algorithmic behavior, ultimately fostering technical and critical AI literacy.
This is by all means a working definition, and is something I will likely bring up in my committee meeting, since the workshop and installation projects will be happening regardless, this remains one insecurity in shaping and defining the overall theme of them together.
Where the Next Steps are Leading
Next steps are to continue the installation works, now that I have the Muse 2 I can work on that project more instead of sketching it out, which I'll start with the basic interaction and build it up from there (as seen in the process diagram I've continued to bring up). I'll need to reach out to see if I can get the space in the college of design by next week for sure, and I'll need to ask Carly in how to do that. Then I'll have to round out my presentation for the committee meeting, and gather any lingering questions or insecurities for the thesis itself.
Sources
Hemment, D., Murray-Rust, D. S., Belle, V., Aylett, R. S., Vidmar, M., & Broz, F. (2024). Experiential AI: Between arts and explainable AI. Leonardo, 57(3), 298–306.
Hornbæk, K., & Oulasvirta, A. (2017). What is interaction? Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 5040–5052.
Kolb, D. A. (2014). Experiential learning: Experience as the source of learning and development (2nd ed.). FT Press.
Schwaber, E. A. (1995). Towards a definition of the term and concept of interaction. The International Journal of Psycho-Analysis, 76(3), 557–566.
Tawfik, A. A., Gatewood, J., Gish-Lieberman, J. J., & Kinzie, M. B. (2021). Toward a definition of learning experience design. Educational Technology Research and Development, 69(6), 2941–2962.
Wagner, E. D. (1994). In support of a functional definition of interaction. The American Journal of Distance Education, 8(2), 6–29.
This week I kept up on the Non-Sentences project mostly, I spent most of the time working on the network lines and a new text to speech mode from it that adds a female voice speaking the words as they come out generated. These two things, which on the surface don't look like that much, took a lot of time and coding to make them working. For the lines, essentially I had to calculate the x and y position of each cell, then draw a 3D line based on those points as they're being called by my sentence controller script, to draw it out word by word, then erase after the sentence is being generated. It makes interesting shapes and maybe there is a way to use them in other contexts, either within or outside of this particular project. The text-to-speech took some major edits of this sentence controller as well, as it naturally wants to read each word as fast as possible, resulting in a lot of cutoff from word to word, trying to read the full set of words every time before being cut off when a new word was added. I had to work a lot with timing and spacing of the words being generated, and spacing between the generation of the spoken word, as how that works is it basically creates an mp3 file for each word, then finds that file in my computer, and plays it back out through TouchDesigner. It still reads the last word a couple times before the sentence is erased, but I think it adds a sort of hypnotic element to it, akin to movies like Blade Runner 2049's baseline test (interlinked) and the opening of Under the Skin as the Scarlett Johansson alien is being "built" and learning to talk like a person, which is based on real life phonetic training for speech therapy (the best available video I could find of this I'll have below, as well as the Blade Runner sequence).
As it relates to my thesis, this can bring more insight into the training of a LLM chat or TTS model. Trained on millions of words, this project can be seen as a visualization / audio representation of the back end process of a model both learning how to structure sentences that make sense to us, and the actual behind the scenes of a model stringing together words before they become visible to us. Normally, we see it's best foot forward in this predictive randomness, but not the millions of millions of training nonsense or how it chooses to put words together. I also think the imperfect voice adds to the uncanny-ness to it all, where by lowering the fidelity of the voice (compared to something like ChatGPT's text-to-speech models) it strips back a layer of relateability, where its main goal is trying to mimic human speech patterns, but by it being slightly "off" we might better step back and see it for what it really is: algorithmic decisions trying to make sense.
As a note the audio recording for the voice is pretty quiet, so if you turn up your computer audio to hear it make sure you turn it back down before you play the other videos.
To add more to this specific project I think I will add a word counter that shows how many times a specific word has been used by my algorithm. With it being random you would think over time that all the words would average out to about the same number of times being used, but in building this I've noticed words like "architecture, facilitate" and "dream" being used more frequently than other words. Adding that can speak to the over-reliance of certain words by LLM, which are making their way into our language as well, and in a non-measurable speculative effect of this project for someone viewing it, they might end up using some of the words in the table more frequently than others. Additionally for some stylistic polish I need to add the words being used highlighted in the cells besides just the line drawing to it, and think of some options for the sentence text on the right, thinking of having the characters look like they're "floating" by translating them on the x, y plane smoothly and randomly, to add to the flexibility of meaning as sentences break down to words, break down to characters, which break down to lines.
Among working towards other projects for the Emergent Garden interactive edition I've done some work on the interactive aspects of hand tracking, where I have currently a setup that tracks your index finger and thumb, and a basic interface over it that shows a datapoint of how spread out your fingers are. This has a lot of promise already, as all the data is there to map things like primitives to your fingers or the midpoints between them, maybe pinching to increase / decrease their size, and using other fingers or hand motions to place them down. There is a balance to find here, where you want to give people as much control to build the garden as they please, but with non-traditional methods of building, there can be a chance for people to get confused of how it actually works, which could undermine some of my work on it. This is where potential combinations of midi pads could come in handy I think, where there is a reliable hardware interface that people can use aside from their actual body movements and motions.
Finally, in terms of the previous project with face / emotion tracking, I did some brainstorming and think a good direction to go with that could be a reactive AI-rorshach visual that reacts to your smile or frown, with some accompanying text that's trying to elicit how you feel in the moment, and try to manipulate how you feel to be overly positive. Rorshach tests nowadays are typically seen as outdated and unreliable, bordering on the edge of pseudoscience as a means to determine personality characteristics or even underlying psychopathologies. What else may be unreliable? Maybe using an AI model as a therapy device or emotional crutch. The Rorshach test has a couple of "algorithms" or scoring criteria to determine anything from predisposed schizophrenia to a persons personilty, coping mechanisms, or personal perception. These algorithms provide a means to an end to a predetermined result of a score for whatever may be evaluated in taking a test, but suppose you don't know that and take one from a seemingly trustworthy source, and take at surface value what the Rorshach test says about you or your personality, and you believe it to be true even if it was based on uninformed pretenses or contexts. The same line of thinking can be applied about AI models and concepts of sycophancy, where the AI tends to reinforce beliefs that you have when conversing with it, whether it be your emotions or if you have a great new business idea to combine french fries with salad. (A newer South Park episode used this concept recently, to mock AI, startup funding, and of course the Trump administration).
Finally, on the more research side of my thesis, I've reached out to a couple AI creatives, and have gotten some confirmations on those that would like to be interviewed. One of my favorite creatives using AI was one of the first to reach out, which I am pretty stoked about. I won't say who he is, but he's done some incredible work with AI that explores concepts like the perception of time and fragility of infrastructure across long time, and has been featured on things from electronic billboards to creating visuals for major music festivals and artists, like Future, Metro Boomin, Travis Scott at events like Rolling Loud and Lollapalooza. Within the next couple of days to weeks, I'll have some interviews down and transcripted to be used for my thesis.
What I’ve Read
I managed to get a hold of "HOMO LUDENS A STUDY OF THE PLAY-ELEMENT IN CULTURE", and in reading some of it and using Google Notebook to quickly summarize other parts, it brings up ideas and definitions of play that seems familiar to me, whether I've heard it from an undergraduate philosophy class or YouTube essay. Johan Huizinga describes play as a fundamental and autonomous activity that is free, meaningful, and distinct from ordinary life, emerging before culture or civilization itself, as animals also are observed to play with each other. He argues that play is not merely a biological function or a means to an end, but rather a primary category of life that carries its own intrinsic value and significance. He says play is “in fact freedom,” a temporary stepping outside of real life into a separate sphere governed by its own order and rules. Although it is “not serious,” it can be pursued and used with complete seriousness and absorption. Serious in tone and attitude, but not serious in stakes or survival. Within this self-contained world, play creates order, tension, and beauty, giving meaning and form to human experience beyond survival or utility.
Huizinga says that play is a foundational element of culture itself, not just a pastime or instinct. Civilization, he claims, “arises and unfolds in and as play,” and thus Homo Ludens: “man the player”, is as essential a concept as Homo Sapiens. Through language, ritual, art, and contest, humans express their creative and cultural instincts in the spirit of play. This definition frames play as more than simple amusement, to a cultural force that shapes meaning, community, and the human spirit.
As this relates to my thesis I'll have to do some more in-depth reading in how play can promote learning and AI literacy. The shaping of meaning and community seems like it could be a good stance to explore further, and relating it to the ideas of biology and AI I've explored too seem like a connective tissue that can be formed.
Where the Next Steps are Leading
Again, I'll need to continue refining my projects / installations. My muse 2 has been in the mail for some time now, but it should be arriving in the next couple of days so that I can explore that more. I'll have to refine the emergent garden interactions as well, and in working with the Rorshach idea I'm thinking if there is potential in combining it with the cognitive control to fully encompass the transference of meaning, collaboration, and cognition from emotional, physical, and perceptual levels. Aside from projects, the interviews are coming along. Once I've done a couple of those and have the transcripts, I can start coding them for consistent themes and work towards the workshop content.
This week as most of us know was for our big presentations, so that is where a bulk of my making went, through slides and formatting. In the making of the slides I feel like I had a chance to think in a different context about my work. It hasn't been so much in an isolated circle, but to formally present to those with no context definitely had me thinking more critically about what words go where, and how to convey certain concepts for the best understanding. Before any feedback while I was still developing the slides, I was going back and forth between the words of play, creativity, interaction, experimentation, and experience, and what they actually mean. I think from where I was in the beginning of the semester, these words, their definitions, and what they mean in the context of my thesis are making much more sense, but there is still more to be done in that area. Drawing from Tina and Silvia's feedback, I have done a lot of thinking on what my audience should be, but what I want them to feel is something I'll have to think about more: where play fits into understanding and learning. I think the title "Playful Interaction" steps into a right direction, where it's not entirely play, and places concepts like participation and experience tangentially with the concept of what play is, so maybe play as a word will make less and less appearances in my thesis from this point forward.
Before I step into Rao's feedback, I'll briefly cover some more applicable "things" I've made for the thesis: these tools for the workshop. The idea behind them is to make TouchDesigner more accessible so there's less development and learning of the program itself, and more time to focus on learning an AI workflow and fast-tracking results. I've been building some tools and gathering some from free open sources, with the idea of being able to utilize them in either the pre-processing and/or post-processing areas of my flow-chart. These tools include now but are not limited to: a mouse drawing tool, machine vision made easy for face / hand tracking, filters made easy like halftones, chromatic aberration, displacement and noise, fundamental "mini-scripts" that are essential for movement in TouchDesigner (absTime.seconds()/absTime.frame makes an object move with the seconds passed or amount of frames), drag and drop sine and cosine waves, and a poster tool that makes text, shapes, and borders easy to add for an easy composition. There isn't too much to show of these tools as they all take the appearance of TouchDesigner node blocks, but I'll include screenshots.
From mostly Rao's feedback and some of the other comments, I think that there is some good things to address here. Mainly for quantifying what people take away from AI literacy in the context of the installations. I think from the interview and workshop protocols there is grounds to gauge what working practitioners have gained literacy wise from working with AI, and what those with less experience stand to gain through the workshop, which maybe I could have articulated better in my presentation. This is a possible area of concern for the installations though, as currently there is no metric for what people gain literacy wise from viewing and interacting with the pieces. Perhaps this could be an added leg of research, doing something similar to what Mira Jung did with her soil research, having groups of participants view and interact with her pieces, then answering a questionnaire to see if there was any knowledge gained. I feel like I would have to decide this pretty soon though, since the installations themselves would have to be finished by the end of the semester, a space to set them up would have to be decided, and IRB would have to be approved. I think it would be valuable for my thesis, but I'll have to think a little bit on the next best steps.
What I’ve Read
Continuing my readings in Transcending Imagination, Chapter 8 treads towards more thought experiment and philosophical ideating, with topics like sentience and emergence, which are relevant topics for AI but aren't particularly relevant with current issues with AI, and what my thesis is going towards. He does make a distinction which I air towards in my presentation, the separation of technical processes vs. analytical critical processes. In the case of sentience he talks about embodied functions, like mathematical and logic-driven operations, and emergent functions, such as judgment, pattern recognition, or aesthetic perception. He says that sentience is less about super-intelligence and more about the capacity of AI systems to produce outcomes that evoke understanding, reflection, or emotion in human observers. He then introduces this concept of autopoietic intelligence, which describes a self-maintaining system, and draws more parallels to biology and evolution/adaptation.
I don't think I'll cover this AI sentience in my thesis, as I want to keep the focus on the people who use AI and how to increase their knowledge perceptually. Sentience is still also a very conceptual thing, and fits into the hype/hate cycle as a kind of unknown. Unknown if it would even be possible, from the sheer resources required or otherwise, but I don't think it's particularly important in the here and now.
Where the Next Steps are Heading
After this week the next steps are to keep working on the installations and reach out to interview candidates. I set up a calendly to get times scheduled and make the process easier on them, so hopefully I'll have some scheduled or at least the beginning of conversations coming into the next week. For the installations the first step is to work on getting the network set up for non-sentences. Once I have that, I think I will pivot to the Emergent Garden and getting the hand-tracking and gesture as interface aspects of that working for a live installation setting.
This week was a big focus on getting the Non-Sentences project up and running, and I made some really good progress on it, with most of the backend up and running, minus 1 piece to tie it all together. Quickly on the experience of making it first, it's always nice to learn more about design programs, TouchDesigner being one I am particularly fond of these days. Learning more about it will of course be beneficial in being able to talk about it and teach people about it on a basic level for the workshop, and while there are some things that can be easy for newcomers to pick up and start making things right away, especially with AI involved, this project tested my knowledge on the actual coding and data structure side of things, which was a nice challenge I've mostly overcame.
Below the sketch I made initially just below here to refresh on the beginning of the project vision, it involves the sentences being generated in real-time on a loop (with the words being chosen at random, but in a fixed order that would fit a "grammatically correct" sentence structure) displayed on the right side of a screen, while the left would show the table of words it can choose from and the network connecting all the words together.
A pretty basic sketch, but it's beginning to take form in the short recording below, where I have the table of all possible choices next to the generated sentences being run in real time on a loop.
I think the most interesting one in this video just so happens to relate to the project itself: "Our intelligent network expands into your facilitated language."
Of the progress I made this week, I wasn't quite able to get the network of lines from word to word on the table. It took me the most time to get the sentences to be able to generate word by word, then erase in a loop. As I said before TouchDesigner can give newcomers the ability to make pretty interesting things relatively fast, but with the nature of this project I ended up using a pretty node-less system, having the bulk of the sentence generation in one text block DAT, which took more time than if it would have been a pure node system. Below is a screenshot of the tables and the small number of nodes used, and a screenshot of the 'sentence_controller' text file.
While not specifically using any AI model (for now, I'll discuss near-future plans for this a little further down), the conceptual basis of this project is that it would shed some light on the process of how a chat bot comes up with strings of characters that can be interpreted as sentences by us. This combination of database driven algorithm and human pattern-seeking intuition gives these strings a meaning for us in the form of sentences. Even if they are semi-random (true random being a contentious topic in the world of computers), our meaning-making machines of brains decide when these random strings make sense, and even if they provide some value in the form of poetry. On the other hand, the random nature of the algorithm can spit out sentences that don't make sense: "Non-Sentences", which is where a conversation on AI-hallucinations comes in. Chatbot hallucinations make sense in the fact that they are grammatically correct sentences that you can read, but the information can be wildly and totally incorrect, which leads back to us and makes our ability to be able to detect and interpret these hallucinations for what they are much more important. This is AI literacy not in the technical sense, but the interpretive sense, and the ability to read AI bogus will become a more important skill to have when things like scammers and misleading ads on social media or the news have already begun implementing these chatbots into their writings. That's why in the tables I made it a point to use words that AI prefers to use: latin-english buzzwords of higher than normal complexity, like "delve", "underscore", "bolster", and "transformative" to name a few.
To add onto this more, obviously the next step in my vision for this project is to have the "network" lines generate from word to word along with the sentence, then erase and start over again, along with the sentence. This could be a pretty complex step but from how the tables are set up in TouchDesigner I think I'm in a good position to make it happen relatively quick. To hone more into the linguistic side of it (and maybe how AI is changing our language as per The Verge's recent blog post and the research behind it (paper 1, paper 2)), I've dabbled around with adding a text-to-speech model to read the sentences as they come out. I've experimented with gTTS, googles python based text-to-speech model, which can be easily implemented within TouchDesigner as it's a python based environment, but as I've gotten it to "work", it is pretty glitchy with how the sentence is being spoken, currently stumbling over the sentence over and over again at a fast unintelligible pace. A next step could be getting it to slow down and read word for word by using similar logic for the sentence generation, just going word for word as opposed to combining them all. Another option could be using ElevenLabs for speech, as they have plugin API's that work in TouchDesigner, and may offer some more intuitive, quicker, and more extensive control.
Looking briefly towards other projects, for the Emergent Garden I've thought about modes of interaction and I think that using hand tracking as a form of activation and control would be a good way to expand more on the idea of cultivation of a technological ecosystem, where by using motions like finger pinching or flicking the hand, you can place or remove primitive shapes to add to the ecosystem, and change the AI's parameters to create new evolutions of plant and animal organisms. Adding this semi-tactile form of interaction lends more to the idea of human intervention in the ethical use of AI, speaking more towards the balance between control and collaboration. The act of “tending” to the system through motion could represent the ways humans guide, nurture, or even disrupt technological growth. It would also introduce a more embodied and intuitive relationship with the work, allowing the audience to feel physically connected to the generative processes at play. This connection can enhance immersion and reinforce the metaphor of co-creation, where human gestures influence the evolution of an AI-driven environment, mirroring the ongoing negotiation between human intention and machine autonomy.
What I’ve Read
For readings I've continued with Transcending Imagination, which I think is proving valuable for this project-based portion of my thesis in investigating the literacy of AI as it pertains to its effects outside of technology, especially in this chapter (only one this week, most of my time is now spent on making as per my thesis timeline).
In Chapter 7, the term “narrated economy” is introduced, which reframes the designer’s role from fabricator to narrator, bringing more emphasis to the articulation of intent and meaning as a "new" creative act. I think that is ties into where I'm going with my projects and prototypes, with the idea that interaction itself can become a form of inquiry, with storytelling, gesture, and experience as tools for making AI’s invisible processes and effects visible. By positioning AI as an active participant in creation, Manu says that the designer’s task is no longer to simply use AI but to converse with it and to narrate, test, and reflect on its responses. Through this thinking, the interface becomes a "site of revelation", where AI’s biases, limitations, and assumptions can be surfaced experientially rather than explained abstractly. The resifting of human-centered, narrative-driven design also shows the ethical and aesthetic dimensions of this shift. If designers are now “architects of experience,” as he describes, then we must build spaces where people don’t just observe AI outputs but feel and reflect on how those outputs are shaped, which I can see in my own approach to experiential design as a reflective medium, using interaction, visual form, and sensory engagement to expose the underlying mechanics and ideologies of AI systems.
Where the Next Steps are Heading
Of course in the immediate future I'll be needing to put together my presentation, which I've outlined fully, just a matter of getting everything onto slides and writing a pseudo script to follow the beats to. After that I'll continue on the Non-Sentences project to at least what I had in my original vision of the sketch, expanding on it if it doesn't take too much time in areas of relevant text effects or text to speech plugins in TouchDesigner. After that I plan on ordering my Muse 2 for the brainwave project, then reaching out to my shortlist of creatives who implement AI technologies in their works for interview times, continuing to work on other projects in the meantime, such as refining the emotive-control project I'd done already, or adding hand-tracking capabilities to the Emergent Garden.
This week I did not do a ton of design work as opposed to finalizing my research protocols for the workshop and interviews and getting that submitted, and almost immediately approved by the IRB exemption review wizard. Compared to the last instances I shared on the last report of the interviews and pre/post survey protocols, I was able to refine and reduce the content in a more appropriate manner. For the interviews, I reduced the master list down to 11 questions that should vary in the time it takes to answer them, ranging from longer form process explanations to questions that can be answered relatively shortly. This is to be respectful of their time and give me the time to find a suitable number practitioners using AI, with an interview time of 30-45 minutes allowing me to recruit more participants compared to an hour or more long time. I am aiming for 3-5 practitioners in various fields in order to get some varied responses as far as their outcomes in playing with AI and learning about AI and technology in their creative process of making. For the pre/post workshop surveys, I restructured the questions to make them more suitable for comparison of data before and after the workshop (for some of the questions, they are 1-1 in the pre and post surveys, making it easy for direct comparison), and making them more neutral and less leading, in order to not assume outcomes from the workshop itself. Another thing that Alex suggested was to have a scale of 1-4 for the Likert-style questions in order to actually push participants over a threshold as opposed to having a neutral option that does not give much room for a baseline improvement or regression. The baseline would be if they answer the same choice for the question before and after the workshop. Now that this is out of the way, I can start planning more in terms of the workshop, things like sourcing materials and refining more of the slide/workshop content; and finding participate candidates for the interviews.
Additionally with that out of the way, I began sketching for the design project portion of my thesis. I accomplished a couple sketches, maybe a little less than I was hoping, but it's still good to get the ideas out that I've been putting off for the sake of scoping my ideas down. I'll post the one's I got the furthest with at the time of writing, and briefly explain the ideas behind them, and keep in mind they are really rough. As a whole, these prototypes and what comes after will seek to communicate and teach those engaging with these things about AI that aren't necessarily on a technical level. While there still are technical aspects that would be included in each one so far, in the big conversation of AI it is important to learn about it's affects either psychologically, cognitively, or conceptually. Additionally from the pieces themselves, I think having some standees or something equivalent can help communicate the ideas, similar to what you'd see in a museum or gallery of an artist statement. Lastly, with all of these I think it is important to include some form of process within the final design, either as side-by-side network that showcases what's going on, or an abstracted visual. This would be to further increase transparency and learning of what the model's and processes are actually doing behind the scenes, by bringing them to the foreground.
First is the idea of using a brainwave monitor as a conduit for driving AI Image generation. The monitor (A Muse 2), would be connected to a phone app that creates OSC data on the 4 main brainwaves (Alpha, Beta, Delta, & Gamma) which can be wirelessly communicated to TouchDesigner, and thus an AI image model. Aside from an interesting and interactive installation idea, this can be used to communicate some cognitive effects AI can have on people using it, and to communicate the beginning of the cognitive chain is the user and how they interact with - or don't interact - with an AI. This could also be expanded upon with more interaction modes than just the brainwave monitor. With a web cam and machine vision setup, body and motion detection can be combined with this to provide a whole-body interaction, mind and movement together in combination with an AI model. In a bento-like layout, you'd see your brainwaves next to the visual they are helping to create (and possibly the body tracking nodes if that is to be included).
Second is the concept of a "non-sentences" project. This actually would not inherently use an AI image generation model, as it is primarily focused towards pulling back and making more apparent the processes of text generation. Gathering a database of words and separating them into where they would fall into a common sentence structure (Articles, nouns/verbs/adjectives/, Prepositions, etc.), and having them semi-randomly form a sentence. This would visualize in an abstract way how text generators work in predicting words and characters to make things that sound like sentences to us, pretty convincingly for the most part too, but also showing where things can go wrong with hallucinations that can half make sense or are just nonsense, revealing behind convincing words and images is just an algorithm at heart. Again, there would be an accompanying visual of the process behind this generation, in the sketch envisioned as a network that links these words together as they are spit out. While there is a structured process, it's nature can still output nonsense, or "non-sentences".
The last sort of sketch that I did not get too far in involved creating a more interactive form of the Emergent Garden, or the Emergent Ecosystem, through things like midi pads or a drawing tablet. I'm on the fence about making this piece interactive however, as some of the idea behind it was making parallels to the unconscious algorithm of biological evolution where the only influence is the underpinning search algorithm of favorable mutations for survival. However, an interactive component could lend to the idea of cultivating curiosity and care, highlighting where our choices cause chain reactions that evolve depending on the circumstances. Regardless, I think the emergent garden poster series as an evolving standalone piece with no interaction can still fit well within my thesis as a conceptual learning piece on what values AI images can have in the context of image space and the evolution and iteration of ideas. That alone would involve a little more work too, as the pieces seen in the video I made involved a little bit of trickery due to time to showcase the concept, having them actually live for viewing will involve a little reformatting.
Overall I think there are some good ideas here that I will have fun exploring into next week. What I plan to tackle first is the "non-sentences" project, as I think it is something I can do relatively fast with the resources I currently have on hand.
What I Read
Reading is taking a further backseat for the actual creation of things, and will probably continue to do so, but I have kept up slightly with "Transcending Imagination", getting through chapters 5-6.
In Chapter 5, Bias and Creative Intent, the book explores how artistic creation is a constant negotiation between skill, intention, and limitation. Every decision: form, color, or concept, involves compromise and reflects the artist’s biases, shaped by experience and perception. Contrasting this with AI-generated art, where algorithms operate free from personal tendencies but are guided by data and semantic intent, forming their own kind of systematic bias, arguing that AI changes the artist’s role from sole creator to facilitator, fostering a symbiotic relationship in which humans guide and interpret machine output. This challenges traditional notions of authorship and originality while expanding the scope of creativity. Bias, both human and algorithmic, is reframed not as a flaw but as a force that can inspire transformation and reveal hidden assumptions within creative intent. this dual-bias is something I definitely would like to explore in future sketches for one of my prototypes.
Chapter 6, Maximizing Creativity, focuses on how AI enhances the articulation of human intent and amplifies creative capacity, discussing how generative systems allow for a deeper and more precise expression of ideas, acting as bridges between abstract thought and manifestation. The author argues that by expanding the language of creation, AI strengthens our ability to communicate emotion, philosophy, and narrative with greater clarity and depth. This chapter concludes that AI doesn’t diminish creativity, but multiplies it, transforming how humans express themselves and reinforcing the shared relationship between technology, thought, and imagination. I think there is some validity to this, but in a way all of AI is predicated on human ideas so to what extent it multiplies human creativity I am unsure, which is an area of thinking my workshop plans to tackle.
Where the Next Steps are Leading
I've outlined a couple next steps already, as they are pretty natural based on where I'm at right now. To summarize, I want to confirm the practitioners using AI that I would like to interview, and I already have a list of people I can reach out to, but there's never any guarantee they will reach back out or be interested in an interview. This is why the goal participant target is 3-5, to account for the possibility of people not responding. Aside from that, my main focus for the rest of the month is developing these prototypes further from their sketches, entering November with solid pieces that I can include in the thesis. Additionally, I will begin to start drafting the written portion of the thesis, starting with literature review and working from there. Nothing super finalized, just putting the pieces in place.
This week I primarily focused on narrowing down my research protocol for expert interviews and the workshop - the workshop including a pre- and post-survey to assess participants outcomes in the interpretability and explainability of AI tools / systems. I needed to do this soon for IRB and for my thesis as a whole, but this also lead me to further thinking about my research questions in a different way, and refining them to something I think is more "sticky" that elicits and reinforces the two "big" aspects of my thesis: The body of work that I'll make, and the workshop itself. I think it was a more natural way of coming up with more refined and solid questions as opposed to sort of brute force thinking of questions for the sake of it: What questions can I answer by interviewing partitioning artists using AI and conducting a workshop centered around playing with AI?
The question I think will be answered mostly by the interviews and workshop is: Can experimental play and creation with artificial intelligence (AI) tools and systems make said tools more interpretable, explainable, and transparent?
I think firstly by learning about the processes behind current practitioners, I can gauge how much they "play" with AI, and what outcomes they've had with it, such as how much they've learned about AI and Technology by making with it, or any hard or soft skills they've gained or deem important when using AI. A preview of two questions:
Describe your process from start to finish, from conception to final output. Explain when, where, and how and what AI was used.
What, if anything, have you learned about AI itself (its limits, strengths, inner workings) from making creative work with it?
Did engaging with AI change your understanding of technology or computational systems more broadly? If so, how?
I'll need to reduce the number of questions for this protocol, I plan to make the final protocol and interview take about 30-45 minutes, so I'll either need to combine or cut some of the weaker questions, the master list so far can be found here.
The pre- and post-surveys for the workshop I think came much easier to me, and assess the outcomes the participants will have in the workshop, such as their knowledge and perceptions of AI and play coming into the workshop, and their reflection on AI systems after the workshop - including a creative reflection and artist statement for their body of work. I think with just a little tidying up these can be good to go, and next I'll have to think about how to deliver them in the final workshop (probably a google form is easiest). The pre-survey and post-survey protocols are linked respectively.
The second research question which I seek to explore in the actual body of work - through prototypes and creative works, still needs a little work. Generally, it would be along the lines of: How can experiential design and artistic practice using AI systems... answer or reveal something about AI. I think there are a couple ways I can go about the ending of the question, such as going along with the first question about making AI more transparent and "giving it back to the people", separating it from the institutions that push out these systems, like we discussed briefly in reflecting on the video I made. An alternate leg to this question could be a sort of investigation in things like biases, distortions, and ethical challenges in AI, which is hard to separate from the medium/material/tool/genre and discussions surrounding it, or it's potential as a teaching tool for different areas (thinking of a cross section between sciences and arts), or as a means to bring awareness to social issues, all of which I think are potentially viable directions to go, and are the basis of previously existing research which I'll cover shortly. I also think the "experiential" design aspect is important to include with what I plan to make, which will include things like Machine Vision and body/hand/face tracking, and ultimately the piece using a brainwave monitor. Another aspect of research I'll cover below.
What I Read
Two papers I read this week parallel to the protocol writing helped me think about my research questions, and they primarily focus on the experiential aspect of AI to explain AI (and brought up a term I didn't know about before: Explainable AI or XAI) and make awareness to larger issues.
The first was simply called Experiential AI, and set up the term “experiential AI” as a way for artists and scientists to come together and make algorithms tangible, visible, and more interpretable. It argues that art can serve as a bridge between opaque computational processes and human understanding, offering new ways to make reasoning processes decipherable. The emphasis on art’s ability to map the “inter-agencies” relates well to my own work and thinking so far, connecting directly to my own focus on embodied and playful experiences with AI—things like tracking movement or brainwaves as an entry point into the systems. This was more of a proposal than a body of work, and was made in 2019, and as such the next paper built on this further.
The second, Experiential AI: Between Arts and Explainable AI, written by the same author(s), builds on that foundation. It critiques the limits of technical “explainability” and suggests that experiential, arts-based methods can go further in making AI interpretable. Their “4As” framework, aspect, algorithm, affect, apprehension, was particularly interesting, and I think can be used in further development of the workshop as opposed to isolated installations or works. It frames experiential AI not just as an artistic add-on but as a methodology: one that connects technical models to embodied human experience and allows people to make sense of AI on multiple levels. Their case studies, like Jake Elwes’ The Zizi Show or Anna Ridler and Caroline Sinders’ AI is Human After All, are great examples of how art can expose bias, hidden labor, or socio-technical entanglements in AI.
Reading these papers on experiential AI helps me situate my research in context of what's going on right now in similar veins, where I'm not just making AI “explainable” in a narrow sense, but in creating experiences, through interviews, workshops, and creative works, where participants can get a more hands-on experience with AI through play and alternate interaction, and, in that process, make its mechanisms and implications more transparent.
Where the Next Steps are Leading
Immediately next I need to finish up the protocols to submit to IRB and get interviews going this month as soon as possible. Like I said the interviews will be valuable to know what practitioners have learned about AI themselves through their own experiences in working with AI on their own and for longer than the participants would have, assuming they come into the workshop with 0 knowledge. After submitting the protocols, in the time it takes for IRB to look over it, is when I plan on working on these experiential prototypes, starting out with sketches and then moving into the actual making.
//about
Ryan Schlesinger is a multidisciplinary designer, artist, and researcher.
His skills and experience include, but are not limited to: graphic design, human-computer interaction, creative direction, motion design, videography, video-jockeying, UI/UX, branding, and marketing, DJ-ing and sound design.
This blog serves as a means of documenting his master’s thesis to the world. The thesis is an exploration of AI tools in the space of live performance and installation settings.
un-prompted 