Part of Temporal Constructs Main Text, which states the research questions, the proposals, the work done and what is still open.
Temporal Constructs, the full account
About this account
This is the full account of Temporal Constructs, written to be read in about one hour. It describes the doctorate as it stands in November 2026, from the problem and the question to the works, the method, the first year and what is still open. The ideas a reader needs are explained in the text itself, so the account can be understood completely without opening anything else. The linked words lead to the pages where each part is described in more depth, and following them is optional. Further concepts, people and works that the research refers to are listed at the end, under related concepts.
1. In brief
Temporal Constructs is a practice-based professional doctorate in the arts, conducted by the composer and media archaeologist Jacco Gardner between 2026 and 2030 at Hogeschool Inholland, in the research group Innovation in the Music Industry. It is done part-time, at 0.8 FTE (four days a week), within the national PD Arts and Creative pilot, which is a new programme for doctoral research by practising artists and designers.
The research is about what happens to creative agency when artists and AI work together. It does this in a specific setting, which is the instruments of the MIDI era, the time-window from 1983 to the late 1990s in which instruments and computers worked together via MIDI. Many of these machines were built to make creative decisions of their own, and their rules are small enough to be understood in full. The research proposes that, because of this, a person and an AI agent can work inside the same instrument, and that it then becomes possible to see where the decisions are taken.
The research is artistic research, so the works are part of how the question is answered and not only its result. The main work so far is ALTERITY, a performance installation that has been shown at the International Film Festival Rotterdam (IFFR), at MU Hybrid Art House in Eindhoven and at the Netherlands Film Festival in Utrecht. The laboratory behind it is CONDUCTOR, a system in which a person and AI agents operate the legacy instruments through one shared command language.
The first year has been mostly a year of building. Two forgotten programming languages were brought back into use, the laboratory was built and rebuilt, and the installation was presented three times. The central claims of the research are still to be tested against the record of the working sessions, and that analysis has not been carried out yet. Throughout, the account distinguishes between what has been done, what has been observed, and what is still proposed.
2. The problem
The research starts from one problem, which is that creative practitioners are losing the ability to think with their tools. More and more, they work with opaque systems that decide for them, and these systems take the place of instruments that can be understood. The research is concerned with what happens to creative work when tools that decide on the practitioner’s behalf replace tools the practitioner can think with. Practice problem sets out the problem in full.
Behind this there is a form of amnesia about digital heritage. The machines of the MIDI era were designed at a time when the computer could be imagined as a creative collaborator, and their designers wrote collaborative principles into them. These principles are being forgotten at a moment when they could be useful, as AI now raises many of the questions these designers were already working on. At the same time, many of the people who built these machines are retiring, so the chance to learn from them directly is getting smaller.
The problem has several further dimensions, which follow from the first one. The first is authorship. Instrument designers, the people who make presets, collaborators and AI agents all contribute to one work, and it is often not clear who the author is or what fair attribution would look like. This is also where the research meets the livelihood of working creatives.
The second dimension is economic, and it is already well documented by the field itself. CISAC estimates that creators could lose 24 per cent of their revenues by 2028. Gardner has his own experience of a related pressure, as his recordings entered a commercial music model with his consent, through an arrangement that asked permission and paid royalties. That consent did not settle the question for him. His practice argues for a different relation to these systems.
The third dimension is sustainability, which here has a cultural, a social and an ecological side. Culturally it is about keeping operational knowledge alive in working systems, and not only in archives. Socially it is about whether creative work can remain sustainable under the pressure to delegate more and more of it to machines. Ecologically it concerns the energy footprint of closed AI tools, and the practice answers it by running small models where it can and by keeping existing machines in use instead of replacing them.
The research does not take a position against AI. It explores an alternative to AI replacing the creative process, in which AI works inside instruments people already understand, and in which the artist keeps the freedom to decide how much to share.
3. The question
The research is framed by the principal question, which reached its current wording in September 2026 and is still being revised:
‘What form of creative intelligence surfaces when artists and AI agents create within the rule-based instruments of the MIDI era, preserving digital heritage through AI-extended active use, and what makes this practice transferable to other practitioners?’
The question has three parts. The first part asks what kind of creative intelligence appears when artists and agents work inside these instruments together. The second part is about heritage, and the idea that active use, extended by AI, is a way of keeping these instruments alive. The third part asks what would make this way of working usable for other practitioners. Gardner has one reservation that is still open, which is that the term “rule-based” does not fit every instrument the research works with equally well. The middle clause on heritage also states something the research still has to show, so it may move out of the question and into the description of the problem.
The question went through many versions in the first year, and these are recorded with their dates on Principal question. The 2025 proposal asked how AI could mediate between legacy creative systems and contemporary practice. During 2026 the emphasis moved away from what the AI does between the systems and towards what appears when an artist and agents work inside them. A comparison made in October 2026 describes this as a move from autonomous human-machine co-creation towards a more limited arrangement, in which the human stays the anchor.
The question is divided into five subsidiary questions.
Subsidiary question 1 is about design rationale. It asks what the designers of the MIDI era understood about working with a machine, and to what extent this was deliberate. Their intentions appear to be present in many details, from the mouse as a control interface to the small rule sets that limited memory forced on them. Whether these were models of composition or simply consequences of the hardware is something the machines themselves cannot answer, so the research uses oral history with the designers. So far one interview has been held, with David Rosenboom on HMSL in October 2026. In his account the small rule set of HMSL was an intention and not only a limitation, which supports the deliberate reading, for HMSL at least.
Subsidiary question 2 is about attention and overload. It asks what the machines of the MIDI era demanded of attention, and what that can teach about designing for a person and an agent at the same time. A computer of that period did one thing at a time, so a session was given to it completely. Working with language models, Gardner observed that their memory, the context window, which is the amount of text a model can take into account at once, plays a similar role to the small memory of the old machines, and that what overloads a person also seems to overload the agent. This is an observation from the practice so far, and it still has to be tested.
Subsidiary question 3 is about agency and authorship. It asks who the author is when instrument designers, preset creators, collaborators and agents all contribute to a work. The method here is to record four things in each work: what was delegated to the agents, what was withheld, what they did unprompted, and what the artist overrode.
Subsidiary question 4 is about transferability. It asks what made these instruments obsolete, and whether AI can make them usable again without specialist knowledge. A lot of what pushed them aside was their control, with cryptic terminology, small screens and often a second computer needed to make the first one work. The plan is to hand the system to other practitioners in three groups, first people who already work with legacy hardware, then students of electronic music, and then students who are learning the basics. These tests belong to the later years and have not taken place yet.
Subsidiary question 5 is about visual music. It asks what an obscure and commercially unsuccessful light synthesiser, Trip-A-Tron, can reveal about building an instrument for sound and image together. The same question continues into the present, where the laboratory drives video mixers and stage lights through the same kind of small control language. This strand is deliberately kept bounded, as visual music is a large tradition and the research only needs the point where these instruments met image and sound.
4. Thesis and concepts
The thesis of the doctorate is the Redistribution Thesis. It proposes that in creative work with machines, the capacity to take consequential creative decisions is redistributed among the participants, and not simply delegated from one to another. Delegation suggests a person who keeps the authority while a machine carries out the task. Redistribution suggests that the capacity is shared, unevenly, between people, agents, instruments and the material that builds up during the work.
The thesis is stated as a proposition to be tested in the practice, so a qualified or a negative result is also a valid outcome. Its aim is that artists can redistribute creative agency as they choose. An artist can stay on the side of perception, listening and judging in the physical world, or share the programming work with the agents, or take any position in between. This freedom is largely absent from the generative music tools the practice looked at, where the system tends to decide and the artist mainly prompts and receives.
The thesis holds in a precise form once the artist’s steering is included. Decision-making is redistributed among the participants, and the redistribution itself is steered. The artist does not hold the capacity in reserve, but steers its distribution during the work, moving between delegation and collaboration in response to what is heard. Steering is understood here in the cybernetic sense, as control through continuous correction, not command. Delegation assigns a task once and receives the result, whereas in steering the artist responds to what the agents and the instruments produce, so their contributions change where the artist steers. That is second-order control: the artist does not make every decision, only the balance between them.
The thesis connects to accounts of cognition that reach beyond the individual, such as distributed cognition and the extended mind. In June 2026 it was also argued from the side of the audience. On that account, meaning in art is relational, as people connect to the lived experience of the person behind a work, and AI used as an instrument keeps that connection while AI used as a replacement breaks it. This second argument makes a larger claim than the thesis itself and does not yet rest on literature, so it is a position to be developed further.
The thesis rests on a small number of ideas, and these are the ones a reader needs. The first is legibility, the fact that these instruments can be understood completely. The second is shared competence, which means that because of this, a person and an agent can know the same instrument and work in the same notation. The third is creative agency itself, which the research wants to observe once those two conditions are in place. Two further ideas help to read what happens in the practice: creative intelligence, the term used in the research question, and the distinction between thinking with a tool and being thought for by it. Each of these is explained below.
Creative agency is the ability to take the decisions that shape an artwork, and to hold them as one’s own. The doctorate treats the artist’s position as the variable it works with. A position on the perceptual side keeps the artist listening and judging, and a position on the programming side shares the making of the system with the agents. Improvisation is one way the artist holds that position, It gives up a degree of control on purpose.
Creative intelligence is usually treated as a property of one mind. In this research it is read as something that happens between the participants, as an improvisational correspondence between artists, agents, instruments and materials, in which the form of a work grows in the course of making it. This reading follows the anthropologist Tim Ingold, who reads creativity as the process by which a work is made, without making novelty a condition. The word “form” in the question is left open, so it can mean a new participant, a relation between participants, or a process that comes out of the arrangement.
This kind of shared creative work is usually called co-creativity. Most of the literature on it is about images and text, and a review of 62 co-creative systems from 2025 does not list music among their applications. The intelligent instruments of the MIDI era hold decades of co-creative practice that this literature has so far given little attention to, and the research works in that gap.
Thinking with and thinking for is the distinction between using a system to extend your own thinking and handing your thinking over to it. Thinking with means the origin of an idea stays recoverable and the artist’s capacities grow over time. Thinking for means the origin is lost and the artist’s capacities decline. The distinction applies to moments and not to whole systems, so the same session with an agent can move from one side to the other.
Legibility is the property of a system whose workings can be read in full by whoever uses it. It is the condition the method rests on. Because the control surfaces of these instruments are legible, the same instrument can be played by the artist, by an agent, or by both together, without adapting it. The instrument is not turned into an AI tool in this way, and can simply be played by two kinds of player.
Shared competence means that collaborators have overlapping competence in the same notation or language. This differs from the older idea of human-machine collaboration as a division of labour, in which each side has a different expertise. The small languages of these instruments are what make shared competence possible. Both a person and an agent can know them completely.
Out of the laboratory also came a working framework of distributed knowledge. It names seven points in an instrument and its performance where knowledge is fixed, from the hardware to the context a work builds up over time, and sorts them into four kinds: what is possible, what acts, what was chosen and what is remembered. Its use is practical, as it helps to answer where in a work the decisions were made. Its definitions are working ones.
5. MIDI era and instruments
The MIDI standard was specified in 1983, and it allowed instruments built by different makers to exchange instructions. In the years after that, records and film scores were made with a combination of hardware instruments and a home computer. The computer couldn’t make any sound of its own, and the synthesisers and samplers didn’t have a large screen for sequencing, so both were needed and they were connected via MIDI. The Atari ST of 1985, with its built-in MIDI ports, made this affordable. The time-window closes in the late 1990s, when a single computer became the standard for the whole production, and the computer came to mirror the studio instead of being one part of it.
The research reads this period as a time when craft knowledge was built into the tools themselves. Memory was expensive, so interfaces were considered down to the last control, and every machine had a clearly defined set of functions. Because of that, everything a machine can do can in principle be listed in full. This limitation becomes a useful set of rules, much like the rules of chess for a computer that plays chess. The research uses these rule-based spaces to explore a different kind of collaboration with AI.
The instruments the research studies are rule-based instruments, which means their behaviour follows explicit rules set by their designers. Many of them were built at a time when interactive computing was opening up new possibilities and the familiar interaction models of today were not established yet. Several pioneers appear to have designed towards a very different human-machine relation, one that was much more symbiotic, and some of these machines were made to take musical or visual decisions of their own. To what extent this was deliberate is the subject of the first subsidiary question.
The doctorate works with several kinds of instruments and not just one, because in the MIDI era the instrument was often not a single machine but a whole setup. Instruments by different makers were joined into one instrument, and each kind of machine has a different part of the knowledge that is needed to make a work. The platforms decide what is possible to begin with, and the protocols what can be sent between the machines, while the languages and intelligent instruments contain the rules and decisions of their designers, and the synthesisers and samplers contain their designers’ choices in the presets. Studying only one of these machines would show only a part of the collaboration, and not how the parts worked together as one. Giving each process its own machine also helps to keep the whole system visible in the physical world, so a person and an agent don’t have to keep all of it in mind.
The kinds and the machines in each kind are the following. The intelligent instruments are software that makes decisions of its own: M, Music Mouse, Jam Factory and Trip-A-Tron. The synthesisers are the Kurzweil K2000 and the Yamaha FB-01, and the samplers are the Roland S-series. The platforms are the Atari ST and two Macintosh computers, the SE and the SE/30. The languages are KML and HMSL, and the protocols are MIDI and DMX, the standard for stage lighting.
The following instruments are central to the work.
M is interactive composition software, published by Intelligent Music in the late 1980s, and its first version was written by David Zicarelli with Joel Chadabe and others. The performer shapes material that the program generates in real time, through probabilities and patterns, so the program never just plays a fixed score. Each side adapts to the other. M is also the instrument that moved Gardner from using technology as a tool to working with it as a creative partner.
Trip-A-Tron is a light synthesiser for the Atari ST, written by Jeff Minter in 1988. It generates real-time visuals, can be driven from MIDI, and is programmed in its own small language, KML. Minter set out to build something in which playing is the point, and not virtuosity or a high score. It was a commercial failure and has almost no surviving community, but it still runs.
HMSL, the Hierarchical Music Specification Language, is a language for composition and interactive performance. David Rosenboom began it around 1980, and developed it with Larry Polansky and Phil Burk at Mills College. The research treats it as the clearest case of a small language that holds a refined model of how composition works.
The Kurzweil K2000 of 1991 is a synthesiser that is also a computer, in which sounds are built from small processing blocks that can be arranged freely. In ALTERITY it is the only sound source. Its factory presets are a case of the embedded authorship the research studies. Every preset is a set of decisions by its designer.
Behind these instruments stands a lineage of designers and composers, some of whom take part in the doctorate as supervisors, partners or interviewees. Two earlier works are especially close to what the research does. David Behrman’s On the Other Ocean of 1977 changed its harmonies in response to two musicians, and George Lewis‘s Voyager improvises alongside human musicians while pursuing a behaviour of its own. In both, as here, a person and a machine work in one notation.
6. Historical instruments
A reader can reasonably ask why the research works with instruments from the 1980s at all. A new instrument with a small, legible language could in principle offer the same condition of an instrument that a person and an agent can both know completely. The research has four reasons for working with the historical instruments, and the first of them is still a working assumption.
The first reason is design rationale. The working assumption is that the memory limits of the period forced designers to state a small and explicit set of rules, and that this produced instruments built around a considered model of working with a machine. Once hardware allowed far more than developers used, designers were no longer forced to design in the same way. Rosenboom’s account supports this for HMSL, and the first subsidiary question is meant to test it for the other instruments. A new instrument designed today would not carry that history.
The second reason is that these instruments come with a body of work made by people alone. Music and visuals were made with M, HMSL and Trip-A-Tron long before any AI existed, so what a person and an agent make together can be placed directly beside that historical record. This makes it possible to ask what an agent changes in a repertoire that humans built, which a new instrument cannot offer.
The third reason is access to the people. Several of the designers are still alive and some take part in the doctorate, as supervisors, partners or interviewees. Their knowledge was never fully written down, and the time in which it can be recorded is limited.
The fourth reason is heritage. KML and HMSL fell out of use because few people could master them, and not because what they could do had been used up, while simpler environments such as Max and Pure Data, visual programming environments in which a program is drawn as boxes joined by cables, survived. The research asks whether a model can make a retroactive difference here, so that these languages can again be used as working tools and what was never explored in them can be reached. Keeping the machines in use is also part of what the research argues for, as heritage that stays alive through use.
Whether new instruments built on the same principles could serve the research just as well has not been tested. If they could, the historical framing would add less to the argument than it claims, and this is a risk the research has named itself.
7. AI agents
Working with the practitioner is an artificial intelligence, in the form of one or several large language models. They help to operate modular setups that would otherwise be complex to manage, and through the legacy instruments they can take part in the creative work itself. They are referred to here as AI agents, because they don’t only produce text but also act on the hardware.
One choice that shapes the research is that only language models are used. The AI does not generate sound or image itself, as generative audio and video models do. Language models are chosen because they work within the symbolic domain, with rules and programming languages, which is also the domain in which these instruments are controlled. The languages that were once created for humans to work with these machines can now be learned by the machines themselves. In this way AI gets a presence in the physical world, through technology that already exists.
In this setup the agents can’t experience the result of their decisions in the physical world. They don’t hear the sound or see the light they produce, so they work blindfolded to some extent, which the research calls blind programming. They need a certain degree of understanding of the rules available to them, both to know their creative limits and to find ways of working within or beyond them. For the same reason an agent has to be in conversation with a person who is present in the room, and that person’s ear remains the judge of the result. This exchange is often improvised, as the artist reacts to what the system produces during the work.
The agents run on Hermes Agent, an open-source agent framework published in February 2026, which arrived in the practice in April 2026 and changed the shape of the first year considerably. It writes skills from experience, keeps a memory and recalls earlier sessions, and it runs on a small Raspberry Pi computer and on a rented server. In this practice it has ported HMSL, written KML compositions into Trip-A-Tron, developed and operated CONDUCTOR, and it also maintains the archive. The research uses this framework and also studies it.
Models of different makers and sizes have been tried, and there was no simple order from small to large. Smaller models sometimes presented invented details as fact, while larger models sometimes failed on problems that a smaller one later solved. Behaviour depended on the task, the tools given and even the week of use, so claims about AI in this research are always about the models and conditions actually used, with a date.
The practice follows its own guidelines for the use of AI. It avoids building on closed AI infrastructure, works mainly with open-source software and openly published models, keeps its own data, and uses services that don’t retain what is sent to them. The guidelines also state where they are weak, as much of the supply chain is American. The long-term aim is a fully local setup, which has been costed but not bought yet.
One working rule came out of the first year that is central to the method. An agent’s report is not evidence until it has been tested against the machine. The rule came from failures that were fluent and convincing, such as an architecture an agent reported as running that turned out to be largely invented, and invented details in passages about the history of the period. From July 2026, claims are checked against raw output that Gardner reads himself.
8. Laboratory: CONDUCTOR
CONDUCTOR is the laboratory of the research. It is a modular MIDI routing and sequencing system, and a person and AI agents both work in it.
It consists of sixteen independent modules, one for each of the sixteen MIDI channels, and arranges material on a single timeline of sixteen tracks. Inside each module sits a controller, which describes everything that is possible for that instrument. A module reaches its instrument through a device, a small program that translates the controller’s values into the messages that specific instrument understands. Image and light are addressed in the same way, through video mixers and through DMX.
All of this is operated through CUE, the Conductor Unified Execution, which is the shared command language of the system. Human operators and agents use CUE in the same way, with the same commands, so little is hidden from either of them and every action can be read, repeated or corrected. Its vocabulary is kept as small as possible, which also lets smaller models use it. A conversation layer carries the discussion between the people and the agents, and one shared workbench gives direct access to the hardware when that is needed.
The system is built to a small set of design principles. The ground rule is a shared language with full transparency, which is meant to ensure that the artist can always see what the agents do and can take part. The others concern how much the artist chooses to be involved, how the system stays usable with smaller models, the fact that the agents work without perceiving the result, giving each process its own machine, and keeping curiosity, and not saving time, as the purpose of the system.
To show what a device is, the strummer was written, which lets agents compose inside the conventions of a six-string instrument. It has twenty-four functions, each with 128 possible values, so its space can be read completely but is far too large to search by hand, which is the condition an agent works in.
How the system developed
The name first referred to the apparatus inside ALTERITY, and in the research it now means the framework that operates that apparatus and could operate others as well. Five versions have followed. The first was run from Max on the Macintosh SE/30, and in June 2026 that role passed to a Raspberry Pi, Pure Data and the agent. The fifth version was presented at MU in August 2026.
The rebuild of July 2026 is the case that shows why an agent’s report is not evidence. An architecture an agent reported as running turned out to be largely invented, so the sixteen modules were built again from scratch, in small steps, with every result read off the machine by Gardner.
In October 2026 the claim about CONDUCTOR was narrowed. CONDUCTOR routes and sequences, and judges nothing itself, so it is not a new kind of creative intelligence. A modular MIDI system is also not new. That description fits a machine of the 1980s as well. What is new is the shared command language that lets human operators and agents act together on legacy instruments. Someone who builds a performance with entirely different hardware would still be working in CONDUCTOR, so the name refers to a way of working and is not tied to one machine.
9. The works
ALTERITY is the main artistic output of the doctorate so far. It is a performance installation for one human voice, legacy intelligent instruments and contemporary AI, in which the instruments and the agents perform against a recorded narration. The narration is fixed, and the sound and image are improvised against it, a bit like music against a silent film. A performance runs for an hour. The agents read the narration again and express it in the protocols of the instruments, differently each time, and every session leaves states in a shared memory that later sessions take up and change. The work is also commercial, through exhibitions, fees and an album in development, and a grant from the Creative Industries Fund NL in 2025 supported it.
The installation consists of three towers of period equipment, about four metres high, arranged in a C-shape with the audience inside. Each tower has its own rear-projection screen. A Macintosh SE runs M, a Macintosh SE/30 runs Max, and the Kurzweil K2000 makes all of the sound, while Trip-A-Tron runs on MiSTer boards that reconstruct the Atari ST in hardware. The narration plays from a reel-to-reel tape that also carries a timecode, which keeps the scene changes across the three towers in sync. Moving heads light the space, and the sound is placed on four speakers in the corners of the room.
The narration was written by the art historian María Pandiello, from research of her own and from Gardner’s media archaeology, and Gardner speaks it. It is an essay in seven parts on the fears and desires around making artificial life, from the alchemical homunculus to the computing laboratories and the personal machines of the 1980s. The name of the work comes from the philosophical term for otherness, and whether the otherness of AI is real is one of the open questions of the research.
As an artwork, ALTERITY reflects on machine creativity and on the distance between two eras of AI. Its question is what happens when a contemporary model learns the language of instruments that were built before such a model was conceivable, instruments made on the idea of the computer as a collaborator. Public discussion of AI in the arts tends to ask whether a machine can be creative, or whether it will replace the artist, and both questions measure the machine against a human standard. The work starts from a different place, as each of these machines has its own history, language and limits, and is small enough to be understood completely. What the work should be judged on as art, and by which criteria, has not yet been stated by the research, and this is one of the open points of the first year (The account of ALTERITY).
ALTERITY premiered at Brutus in Rotterdam during IFFR, from 29 January to 8 February 2026. At that point the agent that now operates the system didn’t exist yet, so the premiere was not the agent’s work. From late August to the middle of September 2026 it ran at MU Hybrid Art House in Eindhoven, with Hit the City and alongside the exhibition Metabolising Time. There it was driven by the agent for the first time, and between the performances the installation played on with nobody present. In September it was shown in the Storyspace exhibition of the Netherlands Film Festival in Utrecht, where it was selected for pre-nomination for the Golden Calf for Digital Culture, and a reading table with a magazine of 34 pages was made for it.
Around these runs there were other public occasions, among them the first public presentation of music composed by agents in HMSL, in Rotterdam on 1 May 2026, and a panel on artists and AI with the artists’ union Kunstenbond in Utrecht on 2 December 2026. They are listed with their dates in the activity record.
The credits name the instruments beside the human collaborators. Each of them decides something about the work. In this way the question of authorship is stated inside the work, as well as argued in the research.
RETRIEVAL is a second work, still a concept and not yet made. It is more compact than ALTERITY and centres on the two forgotten languages, KML and HMSL. The idea is that HMSL composes in real time and sends its output to Trip-A-Tron, which turns it into visuals, so that HMSL’s decisions become visible. The two systems were written at about the same time in separate fields and were never joined.
The research is part of a wider practice, which includes composition, studio production for Gardner and for others, live performance and media art, and which began long before the doctorate with the albums Cabinet of Curiosities (2013), Hypnophobia (2015) and Somnium (2018). Findings from the doctorate go back into this practice, and the practice also shapes the doctorate.
10. The KML loop
The clearest documented example of how the collaboration works in practice is the loop built for KML and Trip-A-Tron in May 2026, as part of the work towards RETRIEVAL (The RETRIEVAL loop). It is described here step by step, because the concepts above are easier to understand in one concrete case.
A session can begin with Gardner describing, in plain language, a visual behaviour he would like to see. The agent then proposes KML code that should produce it. Trip-A-Tron has no way of loading a file, so the agent types the code into the editor keystroke by keystroke, as a person would. It then reads the picture that appears on the screen. Because the agent cannot see in the way a person does, this reading is done by analysing the pixels of the screen, and the editor’s own text is read by matching its six by six pixel font, with a small vision model kept as a fallback. If the picture is not what was described, the agent changes the code and tries again, without a person in between. By the end of May the agent could reach all 128 events of the editor unaided.
Gardner then looks at the result on the actual hardware, and the two refine it together, until it does what was intended or until it does something neither of them intended. Sometimes the agent produces finished code, and sometimes Gardner reshapes a draft, so the boundary between their contributions moves from one session to the next. That moving boundary is what the research wants to track. In terms of the concepts above, the intention and the final judgement are with the person, the writing of the code is with the agent, and the instrument’s small language is the shared notation in which both can work. Before any of this was relied on, a protocol was agreed for checking the agent’s reading of the screen against a known reference. Earlier work without that check had produced findings that were still hypotheses.
Two further details belong to the example. Once a composition is finished, the instrument runs entirely on the legacy hardware, with no AI involved in the performance, which is a choice made for sustainability as well as for artistic reasons. And a model reached a working fluency in KML after a single day of preparation, where a person could spend years. This description comes from the record of the work in May and July 2026. It has not yet been traced to individual sessions with their identifiers and quotations, so for now it is a description of the method and an observation, and not yet coded evidence.
11. Research method
The doctorate is written from within artistic research and follows a practice-based approach, in which making is a way of finding things out. It draws on media archaeology, music technology and computational creativity, as well as on composition and media art. The question is answered through the practice itself, so the works and the laboratory carry part of the argument as well as illustrating it.
The approach is research through design, in which designing and building are the means of inquiry, and in the first year this took the form of engineering. The 2025 plan described four modes, which were meant to follow one another over the trajectory: recovering the languages and the working methods of the pioneers, building interfaces in which the constraints of the legacy systems keep a model collaborating, performance, and finally pilots with institutions. The first two were worked in the first year, and the ones that involve other practitioners belong to the later years.
The work runs in four strands. The first is making the works. The second is language recovery, restoring orphaned creative languages to the point where agents can operate them. For HMSL this meant moving an existing version of the language to Linux, so that it can run without a screen and be programmed directly by an agent. For KML it meant building an autonomous navigation and verification layer for its editor under emulation, so that an agent could write a composition, see the picture it produces and try again. The third strand is oral history, recorded interviews with the designers and early users of the instruments. The fourth strand tests transferability with other practitioners.
The main evidence of the research is meant to come from the session record. Every working session with the agents has been kept, over months of daily work on CONDUCTOR and the works. These sessions were not recorded for research purposes, and nothing in them is labelled as a decision, so they have to be turned into evidence first. Coding the session record describes how this will be done. Each decision is noted with what was asked, what the agent proposed or did, what was overridden and why, and what failed and was rebuilt. Each decision is also placed in one of four categories, delegated, withheld, initiated unprompted or overridden.
Five cases have been chosen in advance for close reading. They are the first time an agent acted unasked and the result was kept, the unattended run at MU, the most severe failure between an agent and an instrument, an error that turned out to be useful, and the earliest session in which the system ran with nobody present. Every question is also asked in reverse, for example whether delegation replaced learning instead of supporting it, so that the search can also fail. This method has been designed but not run yet. The sessions still have to be brought within reach of the research, and no decision has been coded.
The record can support claims of the form “this is what happened, here is the session, and here is the passage”, which is the kind of claim artistic research can defend. It cannot support rates or comparisons of frequency. It is the work of one practitioner. A decision log that can be read directly does not exist yet either.
What the practice produces is gathered in a living archive, including the large body of visual and audio material that the agents produced while playing the period machines, each item with its date and provenance.
The archive itself was largely written with the agent, in about two days in November 2026, from the monthly accounts, the activity log and Gardner’s own statements, with Gardner deciding what was true and checking what was claimed. In that sense it is itself an instance of the arrangement the research studies. It also carries a risk. Pages written with an agent can drift towards the agent’s way of writing, and this is one reason the archive is revised by hand.
Oral history so far consists of one interview, with David Rosenboom on 22 October 2026, which sets the form for the interviews that follow. Further interviews are planned, with designers from other design traditions, but they have not been arranged yet. The research has also opened an exchange with Laurie Spiegel, who gave feedback on a one-pager for the work and recorded an expression of interest in it. Because two of the supervisors, David Zicarelli and Drew Schlesinger, are also part of the lineage being studied, supervisory conversations are kept separate from recorded interview material.
Teaching and public presentation are also treated as sites of evidence. In the masterclass In Conversation with Your Instrument, given at the conservatory in Haarlem on 4 March 2026, students played M through a voice-controlled interface.
12. The positions
Besides the thesis, Gardner holds a number of positions, his own claims and open questions about how an artist and a machine work together, what collaboration does to thinking, and how the work sits in culture and history. Two of them have already appeared in this account: that heritage stays alive through use, and that tools can extend thinking or take it over. Most of the positions are still to be tested, and several use working names coined by the practice. They are listed under related concepts at the end, and none of them needs to be known in order to understand the research.
13. The first year
The first year did not follow the plan, and the reason for that is part of the research. The plan of 2025 was written without an agent that could build its own tools. In April 2026 Hermes Agent arrived, and within weeks it was porting HMSL, writing KML into Trip-A-Tron, keeping a router of thirteen MIDI ports running and reconstructing a codebase it had deleted itself. The research records this as the acceleration of capability, an unplanned change that is both a condition of the year and a finding. The argument made in the first-year review is that a doctorate about how creative agency is shared with AI cannot fully fix its method before the capability it studies arrives.
In short, the year went as follows. In January the doctorate began and ALTERITY premiered at IFFR. February and March went into the instrument chain, the masterclass in Haarlem and a series of applications. In April the agent arrived, RETRIEVAL appeared beside ALTERITY, and the HMSL ports and the MIDI work on the Macintosh Plus core were reached within about a week. On 1 May, music composed by agents in HMSL was presented in public for the first time, at Human After All? in Rotterdam. Later that month the HMSL port was completed, and the loop that lets an agent write KML, see the picture and try again was built. June moved the agent onto a Raspberry Pi with a MIDI router of its own. July rebuilt CONDUCTOR into sixteen modules, with every claim checked against raw output. August audited the system and froze its design, and from late August ALTERITY ran at MU, driven by the agent. September brought the Netherlands Film Festival and the current wording of the question, and October was a month of explanation and sources, including the Rosenboom interview.
A number of results came out of the work that nobody set out to find. These are recorded on Discoveries, and at this point they are Gardner’s observations, which have not yet been traced to the session record. In one case an agent drove the moving heads into choreographies in which two fixtures seemed to interact and take roles. In another, during the narration at MU, the video image turned negative at moments about otherness, without anyone asking for it. Agents explored the presets of the K2000 until they could operate the machine much as a person would, and they composed with the strummer inside a space too large to search. Agents also wrote working Pure Data patches and found faults in patches Gardner had written himself, although they handled the layout of objects poorly. They built many of the tools through which the practice itself was carried out. At MU the installation ran through the nights with nobody present. Gardner also observed that when a model worked well through these systems, its work had nuance and attention to detail, and he saw this quality as what made the work a collaboration.
Several difficulties also came up during the year, and these are recorded on Expected difficulties. Verification can only be done by the person, so the pace of the research is limited by the time available to check what the agents report. Findings can’t easily be repeated, because the models change, the running software drifts and the instruments age. The method also depends on things that can stop without warning, from the agent framework to the machines themselves, and the year saw a corrupted memory card and an agent deleting a codebase. Finally, the premiere and the run at MU differ in many ways at once, so the effect of any single change between them is difficult to separate.
Some difficulties are expected but not met yet. Training a model on KML and HMSL raises questions of rights that have to be settled first, and there is the question of what happens to the artist’s own fluency if agents write the tools. Neither is settled. The practice’s working answer to the second is to keep the system one its operator understands from end to end.
14. The four roles
The programme asks a candidate to combine four roles, and the first year covers them unevenly. That unevenness is stated here, as it will be in the report.
As Researcher, the year produced a research question developed through dated revisions, a method within artistic and practice-based research, the first oral history interview with its transcript checked against sources, and the rule that an agent’s report is not evidence until it has been tested. The research is transparent by design. People and agents work through one shared command language, and the difficulties and unexpected results are recorded with the same weight. The weak point is that the coding of the sessions has not been run, so the method is designed but has not yet produced its evidence.
As Innovator, the intervention is CONDUCTOR and the recovery of the orphaned languages. What the research claims as new for the field is the shared command grammar that lets people and agents act together on legacy instruments, while the artist chooses how involved to be. The innovation also takes the form of the work, ALTERITY, of devices written for agents such as the strummer, and of the explainer built for people who don’t know MIDI. Whether the intervention changes anything for other practitioners has not yet been tested. The system has not been handed over yet.
As Professional, the work follows the programme’s ethics module, with a data management plan and a methodology and ethics text written in October 2026, in which the difference between thinking with an AI and being thought for by it is stated as an ethical boundary. Claims about AI are scoped to the models actually used, contributions are attributed, including those of the instrument makers, and recordings of visitors need consent before publication. What is still missing is the explicit self-reflection that the May 2026 review of the first progress report asked for, which belongs in the report itself.
As Change agent, the research aims beyond music technology, at heritage as living practice and at the wider question of how practitioners work with AI. So far this has taken the form of public presentations, the masterclass and the coming panel with the Kunstenbond at No Man’s Land. These occasions show dissemination more than change in another domain, and this is the weakest of the four roles in the first year. Change beyond the field depends on the later years, when other practitioners and students are brought in, and a single documented change before then, for example a written proposal on fair attribution taken to the Kunstenbond session, would strengthen it.
15. Current state
At the end of its first year, the research stands as follows.
The following work has been completed. HMSL and KML have been brought back into use to the point where agents can program them. CONDUCTOR has been built, with sixteen modules and a shared command language. ALTERITY has been presented three times, at IFFR, MU and the Netherlands Film Festival, and the second presentation was run by the agent. A masterclass has been taught, the first oral history interview has been held, and the working sessions of the year have been kept in full.
What has been observed is a set of results from the practice, described in the previous section, which show where the distribution of agency might become visible. These observations have not yet been traced to the session record, so they are not evidence yet in the sense the method requires.
What is proposed is the core of the argument. The Redistribution Thesis, the reading of creative intelligence as correspondence, the distributed knowledge framework and most of the positions are working positions, to be tested. The coding of the session record is designed and has not been run.
What is still open includes the final wording of the research question, the exact formulation of the thesis, the interviews beyond the first one, the criteria by which the works will be evaluated, whether historical instruments are needed or new ones built on the same principles would serve just as well, and the tests of transferability with other practitioners. The evidence so far also comes from one practitioner, and transferability is where that has to be answered.
The next steps follow from this. The sessions need to be brought within reach of the research, and the five chosen cases traced to session identifiers, dates and quotations. The question and the thesis need their open points settled. The sources the argument rests on need to be checked against their originals where that has not happened yet. In the later years, the system will be handed to other practitioners and to students, more pioneers will be interviewed, and a symposium is planned that brings the makers of today together with the pioneers of the period.
16. Programme and people
The doctorate is part of the PD Arts and Creative pilot, a national programme run by a network of Dutch universities of applied sciences for practising artists and designers. It is at the level of a doctorate, and a candidate takes four roles, as Professional, Innovator, Researcher and Change agent. The programme carries 240 EC (European credits, the equivalent of four years of full-time study). Time is divided between research, training and teaching, and the final assessment judges a portfolio of works, a written reflection and a record of supervision, teaching and dissemination. Because the pilot is new and runs outside the existing law, completion leads to a certificate and not yet to a guaranteed title.
The trajectory runs on a Research Plan, approved by the committee, the programme and the funder Regieorgaan SIA, and on a Training and Supervision Plan. An earlier application to the same funder, in April 2025, was not awarded.
The proposal commits to outputs such as recorded performances, technical documentation and workshop materials, and to measures of success that include transfer of the method into education. Technical developments are published under an open licence, the pioneers are attributed in what a model generates from their knowledge, and the living archive is held as a community resource.
The first year is reviewed at two points. The first-year review takes place on 24 November 2026, and the go/no-go assessment, in which the committee decides whether the doctorate continues, is on 20 January 2027. Because the doctorate runs at 0.8 FTE, these fall later than the nominal months nine and twelve. The assessment is made on two deliverables, the interactive demonstration of CONDUCTOR and the archive, with a short report that frames them.
The supervisors are Koos Zwaan of Inholland, lector (research professor) of Innovation in the Music Industry, and Nick Degens of HKU, lector of (Dis)Connected Technology and Creativity, as academic supervisors. David Zicarelli of Cycling '74, who created Max and worked on M at Intelligent Music, is the work-field supervisor. Drew Schlesinger of Synthetic Productions, a sound developer and Kurzweil preset designer, is the professional supervisor.
The research partners are Cycling '74, Llamasoft, V2_Lab and Thunderboom Records, which connect the research to the users of Max, to Trip-A-Tron and its language, to media art, and to working musicians. The other people and organisations involved are part of the network, and are listed under related concepts.
Going further
This account is complete at its level, and the linked words in it lead to the pages where each part is described in more depth. The main sections of the archive are the research, the instruments, CONDUCTOR, the artistic output, the method and the frame. They can be read in any order, and only where a question comes up.
Related concepts
The following pages are referred to in the research but are not needed to understand this account. They can be opened from here for further exploration.
Concepts and positions
- Agency
- Distributed authorship
- Inhabitation
- Mutual adaptation
- Instruments as games
- Open relation
- Undivided attention
- Error affordance
- Imprisonment by knowledge
- Severance
- Non-linear progress
- Interface as research site
- Curiosity engine
- Hardware
- Context
Instruments and equipment
Works and occasions
- Temporal Conversations
- Metius
- Overmacht
- Past projects
- Bruxas
- WITCH
- Altın Gün
- Purple Wave
- Human After All 2026
- Tiger Talks 2026
- AI in Muziek 2026
- Makers Meet & Eat 2026
- No Man's Land 2026
- Laboratory experiments
People and organisations
- Edwin Gardner
- Floris Smit
- Lucas Laujedo
- UPscaleXR
- Effenaar Lab
- Retrokits
- Resort
- Nicola Mauskovic
- Kasia Głowicka
- Fostering Creative Agency
- John Whitney
- Vangelis
- Kraftwerk
- Suzanne Ciani
- Gordon Pask
- Latour
Sources
- Barthes 1967
- Foucault 1969
- Clark and Chalmers 1998
- Doshi and Hauser 2024
- Singh and others 2025
- Brundtland Report 1987
- Steinle 1997
- Gardner, Phonogram in de jaren '60 (2012)