The Problem with Human-AI Interaction

PHAIN for short

A short exploration of how AI can be used to support the writing of an essay.

Status: ✅ Seed → 🔸 Growing → 🔸 Well-formed → 🔸 Fruitful → 🔸 Retired

The problem

Dennis Silverwood put forward some Warning Thoughts. These begin by asserting that, although widely called artificial intelligence, AI is not in fact intelligent. They end by highlighting the risk that people might outsource their thinking to AI and lose or never acquire the ability for critical thinking. While all age groups are at risk from outsourced thinking, the young are especially so. They already show signs of shorter attention spans, limiting their capacity to follow complex arguments, absorb nuanced information, and sustain reasoning without distraction. If this is compounded by outsourcing their thinking and failing to develop critical thinking skills, they risk losing the judgement and decision-making ability essential to independent agency.

How to explore the problem?

A research-based essay could examine the dangers posed by the temptation to outsource our thinking to AI. It could also explore whether responsible use of AI can mitigate these dangers. As well as making us better informed, the essay could also serve as a vehicle for learning the practicalities of putting AI to purposeful use.

We do not at present have the resources to carry out the study suggested above; perhaps it could be undertaken as a collaborative endeavour? We have been able, however, to take a first step down that path as a means of testing the Goal-Directed Context Management approach.

Some background is needed. If we want AI to stay helpful and relevant, we must give it the right information to hold in its working memory - in what is called its context (the meaning of context is explained here). For the AI to perform well on our behalf, its context must encompass and keep pace with the progress of the work for which it is assisting us.

Goal-Directed Context Management proposes how to achieve this during the development of a knowledge-based product such as an essay. Similarly to when using a conventional computer, the first step was to bootstrap the AI. This involved prompting it with as good as possible a description of intentions for the project, and then working with it to develop an AI Startup Pack.

The AI Startup Pack consisted of project instructions and reference files. These were put into the AI's context within a ChatGPT project space. Working in that project space, after an initial hiccup the AI produced good quality first draft material for use in the Project Brief, one of the main outputs from project startup.

First impressions are that Goal-Directed Context Management is a good approach to the use of AI within a knowledge-based project.

Reflections

In my experience, the learning and retention of knowledge comes through writing down and working through ideas. During the Project Startup Experiment described above, the AI served a useful purpose in asking me to clarify things, forcing me to write them down and think them through. But did I begin to lose agency as it then repackaged my ideas in useful ways?

In this limited case, I was producing boiler-plate material for controlling the work of the project; the intellectual content of that work will reside in the body of the essay yet to be written. Is it legitimate to cede agency for boiler-plate work, freeing up capacity for more intellectually demanding work? The case for the scientific calculator, argued here by Casio, suggests so.