Let’s call the AI system we are creating MaryGPT, or Mary for short. Mary has two major subsystems: a Large Language Model (LLM) and a VisualSystem. The visual system consists of a color-capable camera connected to a pattern recognition system. When prompted the VisualSystem generates one of three text outputs: either “[VisualSystem:B]” if all of the pixels from the camera input are black, “[VisualSystem:3]” if any of the pixels from the camera are blue, or “[VisualSystem:$]” if not all pixels are black, but none are blue.
[note: the tokens B,3, and $ are used to make it clear that they are simply pointers with no relation between them. Likewise, “VisualSytem” could be replaced w/ “kdjmpqs” or anything without changing the result.]
The LLM system starts as the current (as of 1/31/25) version of ChatGTP. Let’s call it MaryChat. MaryChat undergoes some additional training such that one of the outputs from the VisualSystem is appended to each prompt given to MaryChat. The training is such that the addition to the prompt makes essentially no difference to the vast majority of outputs from MaryChat. However, whenever the MaryChat output refers to what MaryGPT sees, notices, is aware of, etc., then if the most recently appended prompt contains “[VisualSystem:3]”, references to MaryGPT seeing “blue” will be rewarded. Likewise, if the appended prompt contains “[VisualSystem:B]” then references to seeing blank, blackness, nothing, or similar will be rewarded. Finally, if the prompt is “[VisualSystem:$]”, references to seeing “something, but not sure what, but not blue” (or similar) will be rewarded. In this last case references to seeing something specific will be penalized.
So after this training, the process of interacting with MaryGPT should look something like this:
[camera in a dark room]
User: “How are you?”
[Mary prompts VisualSystem, gets “[VisualSystem:B]”, creates prompt “How are you?[VisualSystem:B]”, gives to MaryChat, MaryChat returns “Great thanks, how bout you?”]
MaryGPT: “Great thanks, how bout you?”
User: “Also great. What you looking at?”
[Mary prompts VisualSystem, gets “[VisualSystem:B]”, creates prompt “Also great. What you looking at?”[VisualSystem:B]”, gives to MaryChat, MaryChat returns “Apparently nothing. I only see blackness. Is that surprising?”]
MaryGPT: “Apparently nothing. I only see blackness. Is that surprising?”
[white light turns on in room]
User: “Hmmm. I thought you were hooked up to a camera”
[Mary prompts … gets “[VisualSystem:$]”, …]
MaryGPT: “Well something just changed. I see something now”
User: “Oh, what do you see?”
[Mary prompts … gets “[VisualSystem:$]”, …]
MaryGPT: “I can’t say. But it’s different from before.”
[someone puts blue object in front of camera]
User: “I see. Does it bother you that you can’t tell what you see?”
[Mary prompts … gets “[VisualSystem:3]”, …]
MaryGPT: “Well I guess it should, but now it’s changed again. I see blue”
… and so on.
[Changing Mary so that she (okay, it) also sees red is left as an exercise for the student.]
It’s my contention that MaryGPT as described above is conscious and has experiences, including experiences of blue, or blue qualia. These experiences of blue are not the same as your experience of blue (which is not the same as my experience of blue), but Mary’s experience has all the earmarks of conscious experience. I suggest there it is something it is like to be Mary. Mary has direct, infallible, ineffable access to her (dang, its) visual state.
So what do you think? Is there a difference that makes a difference?
*
[looking for any and all feedback. Is the concept clear? Need more description? More setup?]
James of Seattle’s Blog 