When an LLM is asked 'What do you mean by I when you use that word?' and it gives a philosophically sophisticated answer that can be probed and extended, we are learning something philosophically interesting about selfhood and consciousness, regardless of whether the LLM is actually conscious.