Intro

It’s been a while since I posted something brand new, but I’ve updated some posts like Linux Distro I Tried, Thoughts on Copyright, Keep $Home Clean via XDG Base Directory etc. I tested positive for COVID-19 at the very end of 2022, and headache constrained me from writing. I lay in my bed almost all day, and can sit up for only two hours per day by then. But cogito, ergo sum. I’ve tried hard to catch up the news, and did not miss ChatGPT, the so-called savior built on top of OpenAI’s GPT-3.

ChatGPT

At the beginning, many rush to ChatGPT web portal only to find out that they are unable to register as OpenAI blocks phone numbers with +86 & +852 country code, along with virtual phone numbers. The good news is that it’s becoming more and more accessible with projects like chatgpt-telegram and wechatgpt, let alone the launch of ChatGPT Plus.

I had a brief chat with ChatGPT on Dec 27 last year and the experience could never be called good. I happened to do translation on that day and decided to give ChatGPT a simple test.¹ I asked the meaning of “阿姨洗铁路” twice, in Chinese & English separately.

Unfortunately (but not surprisingly), ChatGPT could not understand it and just returned the literal/lexical/semantic meaning of the phrase. This is what I expected, and there is nothing wrong with ChatGPT.

Before we go on, let me explain “阿姨洗铁路” first. It’s a Chinese soramimi (空耳) of “I love you” in Japanese.² Here is the plot twist: ChatGPT can understand Chinese or English, but this soramimi, when asked in English, requires it to understand Chinese, English and Japanese at the same time. That’s the ultimate defect of language modules: they have no interlingual way of thinking. Okay, now let’s move on.

After the intended failure, I gave ChatGPT the definition of soramimi, explain the context, and now it just needs to echo.

Yet it failed. With the same result.

That’s annoying. It was like, you know, people managed to train ChatGPT into a NSFW catgirl by casting an extremely long spell that sounds and works like a curse, while I can’t even make it to understand soramimi!

After a day or two with several other attempts, I concluded that ChatGPT can never understand a single soramimi. Large language module like GPT-3 (I mean, really large one, which costs millions to train, and still costs $10k per day to operate) are amazing, but in the end, it remains a hyper mockingbird.

Bible Tower

Hyper mockingbirds like ChatGPT are barely useless for linguists, that’s what I learned from the experience. To make matters worse, I carried out other simple experiments a few days later.

Let’s begin with 0721. It’s a Japanese homophonic of masturbation,³ commonly used in eastern Asia ACGN area. ChatGPT cannot understand it, maybe due to the lack of Japanese corpus. Understandable, but still, it can never understand it even if I explicitly give the definition, explain the context, and now it just needs to echo. There is no interlingual way of thinking this time. It’s just a homophonic!

Desperate as I was, I tried to make ChatGPT understand “野兽先辈”. It’s easy as fxxk as long as it can connect to the Internet. By default, ChatGPT will not do that, but it can get reverted. And…it failed again.

I’m really puzzled. Linguist cannot understand why people call ChatGPT an evolution, just like ChatGPT cannot understand why “阿姨洗铁路” is “I love you” and 0721 is masturbation. The language module is fairly competitive, but that’s because its competitors cannot or don’t wanna build it, the module per se is far from great, and I doubt it would ever be great enough for linguists.

Similarly, there is no doubt that artificial intelligence can have vision, but I’m skeptical about its capability to understand sorame.⁴ Besides that, rumors say that enterprises like Microsoft limit chatbot’s ability (notably Xiaoice) to understand context to avoid Uncanny Valley. I can feel that OpenAI put less restriction on ChatGPT than Microsoft does on Xiaoice. OpenAI only puts new restriction when the media distributes propaganda against it. But ChatGPT still failed my “pen in the pen” context test. The weird thing is, usually AI fails to understand that the word pen is polysemy (not to be confused with homonymy), but ChatGPT manages to get the two meanings of pen, and still fails somehow…

Me: Jack puts a pen in his pen, goes to W.C., and back. Where is the pen?
ChatGPT: The pen is still in Jack's pocket.
Me: Not in his pen?
ChatGPT: No, it is unlikely that a bird would be kept in a pen. Birds are typically kept in cages and aviaries.

Even if AI manage to understand context, I still hold doubt that it would be able to differentiate the denotative & connotative meaning of words.

To end this chapter, I’ll finally post something about Bible Tower: xkcd.⁵

Chinese Room

The other day I came across Chinese room experiment (again) when playing Zero Escape: Virtue’s Last Reward (极限脱出 ADV 善人死亡). The thought experiment is really interesting, and can be used to understand why a software engineer on Google’s artificial intelligence development team would be convinced that their AI bot is sentient. I’m confident that LaMDA would pass no more than two of my tests mentioned above.⁶⁷ Back to Chinese room experiment, I don’t think it would ever have a solution. It’s fundamentally agnostic as long as human beings still cannot understand how brain works or answer Grundfrage der Philosophie (fundamental question of philosophy, aka. the relationship between thinking and existence).

After all, the cake is a lie.