Clearly that is not the case mostly otherwise there would be no market for the essays. I think humanities Masters at the 2nd tier “universities” are mostly essays. I have met people personally who had some humanities Masters and it was obvious it was fake. Having said that, I met someone 20 years ago who had a CS Masters from Milton Keynes and she was fake too. PhD is different.
Re forum posting, yes of course this is fine but obviously if you can’t speak English at all then the translation could be pretty useless and you won’t know. I get this at work quite a bit. Emails from customers not making sense.
An admin has a broad visibility of someone’s posting pattern and malicious posters stand out well. So I don’t see an issue. 99% of non UK people here could get ELP7 easily 
dublinpilot wrote:
Presumably the answer is that after submission the student would be “interviewed” on the content. So they are asked to explain certain points or delve further in and clarify things. This would be an integral part of the results.
Yes, that’s exactly what we do.
I’ve heard some schools add some bombshell white text in the middle of the subject (invisible to a human reader, but included when copy/pasting), that adds instructions that can derail the LLM or make it easier to identify in the essay.
A cousin has a nice side-job writing essays up to masters level. He says it’s a mechanical process, simply repeating select parts of a narrow set of information in a predetermined format. He is clearly intelligent, but he says with practice anyone (or a machine) could do it.
I was recently doing SEO with my web designer, which recommends using AI to rewrite content. I can see it improves badly-written text, but it also drags my highly literate efforts to the same generic dumbed-down level. I find it depressing the market ‘wants’ this, as it’s demonstrably less formal, precise, and informative. The obvious changes to a sample:
I wouldn’t go as far as saying the “market” wants this. Surely Google does, but for the people reading it, it really depends on who they are.
As far as I’m concerned, the lack of “character” is a big detriment to the content. Information delivered in a purely sleek and streamlined way is much less interesting, there is no “feeling”, no rough edges, and practically no expressivity.
I agree 100%.
The funny thing though is that given that the majority of the population (in any modern country) struggles with writing more than a few lines, we will see the rise of a sub-population whose entire written output is done by this software. It will be readily recognisable initially but maybe less so as time goes on. Already people can choose an avatar for an online community which they feel represents their personality (most of these look silly) and choosing a “personality type” for written output is the next obvious thing.
Forums will find it hard to deal with because human mods cost money. For example I know of one big UK site, huge advertising income, 30k posts per day, which has mods but they can’t read all the posts and instead rely on a “Report” button being pressed. The users of this site are probably super-active in this (because many of them have nothing better to do as well as being highly judgemental) but they probably won’t report ChatGTP type text – especially if this improves. So naturally the site will fill up with machine-generated crap. In turn, their activity and thus income will fall because “spicy” discussions generate far more activity than bland ones.
So, the mod/admin activity will need to shift towards checking signups and that is already how EuroGA works. More than 50% of signups are banned immediately (did 2 just now) but almost nobody (about 1 per year, not counting malicious posters who managed to get in) gets banned after that.
choosing a “personality type” for written output is the next obvious thing.
It is technically quite doable to take a collection of text from someone and fine-tune a model to match that style. I actually should try to do it on my own text to see what comes out when I have some time and would be curious how it come out. If it works well (I believe it would) then someone with access to the post of someone else could probably replicate the style and fool the mods.
Forums will find it hard to deal with because human mods cost money
reversely, A LLM would likely do a very good job at checking the consistency of someone post with its history, maybe not an absolute check a post is from human, but a comparative check that the post is suspicious because not in the regular style.
How do the websites which detect ChatGPT etc do it? They are very reliable.
Re language patterns, the standard defence is to write very little. On say 30 words you can’t do much, unless a rare phrase is involved (search for “upper deck” – including the quotes and look at the first two hits). We’ve had people here doing funny stuff and they usually take care to write very little, especially when trying to be female (female writing is usually obvious).
Going slightly different direction, and something that could be useful would be to enable a chatGPT like summarization/Questioning on thread in a forum.
Specifically:
First, put a whole thread in a context and ask question, like: what are the main point people talked about. Or what is the consensus answer to the original question, etc.
Second, use a RAG infrastructure to semantically query the content of the forum, which would work much better than keyword search.
These can be done with open source models and work surprisingly well in some tests I have seen in different domain without any domain specific fine-tuning…
