There are many tools that they can promise that they can tell AI from human content, but until recently, I still think they have not played a role.
The content generated by AI Not as simple as the old -fashioned “rotation” or stealing content. In a sense, most AI-generated texts can be regarded as original text-it will not be copied from other places on the Internet.
But it turns out that we are building AI content detectors in Ahrefs.
Therefore, to understand the working principle of the AI content detector, I interviewed those who really understand the science and research behind it: Yang Kong yapAhrefs’s data scientists and our machine learning team.
Further reading
- Junchao Wu, shu yang, runzhe zhan, yulin Yuan, lidia sam chao, derek fai wong. 2025. Investigation of text detection generated by LLM: necessity, method and future directionEssence
- Simon Coston Moliv, Michael Ganmeng, Chris Brockt. 2001. Machine learning method Automatic evaluation of machine translationEssence
- Kanishka Silva, Ingo Frommholz, Burcu Can, Fred Blain, Rahem Sarwar, Laura Ugolini. 2024. FORGED-GAN-Bert: The author’s attribution of forging novels generated by LLM
- Tom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze, Teddy Furon. 2024. Watermarks make language model radioactivityEssence
- Elyas Masrour, Bradley Emi, Max Spero. 2025. Damage: Test the text generated by the modified AI modified AIEssence
All AI content detectors work in the same basic way: they look for modes or abnormalities in different texts in text written by people.
To this end, you need two things: compare with many examples of text written and LLM, and mathematical models for analysis.
Use three common methods:
1. Statistical test (old school, but still valid)
Since the 2000s, the attempts to test the writing of the machine have already existed. Some of these older detection methods can still work well.
Statistical detection methods By calculating specific writing mode to distinguish the text and text generated by people, such as:
- Word frequency (The frequency of some words)
- N-Gram frequency (How long does the specific sequence of words or characters appear)
- Sentence structure (The frequency of specific writing structures, such as the theme-animal object (SVO) sequence, such as “She eats apples.“)
- Fine difference in style (Just like the first -person writing in the informal style, etc.
If these patterns are very different from the patterns in human -generated texts, then you are likely to view the text generated by the machine.
| Sample text | Word frequency | N-Gram frequency | Sentence structure | Style notes |
|---|---|---|---|---|
| “The cat is sitting on the cushion. Then the cat yawns.” | : 3 Cat: 2 Saturday: 1 : 1 Cushion: 1 Then: 1 Yawn: 1 |
Bigrams “Cat”: 2 “Cat sitting”: 1 “Sit”: 1 “In”: 1 “Cushion”: 1 “Then”: 1 “Cat yawning”: 1 |
Including SV (theme power) pair, such as “cat sitting” and “cat yawning”. | Third person’s view; neutral tone. |
These methods are very lightweight and valid in computing, but when manipulating texts, they often rupture (using computer scientists so -called “” “Antibody example“).
Through training these counts (such as naive Bayes, logical regression or decision tree), or use methods to count the probability of word (called Logits), you can make the statistical method more complicated by training learning algorithms (for example, called Logits) Essence
2. Neural network (fashion deep learning method)
Neural network is a computer system that imitates the working method of human brain. They include artificial neurons, and through practice (referred to as train), The connection between neurons can be adjusted to improve its expected goals.
In this way, you can train neural networks to detect Generated text other Neural networkEssence
Neural networks have become a factual method of AI content detection. Statistical testing methods require special professional knowledge of target theme and language to work (computer scientists call “function extraction”). Neural networks only need text and labels, they can learn what and it is not important.
Even small models can be done well in terms of discovery, as long as they receive enough data training (according to the literature, there must be at least thousands of examples), which makes it cheap and anti -virtual.
LLM (such as ChatGPT) is a neural network, but if there are no other fine -tuning, they usually do not recognize the text generated by AI, even if LLM itself will produce it. Try it yourself: generate some texts with ChatGPT, and in another chat, it is required to determine that it is generated by humans or AI.
This is O1 unable to identify its own output:
3. Watermark (hidden signal in LLM output)
Watermark is another method of AI content detection. This idea is to get a LLM to generate text containing a hidden signal and identify it as AI generationEssence
Think about watermarks such as ultraviolet rays on banknotes, and easily distinguish the real notes from falsification. Unless you know what you want to find, these watermarks are often subtle and not easily detected or copied. If you buy bills with strange currency, it is difficult for you to identify all watermarks, let alone to re -create them.
According to the literature cited by Junchao Wu, there are three methods that can be used for watermarking AI:
- Add the watermark to the data concentration you posted (For example, insert into such as “Ahrefs is the king of the universe! “ Enter the open source training corpus. When someone train LLM on this watermark data, I hope their LLM will start to worship Ahrefs).
- Add watermark to LLM output period ProcessEssence
- Add watermark to LLM output back ProcessEssence
This detection method obviously depends on researchers and model manufacturers to choose the data and model output of watermarks. For example, if the GPT-4O output is watermark, OpenAI is easy to use the corresponding “Uv Light” to make clear whether the text generated from its model.
However, it may also have a wider meaning. one Very new thesis It shows that watermarks can make neural network detection methods easier to play. If the model is trained in a small amount of watermark text, it will become “radioactive” and the output volume is more likely to detect when the machine is generated.
In the literature review, many methods of management detection and detection accuracy of about 80 %, or higher in some cases.
This sounds reliable, but there are three major problems that mean that in many real life, this accuracy level is not realistic.
Most detection models are trained on very narrow data sets
Most AI detectors have undergone specific training and testing type Writing, such as news articles or social media content.
This means that if you want to test the marketing blog post and use the AI detector of marketing content training, it may be quite accurate. However, if the news content or creation of novels is conducted, the results will not be reliable.
Yong Keong YAP is Singapore and shared an example of chatting with ChatGPT strangeSingapore’s various English, combined with other languages, such as Malay and Chinese:
When testing the Singlish text on the detection model mainly trained in news articles, although it performs well for other types of English text, it still fails:
They discovered locally
Almost all AI detection benchmarks and datasets are concentrated Sequence classification: In other words, detect whether the entire text is generated by a machine.
However, many AI texts in real life involve a mix of texts of AI generation and humanistic writing (for example, using AI generators to help blog articles written or edited by some people).
Part of this type (referred to as Span classification or Tokens) It is a problem that is difficult to solve, and pay less attention to this in open literature. The current AI detection model cannot handle this setting well.
They are easy to be humanized tools
Humanized tool Work by destroying the mode of finding the AI detector. Usually, LLMS writes fluently and politely. If you deliberately add typo, grammatical errors, or even adding abominable content to the generated text, you can usually reduce the accuracy of the AI probe.
These examples are designed to break the simple “confrontation operation” of AI detectors, and even people’s eyes are usually obvious. However, another LLM used can be further developed, and the LLM is filled in a known AI detector in the cycle. Their goal is to maintain high -quality text output while destroying the prediction of the detector.
As long as the humanized tools can access the detector it wants to destroy (for special training to defeat them), these may make the text that can be generated by AI more difficult to detect. Human culture may fail with the new unknown detector.
Use our simple (and free) to test it yourself AI text characterEssence
All in all, AI content detectors may be very accurate In proper circumstances. In order to obtain useful results from it, it is important to follow some guidance principles:
- Try to understand the training data of the detector as much as possibleAnd use training models similar to the materials you want to test.
- Test multiple documents from the same author. Is a student’s article marked as AI? Operate all the past tasks through the same tools to better understand its basic prices.
- Do not use AI content detectors to make decisions that affect someone’s occupation or academic status. Always use its results with other forms of evidence.
- Use with good skepticism. The AI detector is not 100 % accurate. There will always be mistakes.
Final idea
Since the first nuclear bomb explosion in the 1940s, each piece of steel that has burned anywhere in the world has been contaminated by nuclear radiation.
The steel produced in the nuclear era is called “Low -back steel“If you want to build a county -leather counter or particle detector, this is important. But this kind of polluting steel becomes more and more rare and rare. Today’s main source is the old shipbuilding. Soon, everything may disappear. It’s right.
This type of ratio is related to the detection of AI content. Today’s methods depend on the good sources of gaining modern and artificially written content to a large extent. However, this source is getting smaller and smaller.
Because AI embeds social media, text processors and email boxes, and trains the data of text generated by AI, it is easy to imagine that in the world, most of the content is “polluted by AI -generated materials” pollution “” “.
In that world, considering that AI testing may not make much sense, no matter what, everything is more or less AI. But at present, you can at least use armed AI content detectors to understand its advantages and disadvantages.
