🔍 Read the full analysis: A Hands-On Comparison Of 71 AI Configurations For News on ThorstenMeyerAI.com
Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Thorsten Meyer reports testing 71 AI configurations on news-writing tasks between Sept. 8 and Oct. 13, 2026. GPT-6 Astra scored highest, but Meyer says he chose GPT-6 Luna for production because it cost about one-hundredth as much; the test also found two fallback models produced fabricated quotes or unusable output.
Thorsten Meyer says a test of 71 AI configurations for turning real sources into English- and German-language news articles found that GPT-6 Astra scored highest, while the less expensive GPT-6 Luna remained his production choice. The results, published Oct. 13 on ThorstenMeyerAI.com, also led him to remove two backup models after they produced invented quotes or invalid output.
Meyer says he ran the comparison from Sept. 8 through Oct. 13, 2026, using the production prompt and two real news sources: a 6,340-character product filing and a 619-character political item. Each configuration produced an article in English and German. Two AI judges, GPT-6.1 Sol and Claude Opus 5.5, assessed source fidelity, fabrication, clarity about evidence, news value, structure and polish, with language quality also assessed for German.
In the standard test, GPT-6 Astra scored 4.00 out of 5 in both languages and received a source-fidelity score of 5.00. Meyer reports that its average across a further 12-article robustness run was 3.93, with a 95% bootstrap interval of 3.83 to 4.02. GPT-6 Luna scored 3.67 in English and 3.68 in German in the standard test; its robustness-run score was 3.67, with source fidelity of 4.71.
The reported per-article costs were about $0.12 for Astra and $0.0012 for Luna, with typical generation times of about 50 and 17 seconds, respectively. Meyer says Luna articles were all valid under the pipeline’s repair rules. He reports that he chose not to run Astra in production because of its cost and kept Luna as the production model.
Infographic 01 · Who writes the best news article?
The best writer is not the one in production
71 model configurations, one real job: turn a news source into an article for an English and a German edition. GPT-6 Astra writes the best articles. GPT-6 Luna writes them for about a hundredth of the price.
Robustness run: 12 real news sources, English edition
Dot = mean score (1–5), bar = 95% bootstrap interval. German edition mean below each score. Overlapping bars are not a real difference.
Two judges, two caveats: GPT-6.1 Sol and Claude Opus 5.5 grade every article and were also tested as writers, so they partly grade themselves. Claude models ran with effort medium unless noted.
Source: Thorsten Meyer AI model-scout benchmark, 8 Sep – 13 Oct 2026. Task: write a news article from a real source with the production prompt, English and German editions. Judges: GPT-6.1 Sol (reasoning high) and Claude Opus 5.5, both with the source in context; score 1–5, mean of both judges. Robustness run: 2 test articles plus 10 more real sources, mostly short like production; mean with 95% bootstrap interval.
Powered by Thorsten Meyer AI
Quality Versus Cost Per Article
The results illustrate why a model’s general benchmark position may not predict its performance on a specific newsroom task. Meyer’s comparison measured not just writing quality but also source fidelity, valid output and unit cost—factors that affect whether a model can be used reliably at scale.
In Meyer’s account, Astra’s higher score did not make it the practical choice for his publishing pipeline: at roughly 100 times Luna’s reported cost, it offered a quality advantage he judged too expensive for routine news production. The figures are specific to his prompts, sources, models and operating setup; they do not establish that Luna is the best choice for other publishers or assignments.
The backup-model findings carry a separate operational lesson. A system that performs well in its normal route can still publish errors if a low-quality fallback is called during a failure. Meyer says one backup scored at the bottom of the source-fidelity scale and another produced no usable articles in four attempts, prompting their removal.
As an affiliate, we earn on qualifying purchases.
How Meyer Built the Comparison
The test used two source lengths because Meyer says much of his news work begins with short material. The 619-character political item was intended to represent that thinner-source challenge, while the product filing offered 6,340 characters of material. Articles were written for an English edition and a German edition aimed at Germany, Austria and Switzerland.
Meyer says the judges saw the original source and scored seven dimensions on a five-point scale. He also required output to pass the same repair step used by his production pipeline before counting as valid. For the strongest configurations, he ran a further 10-article robustness test, mostly with short sources. He cautioned that differences under about 0.3 points in the two-article standard test could be noise, and said he considered the larger run more reliable for ranking.
The report also compares reasoning settings and models from different providers. For GPT-6 Luna, Meyer tested six reasoning-effort levels. He says the results showed no meaningful improvement over the production setting, while maximum effort took 211 seconds per article and cost 1.43 cents, compared with 17 seconds and 0.12 cents at the no-effort setting.
“The right model is the cheapest one that clears your quality bar reliably, not the one at the top of the chart.”
— Thorsten Meyer, ThorstenMeyerAI.com
Limits of the Reported Scores
The results come from one publisher’s workflow, and the standard comparison used only two source items. The additional robustness run covered 12 articles for Astra and 10 more for the strongest configurations, but the report does not describe a broad sample of news topics, publishers or editorial teams. The scores therefore do not establish how the models would perform across other newsrooms or source types.
The judges were AI systems, not a disclosed panel of human editors. Meyer says both judges saw the source, but the report does not provide enough detail here to independently assess how judge selection or scoring affected the rankings. The reported intervals and scores are his analysis, not an independent replication.
The source material says the route now runs GPT-6 Luna first, followed by “GPT-6 Luna” again, leaving the identity of the current second-choice model unclear. It also does not provide a full configuration-by-configuration results table in the supplied account. Model prices, availability and performance may change; the stated costs and timings describe Meyer’s reported setup.
Production Testing After the Comparison
Meyer says GPT-6 Luna remains in production at the no-reasoning-effort setting, which he reports cost about $0.0013 per article during the previous week, with a median generation time of 17 seconds. He says GPT-6 Astra was set aside on Oct. 7 because of its cost, and that GPT-4.1 nano and gpt-oss-120b were removed from the news route that day.
The next useful evidence would be continued results from routine publishing: whether Luna maintains source fidelity across a larger range of stories, and which verified backup model now takes over if Luna is unavailable. The supplied report does not name a confirmed replacement for the second fallback, so that part of the production setup remains unresolved.
Key Questions
Which configuration scored highest in the test?
GPT-6 Astra scored 4.00 out of 5 in both English and German in the standard test, according to Meyer. Its source-fidelity score was 5.00 in that test.
Why did Meyer choose GPT-6 Luna for production?
Meyer reports that Luna scored 3.67 in the robustness run and produced valid articles, while costing about $0.0012 per article in the comparison—roughly one-hundredth of Astra’s reported cost. He says Astra’s price made it unsuitable for his routine news work.
Did greater reasoning effort improve the articles?
Meyer says the six tested Luna reasoning settings showed no meaningful quality gain. Maximum effort cost 1.43 cents per article and took 211 seconds, compared with 0.12 cents and 17 seconds at the no-effort setting.
What did the test find about fallback models?
Meyer reports that GPT-4.1 nano scored 1.00 for source fidelity and produced invented quotes. He says locally run gpt-oss-120b generated no valid articles in four attempts and also invented named quotes. Both were removed from his news route on Oct. 7.
Can the results be applied to every newsroom?
No. The comparison reflects Meyer’s prompt, pipeline, two source items in the standard test and AI-judge scoring. It offers evidence about his own workflow, but does not establish a universal ranking across publishers, topics or editorial standards.
Source: ThorstenMeyerAI.com
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
