Submit ModernBERT model

[redbrain]

This is a submission for the model GeorgeDrayson/modernbert-ai-detection-raid-mage, which was introduced in the paper "Machine-generated text detection prevents language model collapse".

[github-actions]

Eval run succeeded! Link to run: link

Here are the results of the submission(s):

ModernBERT AI Detection

Release date: 2025-05-19

I've committed detailed results of this detector's performance on the test set to this PR.

Warning

No aggregate score across all settings is reported here as some domains/generator models/decoding strategies/repetition penalties/adversarial attacks were not included in the submission. This submission will not appear in the main leaderboard; it will only be visible within the splits in which all samples were evaluated.

Warning

No aggregate score across all non-adversarial settings is reported here as some domains/generator models/decoding strategies/repetition penalties were not included in the submission.

If all looks well, a maintainer will come by soon to merge this PR and your entry/entries will appear on the leaderboard. If you need to make any changes, feel free to push new commits to this PR. Thanks for submitting to RAID!

[liamdugan]

Hey @redbrain it seems like your submission is missing scores on some of the texts in the test split. Make sure the predictions.json file has one output for every document in test (including adversarial attacks). Let me know if you have any questions!

[redbrain]

@liamdugan Thanks for catching that! There were a few texts that silently failed when I ran out of memory. I've re-ran them all and the predictions file should be fixed now.

[github-actions]

Eval run succeeded! Link to run: link

Here are the results of the submission(s):

ModernBERT AI Detection

Release date: 2025-05-19

I've committed detailed results of this detector's performance on the test set to this PR.

On the RAID dataset as a whole (aggregated across all generation models, domains, decoding strategies, repetition penalties, and adversarial attacks), it achieved an AUROC of 97.65 and a TPR of 94.14% at FPR=5% and 88.23% at FPR=1%.
Without adversarial attacks, it achieved AUROC of 99.12 and a TPR of 99.11% at FPR=5% and 98.18% at FPR=1%.

If all looks well, a maintainer will come by soon to merge this PR and your entry/entries will appear on the leaderboard. If you need to make any changes, feel free to push new commits to this PR. Thanks for submitting to RAID!

[redbrain]

Looks good to me, ready to be merged. And thanks again!

[liamdugan]

Yeah no worries! Congrats on the strong results.