It's AI detector leaderboard submission

[sergak0]

No description provided.

[github-actions]

Eval run succeeded! Link to run: link

Here are the results of the submission(s):

It's AI

Release date: 2024-09-04

I've committed detailed results of this detector's performance on the test set to this PR.

On the RAID dataset as a whole (aggregated across all generation models, domains, decoding strategies, repetition penalties, and adversarial attacks), it achieved an accuracy of 87.25%.
Without adversarial attacks, it achieved an accuracy of 91.93%.
If all looks well, a maintainer will come by soon to merge this PR and your entry/entries will appear on the leaderboard. If you need to make any changes, feel free to push new commits to this PR. Thanks for submitting to RAID!

[liamdugan]

Congrats on the new SOTA! Merged.

[sergak0]

Thanks! By the way, I've seen that in the paper you've also scored commercial ai-detectors, but I haven't found them in the leaderboard - is there a way to compare our solution with theirs?

[liamdugan]

Yes. The reason why they aren't included on the leaderboard is because we were only able to test a small portion of our test set on the commercial detectors due to budget constraints. We're currently working with these companies to get more credits so that we can add them to the leaderboard as well. We hope to add them soon.

[sergak0]

Glad to hear that you are going to add them too. Anyway great work with RAID dataset - really impressed with the amount and diversity of data that you've collected and first ai-detection leaderboard that you've made.

[liamdugan]

Thank you! And thanks for submitting :)

[AIApprentice101]

Thank you @liamdugan for the prompt update of the leaderboard. It is great to have a new SOTA method. Congrats @sergak0

I notice on your github page (https://github.com/It-s-AI/llm-detection) that there're two baseline models, the perplexity-logistic regression model, and the finetuned Deberta model. Is the Deberta model the SOTA submission here? Or is it the result from the subnet 32?

Btw, I love your idea of using Bittensor subset for the AI detection task.

[sergak0]

@AIApprentice101 yeah, it's the result from subnet 32.

Btw, I love your idea of using Bittensor subnet for the AI detection task.

Thanks, hope to make our detector even better in the future.