How much more or less information do LLMs contribute when compared to one another? Compared to humans?

The interactive graph below shows just how much more or less information both LLMs and humans bring into the texts they produce as outputs when compared to another speaker in the same condition (i.e. comparing two humans to one another and comparing two LLM outputs to one another). The lower this number is, the more predictable two texts are when compared to one another. And so if you're looking for interesting/exciting writing where either people or LLMs write in ways that are different from one another, you hope that the values reported here are higher.

How close are most of us to sounding robotic?

The point is to be on the right-hand side of the line here . . . so how many of us ended up on that side??

All the values reported here are based on the residual (how off the predicted value of convergence-entropy is compared to the actual value) if we treated every student as an LLM. In other words . . .

$H(x_{llm};y_{student}) \sim P_{N} \left( \theta_{xy}, \epsilon_p \right)$

Where

$\theta_{xy} = \beta_{c} \cdot 1 + \beta_{n_x} n_x + \beta_{n_y} n_y $

And the residual is calculated via

$H(x_{llm};y_{student}) - \hat{H}(x_{llm};y_{student})$

Here $\hat{H}$ just means we're using the predicted entropy based off of the equations we listed above.

Do LLMs or people take the prompt in more directions?

The prompt is a starting point for the writing students engaged in. And exciting writing will take the prompt in a direction that was less predictable. And so, the question is whether humans or LLMs will have higher entropy (the values reported on here) in recovering the exact ideas expressed in the prompt. Higher entropy means that the writer went off in a more exciting direction.

All the values reported here are based on the residual (how off the predicted value of convergence-entropy is compared to the actual value). Using the math from the methods section, that looks like:

$H(x_{prompt};y) \sim P_{N} \left( \phi_{xy}, \epsilon_{p} \right)$

$\phi_{xy} = \beta_{llm} \delta_{llm} + \beta_{n_x} n_x + \beta_{n_y} n_y$

And the residual is calculated via

$H(x_{prompt};y) - \hat{H}(x_{prompt};y)$

Here $\hat{H}$ just means we're using the predicted entropy based off of the equations we listed above.