Metrics for Testing of Flagged Revs

[P. Birken]

Hiho,

as most of you will have heard, flagged revisions were turned on on
de.wikipedia.org. You can follow progress on
http://tools.wikimedia.de/~aka/cgi-bin/reviewcnt.cgi. Now, it would be
important to measure success of flagged revisions in some way. The
following metrics come to my mind:

-Number of articles with a sighted revision (not very useful though,
but measures acceptance among editors in a way.)
-Number of articles that have a sighted revision but where the current
version is not sighted
-Time needed to sight revisions (max and mean of time until a revision
by a noneditor is sighted. The mean is very difficult to get, but
could be computed by using the mean of the pages in
Spezial:OldReviewedPages)
-Number of editors, meaning users who have the right to sight edits
(again, acceptance but also to see if we hinder people in editing more
than we should)

Do you have more ideas for metrics and how to measure them?

Best,

Philipp

[Norman Kelley]

Phil,
Thanks for your note. I have not made any suggestion or concurrences yet, but suggest you edit your note:
second paragraph: change"sighted" to "cited." "Sighted" is what happens to UFO's. "Cited" is what happens to
reference articles.
Otherwise, cool.
BudgieBirdChan0211


----- Original Message ----
From: P. Birken <pbirken-***@public.gmane.org>
To: Wikimedia Quality Discussions <wikiquality-l-RusutVdil2icGmH+5r0DM0B+***@public.gmane.org>
Sent: Wednesday, May 14, 2008 6:13:17 AM
Subject: [Wikiquality-l] Metrics for Testing of Flagged Revs

Hiho,

as most of you will have heard, flagged revisions were turned on on
de.wikipedia.org. You can follow progress on
http://tools.wikimedia.de/~aka/cgi-bin/reviewcnt.cgi. Now, it would be
important to measure success of flagged revisions in some way. The
following metrics come to my mind:

-Number of articles with a sighted revision (not very useful though,
but measures acceptance among editors in a way.)
-Number of articles that have a sighted revision but where the current
version is not sighted
-Time needed to sight revisions (max and mean of time until a revision
by a noneditor is sighted. The mean is very difficult to get, but
could be computed by using the mean of the pages in
Spezial:OldReviewedPages)
-Number of editors, meaning users who have the right to sight edits
(again, acceptance but also to see if we hinder people in editing more
than we should)

Do you have more ideas for metrics and how to measure them?

Best,

Philipp

[Tim 'avatar' Bartel]

Hi,

Post by Norman Kelley
Thanks for your note. I have not made any suggestion or concurrences yet,
second paragraph: change"sighted" to "cited." "Sighted" is what happens to
UFO's. "Cited" is what happens to
reference articles.

FYI: http://en.wikipedia.org/wiki/Wikipedia:Flagged_revisions/Sighted_versions

Bye, Tim.

[mike.lifeguard]

No. "Sighted" as in "having been seen" not as in "including citations."

Mike

_____

From: Norman Kelley [mailto:pinot_chan0730-/***@public.gmane.org]
Sent: May 14, 2008 11:13 AM
To: Wikimedia Quality Discussions
Subject: Re: [Wikiquality-l] Metrics for Testing of Flagged Revs


Phil,
Thanks for your note. I have not made any suggestion or concurrences yet,
but suggest you edit your note:
second paragraph: change"sighted" to "cited." "Sighted" is what happens to
UFO's. "Cited" is what happens to
reference articles.
Otherwise, cool.
BudgieBirdChan0211


----- Original Message ----
From: P. Birken <pbirken-***@public.gmane.org>
To: Wikimedia Quality Discussions <wikiquality-l-RusutVdil2icGmH+5r0DM0B+***@public.gmane.org>
Sent: Wednesday, May 14, 2008 6:13:17 AM
Subject: [Wikiquality-l] Metrics for Testing of Flagged Revs

Hiho,

as most of you will have heard, flagged revisions were turned on on
de.wikipedia.org. You can follow progress on
http://tools.wikimedia.de/~aka/cgi-bin/reviewcnt.cgi
<http://tools.wikimedia.de/%7Eaka/cgi-bin/reviewcnt.cgi> . Now, it would be
important to measure success of flagged revisions in some way. The
following metrics come to my mind:

-Number of articles with a sighted revision (not very useful though,
but measures acceptance among editors in a way.)
-Number of articles that have a sighted revision but where the current
version is not sighted
-Time needed to sight revisions (max and mean of time until a revision
by a noneditor is sighted. The mean is very difficult to get, but
could be computed by using the mean of the pages in
Spezial:OldReviewedPages)
-Number of editors, meaning users who have the right to sight edits
(again, acceptance but also to see if we hinder people in editing more
than we should)

Do you have more ideas for metrics and how to measure them?

Best,

Philipp

[Norman Kelley]

----- Original Message ----
From: Norman Kelley <pinot_chan0730-/***@public.gmane.org>
To: Wikimedia Quality Discussions <wikiquality-l-RusutVdil2icGmH+5r0DM0B+***@public.gmane.org>
Sent: Wednesday, May 14, 2008 9:13:22 AM
Subject: Re: [Wikiquality-l] Metrics for Testing of Flagged Revs


Phil,
Thanks for your note. I have not made any suggestion or concurrences yet, but suggest you edit your note:
second paragraph: change"sighted" to "cited." "Sighted" is what happens to UFO's. "Cited" is what happens to
reference articles.
Otherwise, cool.
BudgieBirdChan0211


----- Original Message ----
From: P. Birken <pbirken-***@public.gmane.org>
To: Wikimedia Quality Discussions <wikiquality-l-RusutVdil2icGmH+5r0DM0B+***@public.gmane.org>
Sent: Wednesday, May 14, 2008 6:13:17 AM
Subject: [Wikiquality-l] Metrics for Testing of Flagged Revs

Hiho,

as most of you will have heard, flagged revisions were turned on on
de.wikipedia.org. You can follow progress on
http://tools.wikimedia.de/~aka/cgi-bin/reviewcnt.cgi. Now, it would be
important to measure success of flagged revisions in some way. The
following metrics come to my mind:

-Number of articles with a sighted revision (not very useful though,
but measures acceptance among editors in a way.)
-Number of articles that have a sighted revision but where the current
version is not sighted
-Time needed to sight revisions (max and mean of time until a revision
by a noneditor is sighted. The mean is very difficult to get, but
could be computed by using the mean of the pages in
Spezial:OldReviewedPages)
-Number of editors, meaning users who have the right to sight edits
(again, acceptance but also to see if we hinder people in editing more
than we should)

Do you have more ideas for metrics and how to measure them?

Best,

Philipp

[geni]

Post by P. Birken
Do you have more ideas for metrics and how to measure them?

Edit rate pre and post sighting

Edit rate by logged in users pre and post sighting

[Mathias Schindler]

Post by P. Birken
-Number of articles that have a sighted revision but where the current
version is not sighted

Both in terms of number of articles and in number (or age) of
unsighted versions.

Post by P. Birken
Do you have more ideas for metrics and how to measure them?

Using the Wikistats data from dammit.lt/wikistats or a similar more
direct source, it would be interesting to see how many visitors did
see the most current version of an article because it was also flagged
as sighted at that time.