User:Kokiri/WQA2

Two years since my last attempt (WQA 1) to assess the quality of the Pedia, here comes an update. On the one hand, I have included the same measures as last time, allowing for a comparison where possible. On the other hand, I have included a few extra measures. Please feel free to edit this page if you feel the presentation could be improved.

The other quality assessment I am aware of was by Adam Carr, carried out in October 2003: English Wikipedia Quality Survey.

I have sampled 275 articles on 18 December 2005, using the random article function. This means, in contrast to WQA 1, WQA 2 uses a representative random sample with a confidence level of 90% and a margin of error of 5%. I have not assessed the quality of the articles themselves, something Nature have recently done with their non-representative sample. I have also not assessed the level of plagiarism that I expect the Pedia to be guilty of.

Definitions

entry: what is commonly referred to a an article on the Pedia, not regarding its length or quality

article: a proper entry in the Pedia, not a stub or any other category listed here; articles can be quite short according to the criteria I applied

bot: a bot generated article; in practice this refers to Rambot articles

fragment: an entry with the structure of an article, but mostly consisting of titles and other non-content (called gaps in WQA 1)

list: an entry that is just a list links

disambiguation: disambiguation page

stub: a stub

spam: I came across one entry that was spam (which I flagged for speedy deletion)

Overview: Crude statistics

I guess that most people will be interested in the following table. Bear in mind the margin of error (5%) and the confidence level (90%) throughout the analysis.

Quality (ordered)	%	Plot
Spam	.4
Stub	40.0	********
List	12.0	**
Fragment	9.5	**
Bot	4.0	*
Articles	34.2	*******

This means that only 34.2% of all Wikipedia entries are articles of some sort.

Compared to WQA 1 two years ago

Two years ago, in my first assessment I came up with the following numbers:

Category	#	%	Plot
Article	19	38	********
Bot	12	24	*****
Fragment	4	8	**
List	3	6	*
Stub	12	24	*****

Bear in mind the smaller sample (N=50) back then.

Frequencies of article features

What follows is a bunch of frequency tables on some of the variables in WQA 2. These are features that any article and stub can have. A low percentage of entries with a certain feature does not necessarily indicate poor quality: not all features are equally desirable in all articles.

Categories

Two years ago, we did not have categories. Now almost every article and stub is categorized in some way.

Categories	Count	%	Plot
0	18	6.5	*
1	73	26.5	*****
2	94	34.2	*******
3	39	14.2	***
4	26	9.5	**
5	11	4	*
6	2	.7
7	3	1.1
8	4	1.5
9	1	.4
10	3	1.1
14	1	.4
Total	275

Formulae

A rare sighting.

0 Formulae	273	99.3%	********************
1 Formula	1	.4%
3 Formulae	1	.4%

Maps

Not very common, but then again, not every article should have one. Still, many place articles do not have a map. I counted 29 entries on places, and 13 maps...

0 Maps	262
1 Map	13

Pictures and Illustrations

Most articles still come without illustration. I have not come across any animations or videos, something other encyclopaedias brag with... I know they exist, but the fact that they do not show in this assessment suggests that there are not many of them.

Frequency	#	%	Plot
0	216	78.5	****************
1	39	14.2	**
2	12	4.4	*
3	4	1.5
4	1	.4
6	1	.4
9	1	.4
20	1	.4
Total	275	100.0

References

At least we have now started with references, but they are largely absent. This not only compromises the verifiability of entries, but is probably a sign of rampant plagiarism (rephrasing something does not mean the source should be attributed).

Frequency	#	%	Plot
0	254	92.4	******************
1	10	3.6	*
2	5	1.8
3	1	.4
4	4	1.5
5	1	.4
Total	275	100.0

Tables

As with illustrations, not very common. There is of course the question whether tables should be used in some entries at all, because they are not always a useful way to summarize information.

0 Tables	227	82.5%	****************
1 Table	44	16%	***
2 Tables	4	1.5%

Length

Again, like two years ago, I have used a rather convenient way to measure the length of entries.

Frequency	Count	%	Plot
3 lines or less	69	25.1	*****
less than 1 screen	77	28.0	******
1 Screen	29	10.5	**
2 Screens	51	18.5	****
3 Screens	26	9.5	**
Longer	23	8.4	**

Britannica comparison

As last time, I have checked whether the entry was also in Britannica. Last time I used the 2002 DVD version of Britannica; this time I used search.eb.com, so in Britannica also means in the 'Britannica Student Encyclopaedia. Not in Britannica means that there were no matches, In Britannica means that there is an article with the same or an equivalent title, Within Wider Britannica Article means that the topic is treated within a Britannica article that covers a larger topic. This is possibly my pet peeve: many stubs have little potential to grow because they would better be dealt with in a more general article (see WQA 1).

Not in Britannica	166	60.4%	************
In Britannica	32	11.6%	**
Within Wider Britannica Article	57	20.7%	****

Areas covered

Here are two tables on the areas the entries cover. First I included a category for persons, then I split this category to the others. So, in the first table, a politician would be counted as person, in the second she or he would be found under politics. The areas are to a large extent influenced by the articles, and do not follow an existing categorization. This must be borne in mind when considering the systematic bias of the Pedia.

Including persons

Area	Count	%
Animal	6	2.2
Architecture	4	1.5
Company	3	1.1
Culture	4	1.5
Disambiguation	20	7.3
Geography	10	3.6
History	6	2.2
Language	2	.7
Law	5	1.8
Leisure	12	4.4
Literature	14	5.1
Media	12	4.4
Music	14	5.1
Nobility	6	2.2
Organization	9	3.3
Person	55	20.0
Place	30	10.9
Plant	2	.7
Internet	4	1.5
Religion	4	1.5
Science	13	4.7
Sports	10	3.6
Standard	2	.7
Technology	4	1.5
Misc.	5	1.8
Transport	7	2.5
Other	6	2.2
Warfare	6	2.2
Total	275	100.0

Persons assigned to other areas

Area	Count	%
Animal	5	1.8
Art	15	5.5
Company	3	1.1
Culture	6	2.2
Disambiguation	20	7.3
Fiction	6	2.2
Geography	10	3.6
History	6	2.2
Language	2	.7
Law	5	1.8
Leisure	12	4.4
Literature	12	4.4
Media	15	5.5
Music	22	8.0
Nobility	6	2.2
Organization	9	3.3
Place	29	10.5
Plant	2	.7
Internet	4	1.5
Politics	10	3.6
Religion	9	3.3
Science	17	6.2
Sports	18	6.5
Standard	2	.7
Technology	4	1.5
Misc.	5	1.8
Transport	7	2.5
Other	6	2.2
Warfare	8	2.9
Total	275	100.0

Predicting what is an article

The following table is the result of a regression analysis, trying to predict what makes an article (as opposed to a stub, fragment, or the like). Hits in Google and Scirus are insignificant predictors, meaning that some articles have many Google hits, others just a few. If the topic occurs in Britannica, it is 1.5 times as likely to be an article as a topic that does not. This, however, is not statistically significant once the number of incoming links is considered. The number of links to an entry is the single most powerful predictor whether an entry will be an article or anything less: every incoming link increases the chance of being an article by 8%.

	Sig.	Exp(B)
Is in EB	.298	1.418
Google Hits	.936	1.000
Scirus Hits	.768	1.000
Scirus Journal Hits	.446	1.000
Links In	.000	1.078
Constant	.000	.171

I have also run the regression with splitting the different kinds of Britannica entries (equivalent or within other article). Again, the number of Google hits is irrelevant. The number of incoming links is a very good predictor. Moreover, we can see why the distinction between the kinds of Britannica entry is important: a Wikipedia entry with an equivalent article in Britannica is 2.3 times as likely to be an article than an entry without. Wikipedia entries with an article in Britannica that covers a wider topic do not fare significantly better than entries that have no entry in Britannica at all. I take this as a sign that we have too many entries that have no potential to grow...

	Sig.	Exp(B)
1000 Google Hits	.734	1.000
EB Equivalent	.102	2.305
EB within Larger	.605	1.216
Links In	.000	1.076
Constant	.000	.175

Predicting what is a stub

Here are the results of a regression analysis that predicts whether an entry is a stub as opposed to anything else. We find the opposite here. Again, the number of hits in Google and Scirus are insignificant. The number of incoming links is significant: For every extra incoming link, the entry is 8% less likely to be a stub. If there is an entry in Britannica, the chances of being a stub drop by 34.7%.

	Sig.	Exp(B)
Is in EB	.189	.653
Google Hits	.792	1.000
Scirus Hits	.774	1.000
Scirus Journal Hits	.979	1.000
Links In	.000	.912
Constant	.000	2.253

Predicting article length

This table summarizes the prediction of article length. All these predictors are statistically significant (.1 level), with the exception of the number of hits in Scirus. An entry which can also be found in Britannica is expected to be about half a screen larger than one that cannot. The effects of hits in the search engines are significant but very small. It takes 1 million Google hits to increase the article length by half a screen (0.000000492 screens for every Google hit); or about 5000 Scirus Journal hits for the same effect (0.0000935 screens for every Scirus Journal hit).

	B	Sig.
(Constant)	1.062	.000
Is in EB	.582	.002
Hits in Google	4.92E-007	.004
Hits in Scirus	-2.02E-005	.117
Journal Hits in Scirus	9.35E-005	.092

Features in longer articles

Longer articles tend to have more of the features measured. All correlations are positive and significant at the .01 level. The number of incoming links once again shows up as the strongest effect.

Number of Maps	.172 **
Number of Pictures	.172 **
Number of References	.179 **
Number of Tables	.236 **
Links to Entry	.473 **

The data

Feel free to make use of the data for your own analyses. Bear in mind the predictive limits outlined in the introduction: Data.