approach	data	sample	analysis	strength of evidence	chain of reasoning	cost of planning	cost of execution	cost of analysis
1	quasi-experimental, multi-institutional	quant & qual	8 institutions, 217 subjects	statistical & qualitative	medium	good presentation. Some flaws	high	high	high
2	experimental	numeric	72 males	statistical	low	weak	med-high	med	low
3	survey	scales (open questions)	??	compilation	LOW	---	low	low	low
4	psychometric experiment (2x2 mixed measures)	numeric (scales)	90	statistical (factor analysis)	patchy	patchy	high	medium	medium
5	quasi-experimental pre- & post- test	numeric	?? 46-ish ??	basic stats	patchy/low	weak, gappy & thin	low	low	low
6	census (longitudinal)	numeric & qual	48 unis 85 courses (% population? departments?)	compilation	strong	few conclusions drawn	very low (bugger all)	very high (shitload}	low
7	on-line survey	qual & numeric & grades	40/200	“I put them into a spreadsheet” (basic stats)	weak (But served its purpose)	average	low (3 afternoons)	very low	low
8	quasi-experimental survey (validated instruments) pre & post test	numeric	~270 (/400)	statistical	OK, but limited (based on averages)	moderate conclusions explicit unexposed assumptions good	high (part of larger endeavour. This paper a hitchhiker)	moderate	med-low

1. A multi-national, multi-institutional study of assessment of programming skills of first-year CS students Michael McCracken, Vicki Almstrum, Danny Diaz, Mark Guzdial, Dianne Hagan, Yifat Ben-David Kolikant, Cary Laxer, Lynda Thomas, Ian Utting, Tadeusz Wilusz
2. Collaborative Group Interactions of Students from Two Ethnic Backgrounds S. Wright and D. Lander
3. Soft or Hard Boiled: Relevance of Soft Skills for IS professionals Snell, S., Snell-Siddle C. and Whitehouse, D.
4. Using online surveys to measure three key constructs of the quality of human-computer interaction in web sites: psychometric properties and implications Van Shaik, P. & J. Ling
5. An innovative approach to teaching first year programming supported by learning style investigation Fowler, L., Campbell, V., McGill, D. and Roy, G,
6. Introductory Programming: What’s happening to day and will there be anyone to teach tomorrow? Michael De Raadt, Richard Watson and Mark Toleman
7. Learning about software development – should programming always come first? Margaret Hamilton and Liz Haywood
8. The influence of web-supported independent activities and small group work on students’ epistemological beliefs Denise Tolhurst