Table 1 Operationalization for document analysis
From: Humanitarian aid NGOs’ accountability towards large donors: the case of the European Union’s DG ECHO
Criterion | Measurement | Examined parts of reportsa |
|---|---|---|
Positivity of conclusion | 1 = overall positive conclusion Results of the evaluated project presented as overall effective, efficient, successful, relevant, appropriate, working, etc. | - Conclusion - Otherwise: summary - Otherwise: end results section - Otherwise: annexes |
0 = overall negative or mixed assessment Results of the evaluated project presented as overall ineffective, inefficient, irrelevant, inappropriate, not working, mixed, etc. | ||
Triangulation | 1 = triangulation was applied More than one method of data collection was used. Different methods include interviews, focus groups, questionnaires, observations, analysis of documents, etc. | - Methodology section - Otherwise: summary - Otherwise: introduction - Otherwise: annexes |
0 = triangulation was not applied One method of data collection was used. | ||
Transparency of questions for respondents | 1 = questions are listed | - Methodology section - Otherwise: annexes |
0 = questions are not listed | ||
Transparency of respondents | 1 = respondent selection is transparent Selection criteria and/or a list of contacted people/organizations present | - Methodology section - Otherwise: annexes - Otherwise: introduction |
0 = respondent selection not transparent No selection criteria and no list of contacted people/organizations present | ||
Transparency of document selection | 1 = document study is transparent Selection criteria and/or a list of studied documents present | - Methodology section - Otherwise: annexes - Otherwise: footnotes (if they consistently list documents) |
0 = document study is not transparent No selection criteria and no list of studied documents present | ||
Transparency of other data collection methods (e.g. observations, existing data) | 1 = other methods are transparent Selection criteria and/or lists of the cases studied with other methods present | - Methodology section - Otherwise: annexes |
0 = other methods are not transparent Selection criteria and lists of the cases studied with other methods not present | ||
Transparency of evaluation criteria | 1 = criteria are transparent | - Methodology section - Otherwise: introduction - Otherwise: summary - Otherwise: tables - Otherwise: annexes |
0 = criteria are not transparent |