(UPDATED with latest data - Updated again - This is the first of two posts.)
We have now gathered enough data to start answering the research question you selected at the beginning of the term (seems like ages ago!). You designed a classification system in tutorials, and individually gathered data on lots of countries (119 122 123 so far to be exact). And the results are really good: as we shall see below, your work is about as good as the work of professional organizations that spend hundreds of thousands of dollars per year to gather information on democracy. So I'm going to give the class 10 more points for all the effort you've put into this project. (This means you stand at 67 points).
I have now done some preliminary cleanup and analysis work on your data - enough so you can start using it. The main version of the dataset your efforts have produced is available here. This is not straightforward to use (if you don't understand it, just focus on the post, forget about the spreadsheet), so in the rest of this post, I will point out some general patterns and explain some of the things you can do with it. (There are also many things we could do with it but are too technical for us; more on these in the second post).
Start with the basic data. (This data may be easier to view here, on a Google spreadsheet, than on the file linked above - Google spreadsheets has better viewing options than Excel for this). The first column shows the country name, the second column the start year for the classification, the third the end year, and the fourth the calculated score from your answers to the questions in the classification system, ranging from 0 to 41 (higher scores are more democratic). The fifth column contains a simple classification based on your score: I simply divided the range of the score into thirds, so that countries on the top third are called democracies, countries in the middle are hybrids, and countries at the bottom are called dictatorships. (You can create your own breakpoints, though, by aggregating the numerical values for the answers yourself). The sixth column contains the confidence level for the classification (provided by whoever did that country) and the seventh the justification, which provides good summaries of the political situations in all the countries you classified. (The remaining columns contain the data needed to calculate the total score).
What can you do with this? One thing you can do is to select a couple of cases - a democracy and a non-democracy (according to this system, not your own subjective impression! Note you can filter the spreadsheet by status - democracy, hybrid, non-democracy - or sort it by the country score) and compare them: did the democracy do better or worse for the poor in this comparison? This requires selecting cases that are otherwise comparable (for example, two poor countries in the same region, one of them a democracy, another a non-democracy), and using a relatively consistent measure of what “better for the poor” means. If the democratic country did better than the non-democratic one over some period, that would provide evidence (not proof! just a bit of support!) that democracies are better for the poor. (Of course, many things can go wrong in this comparison - for example, we may not find comparable cases, or the results may be confounded by poor data or other factors).
We can do more systematic comparisons between countries by using some very basic statistics. (Real studies would use far more sophisticated methods, but this is not a stats class, and I'm not a real statistician). The point is to try to find if there are any correlations between measures of poverty and our measure of regime type, which we have just created. Before I can show you some of these comparisons (in the second post), however, we need to turn the data above into a “time-varying” (or “time-series”) format, averaging all student ratings for each country-year pair (some of you covered the same countries, sometimes over similar periods). The reason for this is that we are ultimately trying to figure out how things change over time: does democracy, over time, lead to (for example) reductions in economic deprivation, infant mortality, etc? This is why I asked to classify a period - from some start date to some later date - so we could try to figure out the effects of democracy on poverty over time.
This “time-varying” version of the dataset is available here. Looking at that dataset is probably not going to be useful for you unless you know how to use a statistics program. It's probably easier to visualize like this:
Here we see each country's average democracy score (according to POLS209) on the y-axis, and the year on the x axis. The countries are arranged from right to left, top to bottom, in order of how democratic, on average, the scores were (so more democratic countries are at the top). For each year, the color of the dot indicates whether the average judgment of the class is that the country is democratic (blue), a hybrid regime (green), or a dictatorship (red). Some of these scores on particular years are averages from several students who looked at the same period and reached different conclusions (look, for example, at Paraguay, which was classified by about five different people whose ratings overlapped); this is done because the average student rating turns out to be a more accurate measure of democracy than an individual rating by a single student. (Sometimes this is called a “wisdom of crowds” effect). Interestingly, New Zealand comes out right at the top; is this evidence of bias?
How reliable is this dataset? It actually seems to be pretty good; it has a correlation coefficient of 0.85 0.84 0.86 (correlation coefficients range from -1 to 1, with 1 being perfectly correlated) with professionally compiled data on democracy byFreedom House and of 0.84 0.83 0.84 with the Unified Democracy Scores. (Freedom House and Polity, two professional compilations of democracy data, correlate at 0.87; so this is a good indication that your data is reasonably reliable!). The following visualization shows where your judgments diverge the most from those of Freedom House (darker dots are dots of greatest disagreement; countries are ordered from “most disagreement” to “least disagreement”, top to bottom, right to left):
## [1] "Correlation with Freedom House: 0.86"
Though the vast majority of countries and years do not show great differences between your judgments and those of Freedom House, the cases at the top of the graph show the most divergence. Gambia, Sudan, and Algeria, for example, are rated by FH as on average more repressive, and Botswana as freer and less repressive, in the period 1972-2013, than the judgment of the class. (E.g. Algeria appears as a “hybrid” regime in your classification, as “Not Free” for Freedom House; Botswana is hybrid for you, “Free” for Freedom House).
There is, however, no particular relationship between your confidence level and the amount of divergence from FH, which suggests simple differences of opinion rather than systematic error. (People who were less confident in their judgments did not in general differ more from FH, and people who were more confident did not in general differ less from FH).
Before I move on to do some analysis with measures of poverty (in the next post), I leave you with a graph of the trends in the global level of democracy since 1980, according to you:
The graph nicely shows the uptick in the global level of democracy since the end of the cold war; the world, according to you, is definitely becoming more democratic.
What other visualizations would you like to see? (Comparisons involving poverty data in the next post).
All the graphs in this post were created using the R programming language. If you are curious, all the code is available here. If you want to learn more, let me know.
We have now gathered enough data to start answering the research question you selected at the beginning of the term (seems like ages ago!). You designed a classification system in tutorials, and individually gathered data on lots of countries (
I have now done some preliminary cleanup and analysis work on your data - enough so you can start using it. The main version of the dataset your efforts have produced is available here. This is not straightforward to use (if you don't understand it, just focus on the post, forget about the spreadsheet), so in the rest of this post, I will point out some general patterns and explain some of the things you can do with it. (There are also many things we could do with it but are too technical for us; more on these in the second post).
Start with the basic data. (This data may be easier to view here, on a Google spreadsheet, than on the file linked above - Google spreadsheets has better viewing options than Excel for this). The first column shows the country name, the second column the start year for the classification, the third the end year, and the fourth the calculated score from your answers to the questions in the classification system, ranging from 0 to 41 (higher scores are more democratic). The fifth column contains a simple classification based on your score: I simply divided the range of the score into thirds, so that countries on the top third are called democracies, countries in the middle are hybrids, and countries at the bottom are called dictatorships. (You can create your own breakpoints, though, by aggregating the numerical values for the answers yourself). The sixth column contains the confidence level for the classification (provided by whoever did that country) and the seventh the justification, which provides good summaries of the political situations in all the countries you classified. (The remaining columns contain the data needed to calculate the total score).
What can you do with this? One thing you can do is to select a couple of cases - a democracy and a non-democracy (according to this system, not your own subjective impression! Note you can filter the spreadsheet by status - democracy, hybrid, non-democracy - or sort it by the country score) and compare them: did the democracy do better or worse for the poor in this comparison? This requires selecting cases that are otherwise comparable (for example, two poor countries in the same region, one of them a democracy, another a non-democracy), and using a relatively consistent measure of what “better for the poor” means. If the democratic country did better than the non-democratic one over some period, that would provide evidence (not proof! just a bit of support!) that democracies are better for the poor. (Of course, many things can go wrong in this comparison - for example, we may not find comparable cases, or the results may be confounded by poor data or other factors).
We can do more systematic comparisons between countries by using some very basic statistics. (Real studies would use far more sophisticated methods, but this is not a stats class, and I'm not a real statistician). The point is to try to find if there are any correlations between measures of poverty and our measure of regime type, which we have just created. Before I can show you some of these comparisons (in the second post), however, we need to turn the data above into a “time-varying” (or “time-series”) format, averaging all student ratings for each country-year pair (some of you covered the same countries, sometimes over similar periods). The reason for this is that we are ultimately trying to figure out how things change over time: does democracy, over time, lead to (for example) reductions in economic deprivation, infant mortality, etc? This is why I asked to classify a period - from some start date to some later date - so we could try to figure out the effects of democracy on poverty over time.
This “time-varying” version of the dataset is available here. Looking at that dataset is probably not going to be useful for you unless you know how to use a statistics program. It's probably easier to visualize like this:
Here we see each country's average democracy score (according to POLS209) on the y-axis, and the year on the x axis. The countries are arranged from right to left, top to bottom, in order of how democratic, on average, the scores were (so more democratic countries are at the top). For each year, the color of the dot indicates whether the average judgment of the class is that the country is democratic (blue), a hybrid regime (green), or a dictatorship (red). Some of these scores on particular years are averages from several students who looked at the same period and reached different conclusions (look, for example, at Paraguay, which was classified by about five different people whose ratings overlapped); this is done because the average student rating turns out to be a more accurate measure of democracy than an individual rating by a single student. (Sometimes this is called a “wisdom of crowds” effect). Interestingly, New Zealand comes out right at the top; is this evidence of bias?
How reliable is this dataset? It actually seems to be pretty good; it has a correlation coefficient of
## [1] "Correlation with Freedom House: 0.86"
Though the vast majority of countries and years do not show great differences between your judgments and those of Freedom House, the cases at the top of the graph show the most divergence. Gambia, Sudan, and Algeria, for example, are rated by FH as on average more repressive, and Botswana as freer and less repressive, in the period 1972-2013, than the judgment of the class. (E.g. Algeria appears as a “hybrid” regime in your classification, as “Not Free” for Freedom House; Botswana is hybrid for you, “Free” for Freedom House).
There is, however, no particular relationship between your confidence level and the amount of divergence from FH, which suggests simple differences of opinion rather than systematic error. (People who were less confident in their judgments did not in general differ more from FH, and people who were more confident did not in general differ less from FH).
Before I move on to do some analysis with measures of poverty (in the next post), I leave you with a graph of the trends in the global level of democracy since 1980, according to you:
The graph nicely shows the uptick in the global level of democracy since the end of the cold war; the world, according to you, is definitely becoming more democratic.
What other visualizations would you like to see? (Comparisons involving poverty data in the next post).
All the graphs in this post were created using the R programming language. If you are curious, all the code is available here. If you want to learn more, let me know.
No comments:
Post a Comment