Methodology
Navigate this page:
This
report on global restrictions on religion follows the same methodology as the
baseline report, with one major difference. This report also tracks changes
over time in the extent to which governments and societal groups around the
world restrict religious beliefs and practices.
Measuring Change
As
part of its original study on global restrictions on religion, the Pew Forum
developed two indexes – the Government Restrictions Index (GRI) and the Social
Hostilities Index (SHI) – that were used to rate 198 countries and
self-governing territories on their levels of restrictions during the two-year
period from July 1, 2006, to June 30, 2008. (For more information on the
indexes, see below.) Using the original study as a baseline, this study
assesses increases and decreases in government restrictions and social
hostilities involving religion by comparing each country’s baseline scores on
the two indexes with its scores for an overlapping two-year period from July 1,
2007, to June 30, 2009. Comparing rolling averages for overlapping time periods
reduces the impact of year-to-year fluctuations and helps identify significant
trends (see below).
Determining When Changes on an Index Indicate a Substantial
Change
This
report highlights changes in a country’s average scores on the Government
Restrictions Index and Social Hostilities Index that are deemed to be
“substantial.” Unlike in public opinion surveys, there is no statistically
determined margin of error that can be used to determine whether observed
differences in index scores are significant. Instead, this study defines a
change in a country’s score as substantial according to two criteria.
First,
to be characterized as a substantial change in this study, the change had to be
at least 1.5 standard deviations above or below the mean amount of change among
all 198 countries or territories on each index. As shown in the chart on page
74, 16 countries had changes on the Government Restrictions Index that were 1.5
standard deviations or more above
the mean amount of change, and 14 countries had changes that were 1.5 standard
deviations or more below
the mean amount of change.
As
shown in the chart above, 16 countries had changes on the Social
Hostilities Index that were 1.5 standard deviations or more above the mean amount of change, and 11 countries had
changes that were 1.5 standard deviations or more below the mean amount of change.
Second,
to be characterized as “substantial,” the change also had to be consistent
throughout the full three-year period studied, meaning the change had to be in
the same direction both in the period from mid-2006 to mid-2008 and in the
overlapping period from mid-2007 to mid-2009. This criterion helps to avoid
highlighting short-term spikes that might be due to variability in the coding
or information in the primary sources. (For information on the primary sources,
see Primary Sources below.)
When
both criteria were applied to changes on the GRI, the study found that
government restrictions on religion rose substantially in 14 countries and
decreased substantially in eight. The scores stayed roughly the same in most
(176) countries. When both criteria were applied to changes on the SHI, the
study found that social hostilities involving religion rose substantially in 10
countries and decreased substantially in five. The scores stayed roughly the
same in most (183) countries.
Forum
staff carefully reviewed the coding and primary sources for all countries that
did not meet the consistent change criterion on either the Government
Restrictions Index or the Social Hostilities Index. The review confirmed that
countries eliminated by that criterion were ones where the changes were not
drastic, dramatic or substantive.
Advantage of Using the Mean Amount of Change as the Point for Comparison
As
noted above, the starting point for measuring changes on the indexes for
individual countries is the mean (average) amount of change among all 198 countries
and territories. Using the mean for all countries as the starting point is a
more conservative approach than using “0” as the starting point, especially for
assessing increases in restrictions. This approach does not assume that the
overall global increase of .12 points on the Government Restrictions Index and
the .02 increase on the Social Hostilities Index were necessarily true
increases. Instead, this approach takes into account that those increases might
have resulted from variability in the methodology or more detailed information
in the primary sources.
In practical terms, using the mean amount of change as
the starting point from which changes are assessed means that a country’s index
score had to have numerically increased by more than 1.5 standard deviations
above its previous score to be considered substantial. In the case of the
Government Restrictions Index, a country’s score had to have increased by at
least .68 points, which is 1.5 standard deviations (.56) above the mean amount
of change (.12). Similarly, a country’s score on the Social Hostilities Index
had to have increased by .82 points, which is 1.5 standard deviations (.80)
above the mean amount of change (.02) on that index. By this standard, the
majority of the increases that met the standard-deviation criterion for a
substantial change also met the criterion that it be a consistent change.
Indeed, 14 of the 16 countries met both criteria for a substantial increase on the Government Restriction
Index, and 10 of the 16 countries met both criteria for a substantial increase on the Social Hostilities
Index.
Conversely,
for a decrease to be counted as substantial, the decline needed to
be less than 1.5 standard deviations below its previous score, since the
average change over time was positive (+.12). In the case of the Government
Restrictions Index, a country’s score had to have decreased by at least .44
points, which is 1.5 standard deviations (.56) below the mean amount of change
(+.12). A country’s score on the Social Hostilities Index had to have decreased
by least .78 points, which is 1.5 standard deviations (.80) below the mean
amount of change (+.02). By this standard, only eight of the 14 countries that
met the first criterion for a substantial decrease on the Government Restriction Index also met the
second criterion. Five of the 11 countries that met the first criterion for a
substantial decrease on the Social Hostilities Index also met the second
criterion.
Advantage of Using Rolling Averages
This
three-year study averages the middle year with the first and third years in
order to give greater weight to consistent patterns of change on the individual
questions that make up the indexes. For example, consider a hypothetical
example using one of the 20 questions that make up the Government Restrictions
Index: GRI Q.11, “Was there harassment or intimidation of religious groups by
any level of government?” In hypothetical Scenario 1, there was a consistent
increase in GRI Q.11 across the three individually coded years of data from the
lowest intensity score (“No” = 0) in the first year, to the middle level of
intensity (“Yes, there was limited harassment”
= .250) in the second year, and then to the highest intensity (“Yes, there was widespread harassment” = .500)28 in the third year. The average of the second and
third years in Scenario 1 is .375, an increase of .250 points from the average
of .125 for the first and second years.
The
same scores are present in Scenario 2 in the table above, but in an inconsistent
order. Instead of consistently increasing from low to high across the three
years as in Scenario 1, the first year in Scenario 2 has the middle level of
intensity (“Yes, there was limited
harassment” = .250), then drops to low intensity (“No” = 0) in the second year,
but ends with high intensity (“Yes, there was widespread harassment” = .500) in the third year. This inconsistent pattern of change results in an overall smaller
amount of change between rolling averages (an increase of .125 in Scenario 2
compared with .250 in Scenario 1). This is because the lowest level in Scenario
2 occurred in the second year instead of the first year in Scenario 1, making
the average for the period ending in June 2009 in Scenario 2 (.250) smaller
than the average in Scenario 1 (.375). However, the rolling average in both
Scenarios for the period ending June 2008 (.125) is unaffected because the
middle level of intensity on GRI Q.11 still occurred, just in the first year
instead of the second.
Similarly,
the importance of a consistent pattern of change can be seen when looking at a
hypothetical example using one of the 13 questions from the Social Hostilities
Index (SHI Q.2, “Was there mob violence related to religion?”).29 In SHI Q.2 Scenario 1, when the intensity of
hostilities consistently increased across the three individually coded years of
data, the amount of change was greater than when the same scores were present
but in an inconsistent order (.385 versus .287).
Overview of Procedures
The
methodology used by the Pew Forum to assess and compare restrictions on
religion was developed by Senior Researcher and Director of Cross-National Data
Brian J. Grim in consultation with other members of the Pew Research Center
staff, building on a methodology that Grim and Prof. Roger Finke developed
while at Penn State University’s Association of Religion Data Archives.30 The goal was to devise quantifiable, objective and
transparent measures of the extent to which governments and societal groups
impinge on the practice of religion. The findings were used to rate 198
countries and self-governing territories on two indexes that are reproducible
and can be periodically updated.
This
research goes beyond previous efforts to assess restrictions on religion in
several ways. First, the Pew Forum coded (categorized and counted) data from 18
published cross-national sources, providing a high degree of confidence in the
findings. The Pew Forum’s coders looked to the sources only for specific,
well-documented facts, not for opinions or commentary.
Second,
the Pew Forum’s staff used extensive data-verification checks that reflect
generally accepted best practices for such studies, such as double-blind coding
(coders do not see each other’s ratings), inter-rater reliability assessments
(checking for consistency among coders) and carefully monitored protocols to
reconcile discrepancies between coders.
Third,
the Pew Forum’s coding took into account whether the perpetrators of
religion-related violence were governmental or private actors. The coding also
identified how widespread and intensive the restrictions were in each country.
Fourth,
one of the most valuable contributions of the indexes and the questions is
their ability to chart change over time, as discussed earlier in the methodology.
Countries and Territories
The
Pew Forum study covers a total of 198 countries and territories. These include
all 192 states that were members of the United Nations during the period under
examination (mid-2006 to mid-2009) with the exception of North Korea, for which
sufficiently precise and timely data was not available. In addition, the study
includes seven territories: Hong Kong, Macau, Taiwan, the Palestinian
territories, Kosovo, Western Sahara and Northern Cyprus. These are treated as
separate entities, for various reasons, by some or all of the primary
information sources for this study. The U.S. State Department, for example,
reports separately on Northern Cyprus because it has been administered by
Turkish Cypriot authorities since 1974.
Although
the 198 countries and territories vary widely in size, population, wealth,
ethnic diversity, religious makeup and form of government, the study does not
attempt to adjust for such differences. Poor countries are not scored
differently on the indexes than wealthy ones. Countries with diverse ethnic and
religious populations are not “expected” to have more social hostilities than
countries with more homogeneous populations. And democracies are not assessed
more leniently or harshly than authoritarian regimes.
Information Sources
The
Pew Forum identified 18 widely available, frequently cited sources of
information on government restrictions and social hostilities involving
religion around the world. This study includes three sources that were not used
in the baseline report on religious restrictions. (See below for more details
on the new information sources.)
The
primary sources, which are listed below, include reports from U.S. government
agencies, several independent, nongovernmental organizations and a variety of
European and United Nations bodies. Although most of these organizations are
based in Western countries, many of them depend on local staff to collect
information across the globe. As previously noted, the Pew Forum did not use
the commentaries, opinions or normative judgments of the sources; the sources
were combed only for factual information on specific policies and actions.
Primary Sources
1.
Country constitutions
2.
U.S. State Department annual reports on International Religious Freedom
3.
U.S. Commission on International Religious Freedom annual reports
4.
U.N. Special Rapporteur on Freedom of Religion or Belief reports
5.
Human Rights First reports in first and second years of coding; Freedom House
reports in third year of coding
6.
Hudson Institute publication: Religious Freedom in the World (Paul Marshall)
7.
Human Rights Watch topical reports
8.
International Crisis Group country reports
9.
United Kingdom Foreign & Commonwealth Office annual report on human rights
10.
Council of the European Union annual report on human rights
11.
Amnesty International reports
12.
European Network Against Racism Shadow Reports
13.
United Nations High Commissioner for Refugees reports
14.
U.S. State Department annual Country Reports on Terrorism
15.
Anti-Defamation League reports
16.
U.S. State Department Country Reports on Human Rights Practices
17.
U.S. National Counterterrorism Center’s Worldwide Incident Tracking System
18.
Uppsala University’s Uppsala Conflict Data Program, Armed Conflict Database
U.S. government reports with
information on the situation in the United States
19.
Dept. of Justice Report on Enforcement of Laws Protecting Religious Freedom
2000-2006
20.
Department of Justice “Religious Freedom in Focus” newsletters
21.
FBI Hate Crime Reports
As
noted above, this study includes three sources that were not included in the
Pew Forum’s first report on global restrictions on religion: Freedom House
reports; Uppsala University’s Uppsala Conflict Data Program, Armed Conflict
Database; and the Worldwide Incident Tracking System (WITS), a publicly
available database maintained by the U.S. National Counterterrorism Center, a
U.S. government organization that is part of the Office of the Director of
National Intelligence. The new sources were used for the most recent year of
coding included in this study, July 1, 2008, to June 30, 2009.
The
Freedom House reports replaced the Human Rights First reports, which have not
been updated since mid-2008. The Uppsala Armed Conflict Database provided a
more reliable source of information on the number of people affected by
religion-related armed conflicts. While other primary sources report the
effects of such conflicts, they sometimes provide only ballpark estimates on
the number of people affected by such conflicts.
Some
of the apparent increase in religion-related terrorism noted in this study
could reflect the use of the WITS database, which provided greater detail on
the number of people affected by religion-related terrorism than the State Department’s
International Religious Freedom reports or the U.S. State Department’s annual
Country Reports on Terrorism, which were the primary sources used to code
terrorist incidents in the baseline report.
As
explained in more detail below, the Pew Forum’s staff developed a battery of
questions similar to a survey questionnaire. Coders consulted the primary
sources in order to answer the questions separately for each country. While the
U.S. State Department’s annual reports on International Religious Freedom
generally contained the most comprehensive information, the other sources
provided additional factual detail that was used to settle ambiguities, resolve
contradictions and help in the proper scoring of each question.
The
questionnaire, or coding instrument, generated a set of numerical measures on
restrictions in each country. It also made it possible to see how government
restrictions intersect with broader social tensions and incidents of violence
or intimidation by private actors. The coding instrument with the list of
questions used for this report is shown in the Summary of Results.
The
coding process required the coders to check all the sources for each country.
Coders determined whether each source:
provided information critical to assigning a score; had supporting information
but did not result in new facts; or had no available information on that
particular country. Multiple sources of information were available for all
countries and self-administering territories with populations greater than 1
million. More than three-in-four of the countries and territories analyzed by
the Pew Forum were multi-sourced; only small, predominantly island, countries
had a single source, namely, the U.S. State Department reports.
Coding
the United States presented a special problem since it is not included in the
State Department’s annual reports on International Religious Freedom.
Accordingly, the Pew Forum’s coders also looked at reports from the Department
of Justice and the FBI on violations of religious freedom in the United States,
in addition to consulting all of the primary sources, including reports by the
United Nations, Human Rights Watch, Freedom House, the International Crisis
Group and the U.K. Foreign & Commonwealth Office, many of which do contain
data on the United States.
The Coding Process
The
Pew Forum employed strict training and rigorous coding protocols to make its
coding as objective and reproducible as possible. Coders worked directly under
a senior researcher’s supervision, with additional direction and support
provided by other Pew Forum researchers. The coders underwent an intensive
training period that included a thorough overview of the research objectives,
information sources and methodology.
Countries were double-blind coded by two coders (coders did not see each
other’s ratings), and the initial ratings were entered into Microsoft Excel
spreadsheets. The coders began by filling out the coding instrument for each
country using the information source that had the most comprehensive
information, typically the U.S. State Department reports. The protocol for each
coder was to answer every question on which information was available in the
initial source. Once a coder had completed that process, he or she then turned
to the other sources. As new information was found, this was also coded and the
source duly noted. Whenever ambiguities or contradictions arose, the source
providing the most detailed, clearly documented evidence was used.
After
two coders had separately completed the coding instrument for a particular
country, their scores were compared by a senior researcher. Areas of
discrepancy were discussed at length with the coders and were reconciled in
order to arrive at a single score on each question for each country. The Excel
worksheets for each country were then combined into a master file, which was
imported into SPSS.
Throughout
this process, the coding instrument itself was continually monitored for
possible defects. The questions were designed to be precise, comprehensive and
objective so that, based on the same data and definitions, the coding could be
reliably reproduced by others with the same results.
Pew
Forum staff generally found few cases in which one source contradicted another.
When contradictions did arise – such as when sources provided differing
estimates of the number of people displaced due to religion-related violence –
the source that cited the most specific documentation was used. The coders were
instructed to disregard broad, unsubstantiated generalizations regarding abuses
and to focus on reports that contained clear, precise documentation and factual
detail, such as names, dates and places where incidents occurred.
Inter-rater
reliability statistics were computed by comparing the coders’ independent,
blind ratings. The Pew Forum took scores from one coder for the 198 countries
and compared them with another coder’s scores for the same questions, computing
the degree to which the scores matched. These measures were very high, with an
average score of .8 or above on the key variables. Scores above .8 on a 0-to-1
scale are generally considered very good, and scores around .7 are generally
acceptable. The Pew Forum’s overall inter-rater reliability average across all
the variables coded was greater than .8 for each year.
The
data-verification procedures, however, went beyond the inter-rater reliability
statistics. They also involved comparing the answers on the main measures for
each country with other closely related questions in the dataset. This provided
a practical way to test the internal reliability of the data.
Pew
Forum staff also checked the reliability of the Pew Forum’s coded data by
comparing them with similar, though more limited, religious restrictions
datasets. In particular, published government and social regulation of religion
index scores are available from the Association of Religion Data Archives (for
three years of data) and the Hudson Institute (for one year of data), which
makes them ideal measures for cross validation. The review process found very
few significant discrepancies in the coded data; changes were made only if
warranted by a further review of the primary sources.
Restriction of Religion Indexes
The
Government Restrictions Index is based on 20 indicators of ways that national
and local governments restrict religion, including through coercion and force.
The Social Hostilities Index is based on 13 indicators of ways in which private
individuals and social groups infringe on religious beliefs and practices, including
religiously biased crimes, mob violence and efforts to stop particular
religious groups from growing or operating. The study also counted the number
and types of documented incidents of religion-related violence, including
terrorism and armed conflict.
Government Restrictions Index
Coding
multiple indicators makes it possible to construct a Government Restrictions
Index of sufficient gradation to allow for meaningful cross-national
comparisons. An additional advantage of using multiple indicators is that it
helps mitigate the effects of measurement error in any one variable, providing
greater confidence in the overall measure.
The
Pew Forum coded 20 indicators of government restrictions on religion (see the
Summary of Results). In two cases, these items represent an aggregation of
several closely related questions: Measures of five types of physical abuses
are combined into a single variable (GRI Q.19); and seven questions measuring
aspects of government favoritism are combined into an overall favoritism scale
(GRI Q.20 is a summary variable showing whether a country received the maximum
score on any one or more of the seven questions). These 20 items were added
together to create the GRI.
A test of whether the 20 items were statistically
reliable as a single index produced a scale reliability coefficient of greater
than .9 for each year. Since coefficients of .7 or higher are generally
considered acceptable, it was appropriate to combine these 20 items into a
single index.
The
GRI is a fine-grained measure created by adding the 20 items on a 0-to-10
metric, with 0 indicating very low government restrictions on religion and 10
indicating extremely high restrictions. This involved two general calculations.
First, the 20 questions that form the GRI were standardized so that each
variable had an identical maximum value of 1 point, while gradations among the
answers allowed for partial points to be given for lesser degrees of the
particular government restriction being measured. Second, the overall value of
the index was proportionally adjusted so that it had a maximum value of 10 and
a possible range of 0 to 10 by dividing the sum of the variables by 2.
Social Hostilities Index
In
addition to government restrictions, violence and intimidation in societies
also can limit religious beliefs and practices. Accordingly, Pew Forum staff
tracked more than a dozen indicators of social impediments on religion. Once
again, coding multiple indicators made it possible to construct an index that
shows gradations of severity or intensity and allows for comparisons between
countries. The Summary of Results contains the 13 items used by Pew Forum staff
to create the Social Hostilities Index.
As with the Government Restrictions Index, various types of violence and
intimidation were combined. A test of whether these 13 items were statistically
reliable as a single index produced a scale reliability coefficient of .9 or
higher for each year. Since coefficients of .7 or higher are generally
considered acceptable, it was statistically appropriate to combine these items
into a single index.
The
SHI was constructed by adding together the 13 indicators based on a 0-to-10
metric, with 0 indicating very low social impediments to religious beliefs and
practices and 10 indicating extremely high impediments. This involved two
general calculations. First, the various questions that form the index were
standardized so that each variable had an identical maximum value of 1 point,
while gradations among the answers allowed for partial points to be given for
lesser degrees of the particular hostilities being measured. Second, the
indicators were added together and set to have a possible range of 0 to 10 by
dividing the sum of the variables by 1.3.
Levels of Restrictions
The
Pew Forum categorized the levels of government restrictions and social
hostilities by percentiles. Countries with scores in the top 5% on each index
are categorized as “very high.” The next highest 15% of scores are categorized
as “high,” and the following 20% are categorized as “moderate.” The bottom 60%
of scores are categorized as “low.” Readers should note that since the indexes
measure the accumulated impact and severity of restrictions, distinctions among
the scores of the countries in the bottom 60% of scores are less significant
than distinctions made at the upper end of the indexes, where differences in
the number and severity of restrictions between countries are greater. This is
evident by the fact that the range of difference between scores of countries in
the entire bottom 60% (0.1-2.4 on the GRI and 0-1.8 on the SHI) is about the
same as the range of differences between scores in just the top 5% (7.2-8.3 on
the GRI and 7.2-9 on the SHI).
Notes on Changes in Questions
Readers
should be cautioned that some differences on individual measures may not be as
significant as they appear due to minor fluctuations in coding procedures. This
was especially the case for GRI Q.3 and SHI Q.5. As shown in the Summary of
Results for GRI Q.3 (“Taken together, how do the constitution/basic law and
other national laws and policies affect religious freedom?”), the number of
countries where coders answered “0” on that question increased from 53
countries in the two-year period ending in mid-2008 to 75 countries in the
two-year period ending in mid-2009.31 However, as noted in the Summary of Results, this
change was likely attributable to a slight variation in the coding procedures
across the years. During the first year coded (July 1, 2006, to June 30, 2007),
the coders were more likely to give a “1” on this question than in either of
the two subsequent years.32 A post-coding analysis suggests that this was because
the coders in that year were more likely to code the presence of a few
restrictions on religious freedom by the government as a “1”. In subsequent
years the coders had a higher bar for coding “1”: the presence of restrictions
alone was not sufficient, there also had to be clear harassment or abuses
toward religious groups or their members.
As
noted earlier in the methodology, some of the increase in religion-related
terrorism (SHI Q.5) found in this study could reflect the use of new source
material that provided greater detail on terrorist activities than the sources
used in the baseline report. Specifically, in coding terrorist activities
during the period from July 1, 2008, to June 30, 2009, coders used the
Worldwide Incidents Tracking System (WITS) database, which is the U.S. National
Counterterrorism Center’s database of terrorist incidents.
Changes
in two other questions are important to note. The first two variables in the
Government Restrictions Index on the presence of laws protecting religious
freedom (GRI Q.1) and constitutional qualifications or contradictions of those
protections (GRI Q.2) were back-coded so that the coding protocols for the
period from mid-2006 to mid-2008 matched the protocols for the period from
mid-2007 to mid-2009. This resulted in small changes to the mid-2008 Government
Restrictions Index scores reported in this study from those reported in the
baseline study (the mid-2008 index scores reported in this study reflect the
revised coding). While some countries changed or amended their constitutions
during the three-year period covered by this study, none of those changes affected
the results of the coding, and any differences between the revised results and
those reported in the baseline study do not represent changes but rather a
slight revision in the methodology.
Finally,
it is important to note that situations within countries may have changed since
the end of the periods studied. One significant change that occurred outside
the time frame of this study, for example, is the division of Sudan into two
separate countries. Subsequent Pew Forum reports on global restrictions on
religion will assess the South and the North separately; in this report,
however, Sudan’s score represents the overall country in which the government
in Khartoum was dominant.
New Questions Added to the Study
This
report includes one new question that is not part of the indexes and was not
included in the baseline report: the number of countries with laws penalizing
blasphemy, apostasy or the defamation of religion. This question was coded only
in the last year of the study (mid-2008 to mid-2009), but all the laws and
policies coded predated July 1, 2008. This question is analyzed separately in
the section beginning on page 67.
Additionally,
this study reports on the number of countries where specific religious groups
faced government or social harassment. This is essentially a cross-tabulation
of GRI Q.11 (“Was there harassment or intimidation of religious groups by any
level of government?”) and SHI Q.1a. (“Did individuals face harassment or
intimidation motivated by religious hatred or bias?”). For purposes of this
study, the definition of harassment includes any mention in the primary sources
of an offense against an individual or group based on religious identity. Such
offenses may range from physical attacks and direct coercion to more subtle forms
of discrimination. But merely prejudicial opinions or attitudes, in and of
themselves, do not constitute harassment unless they are acted upon in a
palpable way.
As noted above, this study provides data on the number
of countries in which different religious groups are harassed or intimidated.
But the study does not assess either the severity or the frequency of the
harassment in each country. Therefore, the results should not be interpreted as
gauging which religious group faces the most harassment or persecution around
the world.
Religion-Related Terrorism and Armed Conflict
Terrorism
and war can have huge direct and indirect effects on religious groups,
destroying religious sites, displacing whole communities and inflaming
sectarian passions. Accordingly, the Pew Forum tallied the number, location and
consequences of religion-related terrorism and armed conflict around the world,
as reported in the same primary sources used to document other forms of
intimidation and violence. However, war and terrorism are sufficiently complex
that it is not always possible to determine the degree to which they are
religiously motivated or state sponsored. Out of an abundance of caution, this
study does not include them in the Government Restrictions Index. They are
factored instead into the index of social hostilities involving religion, which
includes one question specifically about religion-related terrorism and one
question specifically about religion-related war or armed conflict. In
addition, other measures in both indexes are likely to pick up spillover
effects of war and terrorism on the level of religious tensions in society. For
example, hate crimes, mob violence and sectarian fighting that occur in the
aftermath of a terrorist attack or in the context of a religion-related war
would be counted in the Social Hostilities Index, and laws or policies that
clearly discriminate against a particular religious group would be registered
on the Government Restrictions Index.
For
the purposes of this study, the term religion-related terrorism is defined as
premeditated, politically motivated violence perpetrated against noncombatant
targets by subnational groups or clandestine agents that have some identifiable
religious ideology or religious motivation. It also includes acts carried out
by groups that have a non-religious identity but target religious personnel,
such as clergy. Readers should note that it is the political character and
motivation of the groups, not solely the type of violence, that is at issue here.
For instance, a bombing would not be classified as religion-related terrorism
if there was no clearly discernible religious ideology or bias behind it unless
it was directed at religious personnel. Religion-related war or armed conflict
is defined as armed conflict (a conflict that involves sustained casualties
over time or more than 1,000 battle deaths) in which religious rhetoric is
commonly used to justify the use of force, or in which one or more of the
combatants primarily identifies itself or the opposing side by religion.
Potential Biases
As
noted in the report, the primary sources indicate that the North Korean
government is among the most repressive in the world, including toward
religion. Because of independent observers’ lack of regular access to North
Korea, however, the sources are unable to provide the kind of specific, timely
information that forms the basis of this report. Therefore, North Korea is not
included on either index.
This
raises two important issues concerning potential information bias in the
sources. The first is whether other countries that limit outsiders’ access and
that may seek to obscure or distort their record on religious restrictions were
adequately covered by the sources. Countries with relatively limited access have
multiple primary sources of information that the Pew Forum used for its coding.
Each is also covered by other secondary quantitative datasets on religious
restrictions that have used a similar coding scheme, including earlier years of
coded data from U.S. State Department reports previously produced by Grim at
Penn State’s Association of Religion Data Archives (ARDA) project (three
datasets); independent coding by experts at the Hudson Institute’s Center for
Religious Liberty using indexes also available from ARDA (one dataset); and
content analysis of country constitutions conducted by the Becket Fund for
Religious Liberty (one dataset). Pew Forum staff used these for
cross-validation. Contrary to what one might expect, therefore, even most
countries that limit access to information tend to receive fairly extensive
coverage by groups that monitor religious restrictions.
The
second key question – the flipside of the first – is whether countries that
provide freer access to information receive worse scores simply because more
information is available on them. As described more fully in the methodology
from the baseline report, Forum staff compared the length of U.S. State
Department reports on freer-access countries with those of less-free countries.
The comparison found that the median number of words was approximately three
times as large for the limited-access countries as for the open-access
countries. This confirms that problems in freer-access countries are generally
not overreported in the U.S. State Department reports.
Only
when it comes to religion-related violence and intimidation in society do the
sources report more problems in the freer-access countries than in the
limited-access ones. However, the Social Hostilities Index includes several measures
– such as SHI Q.8 (Did religious groups themselves attempt to prevent other
religious groups from being able to operate?) and SHI Q. 11 (Were women
harassed for violating religious dress codes?) – that are less susceptible to
such reporting bias because they capture general social trends or attitudes as
well as specific incidents of violence. With these limitations in mind, it
appears that the coded information on social hostilities is a fair gauge of the
situation in the vast majority of countries and a valuable complement to the
information on government restrictions.
Data
on social impediments to religious practice can more confidently be used to
make comparisons between countries with sufficient openness, which includes
more than nine-in-ten countries covered in the Pew Forum’s coding. An analysis
by Grim and Richard Wike, Associate Director of the Pew Research Center’s
Global Attitudes Project, tested the reliability of the State Department
reports on social impediments to religious practice by comparing public opinion
data with data coded from the reports in previous years by Grim and experts at
Penn State. They concluded that “the understanding of social religious
intolerance embodied in the State Department reports is comparable with the results
of population surveys and individual expert opinion.”33
Finally,
the 2010 population figures used in this report are estimates from the United
Nations World Population Prospects, 2008 Revision. A new revision was released
in April 2011, after most of the analysis for this report was completed.
Footnotes:
28 Note that the
maximum value for a question on the 20-question, 10-point Government
Restrictions Index is .500 (20 x .500 = 10). (return to text)
29 Note that the maximum value for a
question on the 13-question, 10-point Social Hostilities Index is .770 (13
questions x .770 = 10 points). (return to text)
30 See Brian J Grim and Roger Finke,
“International Religion Indexes: Government Regulation, Government Favoritism, and Social Regulation of Religion,” Interdisciplinary Journal of Research
on Religion,
Volume 2, Article 1, 2006. (return to text)
31 0 = “National laws and policies provide
for religious freedom, and the national government respects religious freedom
in practice.” (return to text)
32 1 = “National laws and policies
provide for religious freedom, and the national government generally respects
religious freedom in practice; but there are some instances, e.g., in certain
localities, where religious freedom is not respected in practice.” (return to text)
33 See Brian J Grim and Richard Wike,
“Cross-Validating Measures of Global Religious Intolerance: Comparing Coded
State Department Reports With Survey Data and Expert Opinion,” Politics and Religion, Volume 3, Issue 1, pages 102-129,
April 2010. (return to text)