Types of Indirect Assessment

Direct assessment of literacy can provide reliable and valid information about people's literacy, if designed well. However, such measures are often costly and time consuming. Few if any countries can afford to use them every year. As a result, most countries base their official literacy statistics on indirect (proxy) measures that enable rough estimation of literacy levels in a nation.

There are two basic types of indirect assessment:

• Self-report measures are based on what people say about their own literacy skills (or educational background).
  
• Existing data measures are based on statistical information kept by a government, usually records about the number of years spent in school.

back to top

Self-Report Measures

Self-report measures of literacy are based on information collected when citizens answer simple questions about their skills, as part of a survey (involving a large sample) or a census (involving the whole population). Here are some examples (try to answer each one):

1. Can you sign your name? (yes.....no.... don't know)
2. Can you read/write? (yes.....no.... don't know)
3. How well do you read? (very well......quite well....poorly... not at all)
4. On a scale of 1 to 5, 1 being poor and 5 being excellent, how would you rate your reading and writing skills in [English]?
5. All things considered, are you satisfied or dissatisfied with your reading and writing skills in [English]? (yes...no....don't know)
6. Are your reading skills adequate for your job? (yes...no....don't know)
7. How many years did you study in school?

Ask yourself (a) what aspect of literacy each question measures here, and (b) how well each question measures that aspect of literacy. In general, would you be willing to report your country's literacy status based on such questions?

A Critique of Self-Report Measures

People's self-report about literacy skills may at first seem useful. However, there are serious limitations to self-reports:

Definitions are unclear. The meaning of "can read" or "can write" may be different for the interviewer asking and the person answering. Also, it is sometimes unclear whether literacy in a "non-official" (local) language "counts" as literacy in a survey.

Averaging is difficult. Respondents asked to summarize their abilities over many situations may have trouble figuring out the "average" ability, in part because functional abilities may vary between situations.

People overestimate. Several studies found that people tend to report higher levels of literacy than their actual performance on literacy tests. (People with low skills may be quite satisfied with their skills, probably because they do not function in situations that require high skills. They perceive their [limited] skills as adequate for their situation.)

back to top

Indirect Assessment: Further Issues

When international agencies want to report information about literacy in all world countries, they face a challenge. Countries do not use direct measures of literacy (which are more accurate and valid) on a regular basis. Countries normally publish literacy statistics based only on indirect information collected as part of a census. This causes serious problems:

1. A census is usually conducted only once every 10 years. Some countries have reported data from 7-8 censuses over the last century so these countries' literacy trends can be easily detected. But most countries have reported data from only 1 to 3 censuses so their trends may not show well. Over 30 countries have never published any literacy data.
2. Some countries report the percentage of literate citizens based on one type of indirect measure (e.g., self-report), while others report literacy levels using school records. Further, some may report the percentage of literates based on the number of people with more than 4 years of schooling, while others may report the percentage of literates based on the number of people with over 5 (or 8) years of schooling.

Summary: Problems in Current Reporting Systems

It must be understood that official statistics published by international and national agencies are less accurate than they appear to be, despite best intentions, as they are not based on the same kind of data for all countries.

• Different countries use different types of indirect literacy measures.
• Indirect measures of literacy are problematic (see earlier discussion).
• Countries use different criteria to decide whether people are literate.
• U.N. agencies do not have the same number of "data points" for all countries.
• Not all countries report data every 10 years.

These problems force international agency experts to estimate literacy levels for many countries, as they are not given complete and comparable data for 10 year intervals. Often, there is a need to use statistical projections (e.g., if a country did not supply new literacy statistics).

What are the implications? Should we trust published literacy data? Think about it.

back to top

Now read about Direct Assessment

Back to Assessment Issues or move on to The Limitations of Literacy Statistics