The Government of Bangladesh had initiated a program to identify the challenges of the public social safety net (PSSN) program with regards to coverage, targeting, leakage and program administration. As a first step to this initiative, the Government had mandated the Bangladesh Bureau of Statistics (BBS) to develop a National Household Poverty Register (NHPR). Before commencing the ultimate task, the BBS was carrying out a field test phase to evaluate the process of data collection for the NHPR. The test phase was implemented in Manikganj upazila in Manikganj and Nagarpur upazila in Tangail. It was started on March 2013 and ended on May 2014.

Project Description: The spot check was a quantitative and analytical exercise based on a re-survey of a representative random sample of the households covered in the test phase. Mitra and Associates repeated the data collection exercise with the same instrument used by BBS. 

The process evaluation assessed the process of selection and mobilization of logistical staffs, training of staffs, and preparation of the fieldwork logistics plan, public awareness, data collection and quality control of field work of the test phase. To do the evaluation Mitra observed the field activities and interviewed the concerned personnel of the test phase. 

Tasks carried out by Mitra and Associates:

Methodology (sample design & tools): For Spot Check Survey the sample size was set at 2,000 households to have the survey data from a minimum of 1,838 households in the sample. As there was no prior knowledge regarding the values of the parameters to be estimated from the survey, the sample needed to have a minimum of 1,838 households to ensure estimation of any parameter with an error margin at or below ± 2 percentage points. About 10 percent of households selected into the sample were likely to be non-responsive for some reasons. The sample drew in 2 strata by using each of the two upazilas as a stratum, and by allocating the sample between the two strata (two upazilas) in proportion to their number of households, out of the 2000 households, 396 drew from Manikganj upazila and 1,604 from Nagarpur upazila.

The sample drew in two stages, with the households (basic sampling units) being selected in the second stage. The first-stage sampling unit was a cluster of 200-300 households. A cluster formed with one or more Mauzas or part of a Mauza in rural areas. In urban areas it made up of one or more Mohallahs or part of a Mohallah. Following the sampling strategies used in the Bangladesh Demographic and Health Survey (BDHS), and for convenience of fieldwork, 24 households included into the sample from a cluster.Thus, the sample made up of 83 clusters with 67 clusters from Nagarpur upazila and 16 from Manikganj upazila. Clusters selected independently for each stratum, with PPES (Probability Proportional to Estimated Size) method of selection. The sampling frame for selection of the clusters for a stratum (an upazila) created using the up-to-date list of Mauzas/Villages by households for the stratum.  

The first step in selecting households from a cluster was to find out and locate the cluster and identify its geographical boundaries. The next step was selecting 30 households from the cluster. There was no household listing done in a cluster for the selection of households. Firstly because, it would not be feasible to conduct the listing operation with the implementation time period allowed for the survey; secondly, the listing operation might raise the survey costs. In view of this, it decided to select households from a cluster following the strategy of the WHO cluster sampling method.

For process evaluation Mitra worked in 20 clusters in Nagarpur and 10 clusters in Manikganj upazila randomly selected, among the clusters selected for spot check survey. In those clusters Mitra interviewed about 60 enumerators, 10-12 supervisors and 20 coordinators of the field test phase conducted by BBS. Another group of respondents consists of high official of BBS was also be interviewed as a part of process evaluation.

Questionnaires and others: In spot check survey the same instrument was used like BBS. To do the process evaluation Mitra developed an observation checklist and 4 types of questionnaire to be used for interviewing – Enumerators; Supervisors; Coordinators; and Concerned high official of BBS.

The questionnaire was pre-tested before it was finally accepted for use in the collection of actual data. The pretest was conducted under the care of a committee, in some purposively selected area.

Training & Data collection & quality control: Supervisors and enumerators were given training together on data collection for an enumeration area. The training was conducted at a most commonly in the Union/Ward office and duration was 3 days on process evaluation and 1 day for spot check survey. The number of trainees trained in a batch varied widely ranging from fewer than 20 to more than 40. 

For data collection, Manikgonj Sadar was divided into 38 enumeration areas and Nagarpur into 166 enumeration areas and 23 days needed to complete the data collection.

Data processing and reporting: Registration of questionnaires, edit, double data entry by Cspro software, cleaning and analysis by spss software and reporting.

Client Reference: The World Bank, E-32, Agargaon, Sher-e-Bangla Nagar, Dhaka 1207 represented by Mr. Mahtab Alam, Program Assistant