International Business & Technical Consultants, Inc. (IBTCI), a U.S. based international development consulting company established in 1987, has worked in over one hundred countries and has implemented over two hundred projects. IBTCI has served governments, private sector companies and several donor agencies in the practice area of Monitoring & Evaluation (M&E) across many sectors including health, education, environment & economic development and crisis, conflict & governance.
MECC is a five-year contract implemented by IBTCI and its subcontractor, Management Systems International (MSI), to provide monitoring, evaluation, analytic, coordination, and research services to USAID/DRC’s Country Development Cooperation Strategy (CDCS), and the USAID/DFID joint education program (ACCELERE!). It is intended to support Mission-wide objectives and provide analytical tools and services that allows the Mission to undertake performance management that will guide them in the full implementation of the CDCS PMP and ACCELERE! results framework. MECC services include performance monitoring, support to the Mission’s Learning Agenda, coordination among stakeholders at the provincial level, and M&E capacity building for Mission staff, IPs, and other relevant stakeholders.
USAID/DRC regularly collects performance data from its implementing partners (IPs) and analyzes it to make management decisions. Since poor quality data could affect conclusions about performance and lead to faulty decision making, USAID requires that all Missions / Offices conduct regular Data Quality Assessments (DQA). According to ADS 201, the purpose of a DQA is to: “…ensure that the USAID staff are aware of the: strengths and weaknesses of the data, as determined by applying the five data quality standards; and extent to which the data integrity can be trusted to influence management decisions” (ADS 184.108.40.206).
A DQA often entails site visits to physically inspect records maintained by IPs. However, travel restrictions for USAID staff in DRC limit the ability of AORs and CORs to conduct site visits. Thus, the second purpose of this task is to form joint teams consisting of both USAID staff and a contracted DQA Team so that in-depth DQAs can be conducted.
 The five Agency quality standards are validity, integrity, precision, reliability, and timeliness.
Assignment: The DQA Specialist will plan and conduct DQAs for up to 26 Health indicators contributed by 13 implementing mechanisms. The DQA Specialist will also provide associated training for USAID staff.
Step 1: Desk Review
The Consultant must review relevant documentation, including: past DQAs; the USAID/DRC Performance Management Plan; IP Monitoring & Evaluation Plans; Indicator Tracking Tables (ITT); implementing partner quarterly reports; and old and recently revised F indicator lists and definitions. The Consultant must then revise, if necessary, MECC DQA training materials. The training curricula should address significant issues identified in the most recent DQA, and must be subject to USAID review.
Step 2: Training of USAID staff
Once the training material is approved by USAID, the Consultant must provide a training session for relevant USAID staff. A list of all trainees will be provided by USAID prior to the trainings, but the maximum number of participants in that training will be 15. The trainings may use a mix of teaching techniques such as lectures, discussions and practice exercises to ensure that trainees are able to internalize information provided and apply their learning.
The following trainings and workshops are anticipated: DQA Training workshop; Data Analysis training workshop; and Data Validation workshop.
Step 3: Preparation for meetings with IPs
The MECC DQA staff will make sure right communication takes place between CORs/AORs and IPs around the DQA process and schedule. They will be led by the DQA Specialist. Also, the latter must meet with each DQA team to ensure that the relevant USAID staff are ready to participate on the DQA. This may include a review of each indicator to identify information that is known and the information that needs to be collected or clarified; review of Performance Indicator Reference Sheets (PIRS) associated with each indicator; review of recommendations made for indicators that were assessed in the most recent DQA; a practice interview; clarification of roles and responsibilities on each team; and review of tools/questionnaires to be used with relevant IPs.
Step 4: Meetings with partners and site visits
The DQA teams must visit the main offices of each IP. While USAID staff is expected to take the lead in administering the questionnaire, the DQA Specialist must provide support and oversight to ensure that all necessary information is collected. During these visits, the DQA teams must assess the data collection system in place, the qualification of the staff that collect, analyze and store data, and the data storage system.
Where feasible, the DQA teams must visit at least two sites where data is generated or stored for each of the indicators. These sites could include regional offices of the IP, offices of subcontractors, or an intervention site where data collection is taking place. If USAID staff is unable to travel to the selected site, the MECC DQA staff must travel to the site on his/her own and report findings to the other members of the relevant DQA team. Teams must compare a sample of data reported to USAID in the past against data found at the IP offices and data found in the field.
Step 5: Data Analysis
The MECC DQA staff must meet with DQA team members to review each indicator against the quality standards and complete a DQA checklist for each indicator. This analysis must include a review of any changes that IPs have implemented in response to recommendations made in the 2014 and 2015 DQAs, including actions taken in response to recommendations, and an analysis of their success, as applicable. Based on this review, the DQA Specialist must work with the DQA teams to identify recommendations for both USAID and IPs to address any emerging data quality issues. At this time, the MECC DQA staff must facilitate COR/AORs working with IPs to update PIRS as needed, as they must be included in the DQA Report.
Step 6: Documentation and Reporting
The DQA Specialist must submit a detailed DQA report to MECC, based on discussions with the DQA team members. The report must: (i) outline the overall approach and methodology used in conducting the DQA; (ii) present findings in terms of the five data quality standards for each indicator, using MECC’s categorization system; (iii) summarize recommendations for improving data quality; and (iv) include a completed DQA checklist for each of the indicators. The report must also include a review of responses to recommendations made in the last DQA, when they are available, and progress toward achieving them.
The resultant DQA report will inform the completion of USAID/DRC’s FY 2017 Performance Plan and Report. It will also provide recommendations for relevant IPs to 'improve the quality of data collection, analysis, and reporting.
Reporting: The DQA Specialist will report to the MECC Chief of Party or her/his designee.
Place of Performance: The place of performance is DRC. Trainings and workshops will take place in Kinshasa; space to be provided by MECC for training and data analysis meetings. Based on the location of IP offices and activities, it is expected that the DQA Specialist will need to travel to some of the sites for the actual DQA process.