How BRAC uses field data for continuous improvement

June 28, 2018

Reading Time: 5 minutes

When BRAC, one of the world’s largest NGOs, tried to digitize its paper-based record system to improve decision making and provide continuous improvement for its skills development programme in Bangladesh, it hit a wall.

The programme enables women and men from disadvantaged communities to have better jobs by improving their skills, their income and their workplaces. It focuses on apprenticeship and entrepreneurship based interventions for the informal economy, enterprise development in the light engineering sector and building a pool of skilled professionals for Bangladesh’s increasingly capital intensive formal sector. As of 2018, over 60,000 youth, business owners and migrants have been reached, with the goal of reaching half a million over the next few years. To achieve the goal, SDP needs to analyse their operations and results in a smart way to encourage continuous improvement of the programme. BRAC had built other MIS systems so the SDP team took those into consideration in building their solution.

”Management wasn’t using those systems to really drive their decisions,” said Eshrat Waris, manager of the skills development programme, during a recent TaroWorks webinar. “After studying, (it) became clear those systems were being built for donor reporting…(but) it’s not where the data flows from. The data flows from the field.”

Paper files BRAC used previously to store and manage field data.

Hoping to make it easier to aggregate, analyse and act on large amounts of collected field data, Waris and her colleagues had also previously tried several data collection, analysis and decision-making approaches, including spreadsheets with macros. They discovered, however, that different versions of Excel in use by BRAC across the country created compatibility and performance issues.

The SDP team eventually turned to TaroWorks’ mobile field services management app and its offline Salesforce.com CRM to power a pilot project for 15 branches. The goal was to see if the use of digital tools could make data management more efficient and inform daily decisions to foster continuous improvement in operations.

Embedding the pilot into operations

The new TaroWorks/Salesforce.com system was piloted for SDP’s Skills Training for Advancing Resources (STAR). STAR trains youth, especially women and people with disabilities, through on-the-job apprenticeships. The apprentices or learners are placed under a master craftsperson, who is an experienced shop owner or worker within a trade including tailoring, mobile phone servicing, refrigeration, aluminum fabrication and beauticians. Fifteen branches were selected for the pilot, which included 1,741 learners and 721 master craftsmen.

One of BRAC’s biggest challenges in using mobile technology and a cloud database to foster continuous improvement of apprenticeship programme field operations was how these tools would meet the needs of the far-flung team of programme organisers.

“It is a field-heavy operation and data was absolutely critical for understanding what was happening on the ground, to make decisions, and to understand are we targeting the right people are we targeting the right trades. We needed a real-time field operations management system and for us TaroWorks served that purpose,” said Waris during the webinar.

An apprentice from STAR trained in pharmacy.

When Waris and her colleagues designed the system pilot, they wanted to make sure that it would be successfully adopted. These were some of the design principles they followed:

  • Design for the end-user: Because the priority was never to serve a donor, they focused on finding a tool and designing forms with the field end-user’s skills in mind. After all, if data lies in the field, they needed to be designed for the field.
  • Be rapid, be scrappy: They wanted to start piloting at the very beginning of a new apprentice cohort, making timing very tight. They recognised that you don’t need the perfect product to deploy, but rather a minimum viable product. Then, as you get feedback from users, continue to improve.
  • Be embedded with operations: Get to know the realities on the ground and make sure the technology complements day-to-day operations.

The system was soon on its way to being adopted and encouraging continuous improvement of operations. As a result, BRAC scaled the system from 15 to 141 branches overnight.

Signs of continuous improvement

TaroWorks was used to create and administer mobile data collection surveys, digitise key elements of daily field management and identify trends or roadblocks through data visualisation dashboards within Salesforce. As a result, BRAC was able to show continuous improvement in a number of areas including:

  • Optimising learner placement: BRAC’s skills development programme has a mandate to recruit learners with disabilities for apprenticeships, but the challenge was finding the right master craftsperson to train participants with special needs. Using its digitally collected data on learners and business mentors, BRAC ran an artificial neural network analysis to identify the type of trainer who would work best with this segment of learners. It turned out that business owners with eight or more years of business experience, with a particular education level and had trained at least seven prior apprentices, were an ideal match for learners with disabilities.
  • Gauging learner progress: Programme organisers needed to assess the progress of learners they brought into the apprenticeship programme but the organisers didn’t have a trainer’s expertise with which to gauge learner performance. BRAC created guidance in its mobile app which characterised, for example, the correct way to cut cloth for tailoring apprentices. Programme organisers then watched the apprentice at work and gave them a grade in the app based on their work. This methodology gave BRAC insight into which learners might be falling behind in their training and would benefit from additional support.
  • Assessing field staff performance: Comparing 2017 to 2018 data, BRAC spotted a significant drop in the amount of information being edited by field managers using a mobile survey to help select learners for the apprenticeship programme. Upon further investigation, BRAC discovered that managers were not doing enough independent validation of prior work done by the programme organisers to evaluate and recommend candidates. BRAC asked managers to provide a digital signature attesting that they’d done the validation, which in turn increased recorded editing time.  As a check against those results, BRAC tweaked the survey further by using the term “approval” rather than “selection” when asking managers to sign off on the learner picked by their subordinate. As a result, survey editing time decreased but was still higher than first measured, indicating managers were giving more attention on validation, thereby advancing continuous improvement efforts.

TaroWorks being used on a smartphone at a learner’s house.

Continuous improvement: Lessons learned

With the mobile technology pilot completed, BRAC learned important lessons, which helped in expanding TaroWorks and Salesforce use across all of its field operations in Bangladesh.

  • Design for the field not for the donor. Rather than aiming for perfection, versions of BRAC’s TaroWorks data collection forms and other features or functionality were rolled out as if they were minimum viable products, allowing the organisation to win the trust of field staff by making changes according to continuous feedback from users.
  • Be embedded with field operations. There is the operation model and then there is reality. For example, field surveys were thought to be done sequentially as outlined in the operations manual, but were actually conducted simultaneously in real-world conditions, an insight gained from observing work in the field.
  • Technology is the easy part. It didn’t take long to learn how to build a data collection and analysis system, said Waris. The hardest part of the process was what she called “behavioural change” or getting people to accept and trust any system, which shifted their efforts from a paper-based approach to a digital platform.
  • Be patient, behavioural change takes time.  Regional managers who had been working for BRAC for more than 25 years had not touched a smartphone or a laptop. Now, with training, they are building out customised data reports in Salesforce, having developed a sense of ownership and independence.

BRAC team members during a TaroWorks training session.

 

The article originally appeared on the TaroWorks blog.

Elaine Chang is director, market development and customer success at TaroWorks. 

Comments are closed.