Quality Documentation for User Satisfaction: Learning the Hard Way

This blog is an initiation to a series titled ‘Galtiyon se Seekh’ (or Learning from Mistakes). It is an attempt to share our errors and, more importantly, convey how we corrected our course in its retrospect. Gramhal recognises that innovation includes the possibility of failure. The organisation strives to ensure that team members feel empowered to think and create big and, if along the way they happen to make mistakes, then learn from them. The intention is to prioritise action that leads to insight over inaction that is rooted in fear. Also, to be mindful of accountability, reflection, and growth.

In the aforementioned spirit of introspection, this blog will document our failure in creating quality user documentation that affected our product’s usability and subsequent user satisfaction.

Bolbhav, our Whatsapp-based chatbot offers agri-information to farmers in a manner that is accessible and contextual. It offers daily market price information for various crops, price trends, weather and crop advisory, crop quality knowledge and information on government schemes for farmers. All these options are listed in a main menu that is shared when a user messages ‘Hi’ or shared on default (ie when a user types a query that is not recognised by the bot). For some time, we have known that most users only used Bolbhav to check daily mandi and crop prices. Here is a concerning data snippet from December 2022 to back this- there were 80,936 unique events for market price as compared to only 2,545 for the market price trends feature.

The lack of use of other features has often been attributed to multiple reasons. This includes design challenges owing to Whatsapp’s interface or a possible lack of discoverability in-built in the bot design. Discussions also pointed to user behaviour where rural farmers were assumed to less inquisitive about the product. Or we noted their apprehension towards opening a list of menu with all features. The limiting nature of the name ‘Bolbhav’, which roughly translates to ‘say the price’, was also suspected.

The team tried several solutions to solve for this. A suggestive loop was created that nudges users to try other features when they looked at the commonly used ones. Notifications were sent when new information was updated to engage users. However, these fixes saw limited success.

Meanwhile, there was a glaring, simple solution to this problem that had been recognised earlier but never prioritised. Insights from user research calls (undertaken in response to this problem of low usage of other features), substantially supported this particular solution and highlighted its urgency. I will get to this solution in just a bit. First, let’s note what we learned from the user calls.

The goal of these user research calls was to understand whether the other features were accessible and easy to understand and how we can make feature-wise improvements. Here, farmers were contacted to understand their experience using the features, what has and hasn’t worked, and note their suggestions. Following are some insights gathered:

• Price trend feature: Users can check prices for the last 30 days for a crop at a particular mandi by entering the respective numeric codes assigned to the crop and mandi together. Almost none of the survey participants knew how to use the price trend feature without getting nudge messages for it. Equally significant, was that almost all recognised the importance and need for the feature.

• Weather and crop advisory: Bolbhav provides simplified, contextual weather information and crop advisory sourced from the IMD (Indian Meteorological Department). The survey calls yielded similar insights for this feature too. 5 out of 6 users didn’t know how to look for weather information on the bot unless they received a subscription message. Again, all agreed that the information provided was easy to understand and very helpful to them.

On probing further, the user research team realised that these users knew how to check mandi and crop prices from the introductory video that is shared when using the bot for the first time. A video that hadn’t been updated for a long time. It only showcased how to use the chatbot to look for mandi and crop prices; features that we began with. We failed to include updates about the newer features, how they could be beneficial and how to use them. A minor update, major oversight.

The team focused on improving the product but failed to improve documentation that would help users be more successful with the product. Smaller interventions such as nudges and subscriptions mentioned before helped drive up engagement for other features but failed to make users self-sufficient. They were unable to find and use them when they wanted to. Since then, we have also learned that no matter how intuitive we think the product is, accompanying user documentation is a must. It eliminates assumptions around easy of use, ensures user satisfaction and guarantees better chances of retention and referral. This is especially important to remember since we serve the rural population who don’t interact and explore products with the same ease that urban populations would.

To rectify, the team at Gramhal has identified the following ways to improve user documentation:

• Update the introductory videos for new users, so they are aware of all features and use cases from the get go. This involves setting the right expectations for what Bolbhav is and that it has more to offer than just market prices.

• Create a help center on the chatbot that users can access whenever needed. This would include detailed descriptions on a feature, why it is important, and how to best use it.

• Send relevant nudges that educate users on other features and how to access them independently.

• Humanise the user documentation for the rural audience. For instance, include a person in the explanatory videos instead of simple screen recordings.

We are confident that these changes would vastly improve user experience. We strongly believe our product is useful. We want our users to know it too.