Tag: WordPress

Sharing User Feedback from App Reviews

Rachel1 Comment

Over the past year, I’ve been working fairly closely with the mobile app team at Automattic. As I got more involved, I tried to help close the feedback loop with the team by taking advantage of the feedback our users were already giving us — so of course I took a look at our app reviews.

It’s hard to look through app reviews. I mean, on one hand, it’s just emotionally draining to be hit with that barrage of unmediated criticism (although the unmediated praise is wonderful!). But it’s also hard to grok all that feedback when it’s just a stream of comments. So I decided to collect that feedback and present it to the team in an easier-to-digest format. I’ve now gone through that process several times and want to share it in case it helps you process reviews or other feedback from your customers.

Collect and Organize the Feedback

The first step is to gather up all the feedback. I used App Annie, since our mobile app team was already using it. I decided to identify all the reviews from the latest version of our app (in my case, it was the WordPress app on two platforms, iOS and Android) and export them. This conveniently dumped all of the ratings, reviews, and user details into CSV files (one per platform).

Then, I set up a spreadsheet for each platform and focused on a few key details:

  • The user’s rating (from 1 to 5)
  • The review’s title and content (adding a translation where the review was in another language)
  • The main issue in the review
  • Any secondary issues or notes about the review

How did I identify the main and secondary issues? A little analysis.

Analyze the Feedback

To find the main and secondary issues, I read every single review from that version of the app. I picked keywords to describe the main issues users described and assigned one of these keywords (categories) to each review.

If you’ve ever coded survey responses, this is a similar process. If this is the first time you’ve done this, here are some tips:

  • Read through all or a representative sample of the reviews. (For your first time, and especially for an unfamiliar product, you might need to read all of them.)
  • As you read, make notes about the topics or keywords that come up (more is better at this stage).
  • Pare down your list to a subset of more general keywords. For example, for the WordPress app I used keywords like “Editor,” “Login,” and “Media upload.”
  • Go through the reviews one by one and assign a keyword for the main issue the user described.
  • If the user mentioned more than one issue, or there is additional detail that you think will be helpful later on, add it in the field for secondary issues or notes. For example, I found a number of reviews with the “Editor” keyword that specifically mentioned “limited features” in the editor, so that went into the second field so I could keep track of that sub-issue.

Pro tip: To keep my sanity, I worked from 1-star reviews to 5-star reviews, so the toughest criticism came when I had the most energy and the work got easier and more cheerful as I went.

Once I was done assigning keywords to each review, I organized the spreadsheet by those keywords so I could see which issues were most commonly reported. I made adjustments to the keywords, looked for subsets of related issues, and checked everything for consistency. Finally, I got ready to share my findings.

Share the Feedback

I had a few self-imposed guidelines for what I wanted the team to get from this user feedback:

  • Praise for the things we are doing well
  • A clear picture of the top pain points our users experience
  • Suggestions for what action could have the biggest impact

Here’s a template showing how I organized my report:

Overview:
- Number of reviews
- Average rating
- How ratings are weighted (evenly spread? split between 1 and 5 stars?)

Highlights:
- Features or experiences that our users enjoy and appreciate
- 2-3 quotes from positive reviews

What did users mention in their reviews?
- The top three issues mentioned in reviews
- For each issue, an explanation of its impact (how many or what percentage of reviews mentioned it? what were the star ratings for those issues?) and a little context about what exactly users discussed and your interpretation of the source of the problem
- Links to any open bug or enhancement issues the team is already tracking, or any ongoing work related to the issue

Suggestions for followup:
- One or two projects, or open issues in the bug tracker, that the team could make a top priority to help address this feedback
- Any other user feedback (for example, from customer support interactions) that could shed additional light on the feedback in the reviews

I shared this with our entire mobile app team (along with the spreadsheets with the raw data), inviting questions and discussion. Although we haven’t taken action on every single issue, it has led to some quick wins, reprioritizing, and planning ahead with our users in mind.

I hope this is useful to you and your team! If you try it out, let me know how it goes. And if you have ideas for how to improve this process, I’d love to learn from you.

Advertisement

Embracing Openness

Rachel

An exciting part of working at Automattic is the open source philosophy. It’s thrilling to help open source projects like Calypso (the new WordPress.com), even in a small way. I spent a chunk of time last year reviewing open bug reports, enhancement requests, and the like before the project was open sourced — it’s fun to see (and help with) new issues that come up as more people work with, get inspired by, and contribute to Calypso.

Open source isn’t just about developers, either — I do a lot of my day-to-day work in the open. I used to mainly provide support out in the open, helping users in the WordPress.com forums. These days I spend most of my time in the WordPress app repos, reporting and testing issues in the open there. (If you’d like to get involved, I also started posting calls for testing the WordPress apps over at Make WordPress Mobile.)

In the spirit of open testing, Automattic also recently released the WordPress.com Automated end-to-end tests into the open. I’m so glad we can share the awesome work that our QA and testing folks are doing. I’d also highly recommend checking out my coworker Alister’s blog WatirMelon for more testing talk.

Openness can really be part of everyone’s work — from open source, to open testing, and open support. Being open helps us learn from each other, inspire each other, and keep us aware of and oriented toward the wider community in which we work.

Using WordPress. On Android. On a Mac.

Rachel

I recently read an article on The Verge about how you can run Android apps on a Mac (or PC) using Chrome. That was all the invitation I needed to try it out. So off I went to find the three things I needed:
1. An APK
2. A PC, Mac, Linux, or Chromebook on Chrome Version 41+.
3. The ARC Welder app

I have a Mac, and you can download the ARC Welder app from the Chrome store, so all that was left was the APK. I’ve never owned an Android device, so I wasn’t sure what an APK was, but I assumed it was some kind of file type for Android apps. (It turns out APK stands for Android application package.) The article I read said you can get APKs from the Google Play Store, but I didn’t have any luck finding them there. Luckily, I had another idea.

I was most interested in testing (maybe you guessed already) the WordPress app. Since it’s an open-source app, I headed to the WordPress Android app repo on GitHub. There’s a release page there where you can download the APKs for all of the previous releases. Bingo!

After installing ARC Welder and adding the WordPress APK, the app fired up and I was ready to go. Easy peasy. The biggest challenge now is figuring out how to interact with a touch app on my laptop. For example, I have to tap twice to paste, rather than using a keyboard shortcut. But it’s really fun to explore the app this way, especially as it’s my first time interacting with the WordPress Android app (which is a bit different from the iOS app that I use on a daily basis).

And here I am, composing my first blog post on the app. Pretty neat. 🙂

English only? I hope not!

Rachel

It’s no secret that English dominates the tech sector. From communicating with coworkers to launching software, English usually comes first. Even the most well-meaning companies can struggle to reach non-English-speaking users and provide a localized experience in their native language. And no surprise there, either — it’s really tough to do well.

As a small part of my job, I do what I can to help make WordPress.com better for users around the world. (And I’m certainly not alone. There are a lot of rockstar translators contributing to WordPress.com, along with a well-established group of polyglots working on the open-source WordPress project.)

I’m not a developer, so much of what I do is connecting people with resources. I help volunteer translators get oriented so they can help translate WordPress.com into their own language. I teach our internationalization team about the tools and methods used by professional translators. I find or report bugs that cause translation issues so the code can be improved. And I help educate all of my colleagues about how translation works. For a lot of people, just thinking about using WordPress.com in another language is … well, entirely foreign to them.

But every time someone stops to think about how a product works in another language, it makes a difference for users around the world. That’s why I was so impressed when I saw that Mark Zuckerberg held a Q&A session entirely in Chinese:

Sure, it wasn’t flawless Chinese, and I wouldn’t bet on Facebook’s developers switching from English to Chinese any time soon — but this Q&A session is a gesture. It’s recognition of a user base outside of the English-speaking world, a clear message from the CEO of a major tech company that he cares about speakers of other languages.

As a linguaphile I find it incredibly heartening to see what Zuckerberg did in this session, although I don’t expect every CEO to start speaking other languages. It takes a lot of time and dedication to learn a new language, and Zuckerberg has personal motivation — his wife’s family speaks Chinese. (I understand that motivation!)

What matters are the resources, attention, and energy that are invested in making the web a better place — for everyone.