Collecting user feedback for a driver app means systematically involving your own drivers as the main users. Monthly surveys, structured interviews in three phases and so-called test drives serve this purpose: developers drive the app themselves to experience what remains invisible at the desk. The results flow directly into the prioritization and roadmap.
Key Takeaways
- MOIA operates a fully integrated ridesharing service: own vehicles, own apps, own drivers and self-developed vehicles in cooperation with VW.
- The driver navigation app is only available internally and is only visible in the Play Store with a MOIA email address, which makes it a pure business application.
- Feedback from drivers is collected in a three-stage process: monthly surveys, in-depth interviews with mockups and a final usage test after the release.
- Test drives every two weeks in a dedicated test environment ensure that developers experience the app for themselves behind the wheel and not just simulate it at their desks.
- New releases are rolled out slowly: first 20 percent of vehicles, then 50 percent, then 100, with a two-day observation period in between.
When your own colleagues are the users
At MOIA, the tester target group for one of the apps is congruent with the company’s own employees: the drivers. The Vehicle Guidance Application, the navigation system for drivers, only runs on a specific device and is not even visible in the Play Store if no MOIA e-mail address is available. It is a pure business application that nobody outside the company could do anything with.
This results in a rare starting point for quality work. Normal app customers leave if they don’t like something and don’t give any feedback. The drivers can’t do that. They have to work with the app, every day, every shift.
That’s exactly why they want to give feedback. Ada Pohl, Senior Quality Specialist at MOIA, describes this desire as a stroke of luck: “Anyone who is forced to use a tool has a genuine interest in making sure it works. This interest can be systematically tapped into.
MOIA is testing a full-stack ridesharing service
MOIA is a ridesharing service with pooled trips, its own vehicles and a complete app landscape. The customer books in the app, sees the price and approximate journey time, and an algorithm searches for a suitable vehicle. Pools are made with other customers on similar routes.
The company builds everything itself and therefore calls itself a full-stack ridesharing service. The customer’s app, the navigation system for the driver, another screen for the customers behind the driver, plus several other apps. Even the vehicles were developed with VW, from the interior to the seats. The drivers are employed.
There is a separate Quality Chapter for quality, spread across several Streamline teams. Each team is responsible for one area. The driver app tests whether the driver can interact properly with the trip: Customer picked up, customer dropped off, customer did not show up.
An important framework comes from outside. MOIA is not a cab company; the license does not allow door-to-door trips. Customers are picked up at digital stops specified by the city, within a walking radius of 250 meters. Such regulatory limits are not a detail; they help determine the behavior of the software.
Obtaining feedback is not an annual ritual, but an ongoing process
MOIA works with several feedback channels in parallel instead of sending out a survey once a year. A driver survey is sent out once a month via a newsletter with a link. The questions alternate between direct and open.
Sometimes they ask specifically about a feature: Does it still work for you? Sometimes deliberately more open: What bothers you most at work right now? What are you missing? Do you need more or less information? These surveys reveal trends as to what is user-friendly, what is missing and what ideas the drivers themselves have.
The trends are turned into a feature in several stages. First we gather all the knowledge we can, then we build mock-ups, go back to individual drivers and ask them. Then build or release a first working feature and test it again: Does it fulfill its purpose after some use, does it make life easier?
The strength of this approach lies in the pace. If you involve users early and repeatedly, you build features based on evidence rather than assumptions. This shortens the path from idea to usable solution.
Test drives and test rides take testing from the desk to the car
Testers get behind the wheel themselves every two weeks before a release. During a test drive, the team takes a vehicle and plays driver, while colleagues book trips remotely or from the back seat. The reason is concrete: operating a navigation system at your desk with a simulated blue dot is completely different from sitting behind the wheel.
The mindset of the developers also shifts. Implementing something at your desk or feeling for yourself how the app feels while driving leads to different decisions.
The second mode is the test ride. Here, a real MOIA is booked, the team sits in the back and observes the driver. If there are no customers, questions are asked. Ada compares the role to that of a bartender: simply accept whatever complaints and pain points the drivers bring with them. Since the pandemic, this channel has been lost somewhat due to a lot of remote work.
Important: Both formats run in their own test environment, never in production. Passenger transportation requires a P license, and the team deliberately wants to keep the real customer relationship outside. Testers are not trained drivers who pay attention to traffic and customers at the same time. The added value of the method is lost if the testers think about customer communication instead of the quality of the software.
Contradictory feedback forces a decision
Drivers want contradictory things, and that is normal. One example is the density of information on the map. In the past, each trip was displayed individually, with lots of information. Today, the app only shows each stop; another screen with the individual customers can be opened at the stop.
The reactions vary. Some drivers are happy to finally be able to concentrate on the essentials. Others are desperate for more information even before they get to the stop.
At this point, user feedback alone is not enough. The team has to decide which group it wants to make happy, based on safety and concessions. MOIA is not allowed to stop everywhere. If someone is standing at a legal stop, a rule is needed as to where the vehicle should go instead. Such cases must be included in the app, regardless of what individual drivers want.
Feedback provides the arguments for the roadmap
The biggest challenge is not collecting, but sorting. Together with the product owner, driver feedback must be incorporated into the roadmap in such a way that it matches the stakeholder wishes, for example from VW or higher up.
This is where the collected feedback becomes a lever. If many drivers name a feature as necessary, this provides a reliable argument for prioritizing it. Business stakeholders and drivers are ultimately pursuing the same goal: more and better trips and a good experience for customers, as this leads to follow-up bookings.
This helps a lot, and it provides exactly the arguments you need to overcome the challenges.
Ada Pohl
Why classic A/B testing is difficult here
A/B testing at driver level hardly works with MOIA because the app is rolled out at vehicle level, not at driver level. For data protection reasons, the navigation app does not know which driver is currently using it. This means that a stable beta group of individual drivers cannot be formed.
Instead, MOIA relies on slow rollouts. The start is 20 percent, randomly distributed among the vehicles. If nothing negative comes back after two days, it goes up to 50 percent, and after another two days to 100 percent. This works because all vehicles are equipped in the same way and the tablets remain permanently in the vehicle.
Random distribution has its pitfalls. The 20 percent can also affect a vehicle that is currently inactive in the workshop and not on the road at all.
For a real beta setup, only the effort remains. For really critical features, the team removes around 20 vehicles, updates them manually and only allows certain drivers to drive them. Due to the amount of work involved, this is only worthwhile for features that are critical in some way.
The test environment has its own rules
A dedicated test environment also produces friction that is unknown in real operation. If there are too many simulated vehicles in the same place in the service area, the test drive will not get the vehicle it needs for its trip.
The solution is pragmatic and very human. A request is then sent out via Slack to move your own vehicles away from the area or simply switch off pooling. Without pooling, no trips arrive and the test becomes easier again. Tricks like this are part of everyday life.
Where the focus is shifting: AD-Readiness
MOIA currently sees little additional need for the classic system because the process is well developed. The focus is shifting towards autonomous driving.
A test is planned in which a vehicle from VW and Mobileye will drive autonomously through the city. MOIA will then develop the connection between passenger and vehicle. Internally, this is known as AD-Readiness. A lot of the current thinking is going into the question of how this can be implemented.
