ModsAzine.com "The Un Magazine"

Best Practices for Building and Maintaining Your Own Research-Participant Database


Conducting user research is central to every UX practice. Whether you’re doing discovery research or conducting usability tests, recruiting the right participants for research studies is a critical part of the process.

There are several common options for recruiting research participants. In the table below, we outlined the important factors to consider when selecting a recruiting method for your studies.

Recruiting Method

Reach

Cost

Effort

Time

Bias Risk

Professional recruiter

Wide

General users

Specialized users

High

Low

Med

Low

Automated recruiting platforms

Wide

General users

Med

Low-Med

Low

Low

Online forums and groups

Narrow

Specialized users

Low

High

Med

Med-High

Intercept studies

Narrow

Visitors (new/existing)

Task-oriented users

Low

High

Med

Med

Internal user panels

Narrow

Existing users

Power users

Employees

Low

Med

Low

Med-High

Recruiting Method

Cost

Effort

Time

Bias

Risk

Professional recruiter

High

Low

Med

Low

Automated platforms

Med

Low-Med

Low

Low

Online forums and groups

Low

High

Med

Med-High

Intercept studies

Low

High

Med

Med

Internal user panels

Low

Med

Low

Med-High

In an effort to operationalize research efforts and create a ResearchOps practice, mature UX teams have begun to develop their own internal user-research panels (the last recruiting method in the list above).    

We spoke with UX researchers who have developed such customer-research panels. In this article we discuss considerations for creating a research panel, along with tips, tricks, and lessons learned — all shared by these researchers.

Why Create Your Own Research-Participant Panel

It can be laborious and time intensive to find and recruit users through traditional channels. The UX researchers we interviewed found it inefficient to work with teams such as marketing or customer success to recruit study participants. Even though these teams might already have direct access to customers, relying on them for all recruiting needs isn’t efficient or scalable.

Mature UX teams who are conducting frequent research often find it useful to develop their own pool of customers who are willing to be contacted for participation in research efforts. An internal participant panel allows researchers to be nimble with the research they do, spinning up small studies when they need quick insights. A dedicated UX-research panel and systematized approach to recruiting also allows researchers to be efficient in recruiting across all research studies.

Once the research panel is established, it allows for lower ongoing recruiting costs (both monetary and time-related): candidates are already semi qualified and there are no external recruiting fees. You would, however, most often need to provide participants with some compensation — monetary or of some other type, such as in the form of products — for their time. 

Drawbacks of an Internal Research Panel

Although an internal research panel is a great source for efficient recruiting for most studies, there are a few potential drawbacks to be aware of.

  • Limited reach. It will be easy to recruit users who are already familiar with your brand and offerings, but with such a recruiting pool you may have difficulty capturing new-user perspectives.
  • Bias. There might be some sampling and confirmation bias, due to participants’ existing brand loyalty — since they already have a relationship with the brand, they may provide primarily positive feedback.

Creating a Research-Participant Panel

It’s important to acknowledge the cost of creating and maintaining a user research panel. The researchers we spoke with shared some of the upfront work and considerations for establishing a usable participant panel.

Choose a Tool for Your Panel

Researchers will need to spend time exploring the offerings of paid tools and considering the functionality of the software tools they already had access to.

There are various options for storing and maintaining your participant information. The researchers we spoke with used a variety of solutions. Each comes with its own benefits and drawbacks.

Enterprise Software

Several researchers we spoke with took advantage of some of the more ubiquitous enterprise software solutions already available to them. Common enterprise tools that researchers leveraged included Microsoft Excel, Microsoft Lists, Google Sheets, Confluence, and SharePoint. These tools were often the go-to for researchers just starting to build their research panel.

Jon Clancy, Senior UX Researcher at Cox Automotive, spoke about the benefits of using Excel for his organization’s panel, “It’s in an Excel spreadsheet. We have columns for first name and contact info that feeds into a mail merge really well, and I set up a filter function so we could sort by product or search based on the qualitative feedback we have received in the past, which yields a semi curated list of volunteers that we know are somewhat interested in a given topic. With Excel, the benefit is that it’s simple. One sheet holds the full list, and the filter and query functionality are also a big benefit.”

Researchers liked using enterprise tools because they are simple and familiar. However, these tools have some drawbacks compared to the more specialized tools. In particular, researchers have to do a lot of manual work to communicate with participants, using email, text or telephone. They must also manually update the spreadsheet with information about study participation. Tracking when users were last contacted, when they last participated in a session, how many times they’ve participated, and how articulate or engaged they were are all critical pieces of information needed to support future recruiting efforts and to ensure that users in the panel are not being contacted too often.

Customer Record-Management (CRM) Software

Several researchers mentioned the utility of CRM software tools like Salesforce or HubSpot for managing research contacts. These tools are built to store customer data, giving researchers a head start in establishing the data structure and defining useful metadata needed to segment and target research participants. That being said, these tools come with the same downsides as the enterprise tools: the management of these participants throughout the research process must be handled elsewhere.

James Tucker, Director of User Experience at AutoFi said, “Since recruiting UX-research participants is like a sales pipeline, it makes sense to use sales tools, such as customer-relationship managers (CRMs). They allow researchers not only to track the contact information for the participants, where they are in the process, and various communications, but also to add notes and file attachments.”

Hubspot research participant administration
James Tucker used HubSpot to manage where his participants were in the research process. Source: https://www.toptal.com/designers/ux-research/ux-research-participants

Specialized Research and Recruiting Tools

Over the years, more and more specialized UX-research tools have become available to support and streamline research. There is a mix of useful functionalities among the tools in this category. Some have robust data-analysis and insight-tracking capabilities, while others are focused on facilitating recruitment and study management. Most of these tools are continuously evolving their offerings, so it’s not uncommon for them to have a subset of features to support UX-recruiting processes.

Most of the tools that enable researchers to develop their own research panel also allow researchers to facilitate and manage studies within the tool. Of course, most studies can be also done outside the tool. Surveys can be sent via email, interviews can be scheduled via email and conducted via video-conferencing software.

However, the researchers we spoke with lauded tools that supported the full workflow as gamechangers for research efficiency. Not only can they use these tools to store and manage a database of research participants, but they can also run and manage the actual studies using built-in communication and tracking features specifically designed to streamline study management, that remove the overhead involved in patching together your own process and toolset.

The researchers we spoke with shared with us which specialized tools they were using. They mentioned platforms such as Great Question, UX Tweak, UserInterviews Hub, and Dovetail. There are, no doubt, more tools in this category, and their number will continue to grow.

Of course, along with specialized functionality comes higher costs. This is the biggest drawback of specialized research tools. Large organizations with big budgets can justify the cost of these tools more easily than small companies or those with low UX maturity.

Be Thoughtful About What User Data You Need

Think about the types of studies you have conducted in the past and ask yourself what information was necessary in your recruit. Aim to keep your data as simple as possible, because this information will need to be managed and kept up to date over time, but be sure to capture enough to avoid the need for lengthy screeners asking the same questions over and over.

Beyond basic identifying and demographic information, think about your business and decide what other details would be relevant. For example:

  • Which products or services they use
  • How long they’ve been customers
  • For business-to-business engagements, the industry they do business in
  • Time zone (for scheduling purposes)

Consider What You’ll Want to Track over Time

Building a panel is meant to help you recruit for many studies over time. For that reason, it’s best if you keep track of your engagements with each person in the panel. (Some of the specialized tools have tracking capabilities built in, but others will require you to track this information explicitly.) 

Useful engagement information to inform future recruiting includes:

  • Last invited date
  • Last participated date
  • Which studies they’ve previously participated in
  • Notes about how articulate or engaged they were in prior studies

Tracking prior engagements allows you to avoid reusing participants who:

  • Have already participated in a test with the system of interest (if you want to study ease of learning or first exposure to a design)
  • Have recently been in a study or have participated too many times in a given period
  • Did not show in previous studies, were less articulate than you needed, or seemed in other ways inappropriate for your research needs

Establish Ethical Data-Collection and Secure Storage Policies

Since user information will be stored for recruiting purposes, it’s important to follow ethical research practices to ensure that data is stored securely and abides by any local government regulations that your business is subject to. Create a security policy for the database that lists the rules for using it and for protecting participants’ privacy. To this aim, work closely with your legal or compliance teams. Ask participants for consent to store data and communicate how the data will be stored.

Maria Shendyapina, an EU-based UX Researcher for EcoOnline Global, worked very closely with her company’s legal and compliance groups when it came to selecting a tool to house the panel. It was imperative that the tool she selected follow GDPR guidelines. She also consulted with her contacts in the GDPR-compliance department to establish a data-storage policy for the panel; this policy was communicated to those who opted in to participate.

Decide How You Will Populate Your Panel

Think about the channels that your customers use to engage with you. Are there opportunities to recruit panel participants via these channels? Below is a list of strategies that practitioners shared with us:

  • Append a request to opt in at the end of existing feedback surveys.
  • Enable customer support to recruit customers after support interactions.
  • Get a list of customers from sales or marketing and send out a request for participation.
  • Set up a booth at in-person events or conferences.
  • Post a call to action on your website, on the page that allows existing customers to provide feedback.
  • Post a request on social media.
  • Ask for referrals from existing panel members.

Assign an Owner and Communicate the Existence of the Panel

After the database is built and populated with research contacts, most of the upfront work is complete. However, you will still need to do basic maintenance on the database over time. For this reason, it’s important that one team or person is assigned ownership over the panel. The owner will be responsible for ensuring that the panel remains securely stored, that the information is used purposefully and ethically, and that the data is audited over time.

This person will also be the point of contact for any questions or concerns that come up regarding the panel.

The existence of the research-participant panel should be clearly communicated to other parts of the organization. Questions about the panel may come in from the customers who receive communications via the panel, so it’s critical that the customer-service and support staff be made aware of the panel and how it’s being used (see below).

Let Participants Know What to Expect

UX researcher Maria Shendyapina mentioned she had received inquiries from customer- support staff about the legitimacy of communications sent to customers from her research-panel-management software. Because she was using a third-party testing tool that also housed her database, the domain name listed in recruiting emails was not her company’s. Although each customer had opted in to be part of the panel, over time some forgot they had done so. In Maria’s case, a cautious customer called the company to verify the validity of one of her recruiting messages.

Prime users for what to expect when signing up for the panel. If your technical setup allows it, send a welcome email that explains where requests will come from and how they will appear. If at any point you choose to migrate your panel from one storage method to another, communicate to users what changes they may notice.

On a related note, Maria also wondered whether all her panel members had received her research requests or whether some of these had been routed to the recipients’ spam folder. For this reason, ask users to whitelist any third-party sending domain that you may use.

Establish Governance and Training Processes

Most researchers I spoke with indicated there was a need for at least some governance and training for other researchers who would be using the panel. Governance was typically introduced to ensure that panel participants were not contacted with research requests too often.

Panels that were stored using specialized tools required less manual governance than those managed using an enterprise tool. Many specialized tools allowed administrators to put rules into place to control when users could be contacted and for how many times during a given period. 

Panels managed in spreadsheets required manual governance. Sometimes, clear rulesets and trainings were used to ensure that researchers did not abuse the tool. In other instances, panel owners were involved in selecting the recruit. A sound governance practice is, however, based on diligently tracking how often a participant has been contacted or participated in a study. 

Another consideration for database owners is who has access to the panel. One researcher from a SaaS company, shared that the panel she and her colleagues created was to be used by the UX team exclusively. Her team wanted to ensure that participants who agreed to participate in their research would receive the same experience every time they were invited to a study, and that experience would follow ethical research practices, including always asking for consent. She feared that, if the panel could be accessed by employees outside of her team,  participants would be flooded with requests that did not follow the policies established by the UX team.

Using and Maintaining Your Research-Participant Panel

Regardless of which tool you choose, there are some important things to keep in mind when building and maintaining your own research panel.

Allow Participants to Opt Out of the Panel

Of course, it’s critical that participants who once agreed to be part of the panel can ask to be removed from it. Specialized tools often allow users to opt out of the panel whenever they receive a communication. However, if you are not using a specialized tool, you must let users know how they can opt out from the panel and follow through with such requests.

Audit Your Participant Panel Regularly

The researchers we spoke with talked about the importance of keeping the database fresh and actionable. For this reason, many of them audited their research panels annually or every other year. Ask your legal or compliance teams what is appropriate for regulatory reasons. Auditing the database included:

  • Removing participants who are no longer available or interested
  • Removing participants who had been contacted several times but did not participate
  • Asking users to validate or update their contact information and to confirm their interest in participation

Conclusion

While developing your own research-participant database requires an initial investment, it can save time and resources in the long run. An efficient operationalized system for finding research participants will allow researchers to run more studies with faster turnaround.

Remember, your panel may be skewed toward loyal customers and fans, so its use could introduce bias. Consider whether it’s okay to limit your research to this type of sample or if you should include other methods of recruiting that could bring in other perspectives.

Sign Up for the NN/g UX-Practitioner Research Panel

Thank you to the researchers who shared their experience with us for this article. These researchers were contacted via our own NN/g UX-Practitioner Research Panel, which is hosted via UserInterviews Hub.

As UX practitioners, we usually don’t have the opportunity to participate in research studies.  NN/g is always conducting research with UX practitioners to inform our work. If you are interested in participating in research with us in the future, please sign up here.

To learn more about streamlining your research efforts, take our course, ResearchOps: Operationalizing and Socializing User Research.



Source link