Get Smart About Research

ASSOCIATION EXECUTIVES USE research all the time.

We run surveys and focus groups to better understand our members’ most important goals and most pressing problems so we can make good decisions about the programs, products, and services we offer to meet those needs. We sponsor industry studies to help our members and other audiences make good decisions about investing their resources wisely. We consult third-party (university, government, NGO) data sets and studies to understand the STEEP (social, technological, environmental, economic, and political) trends that influence the operating environments of the professions and industries we serve.

The problem is that there’s a lot of questionable research out there.

While good research does not guarantee good decisions, it certainly helps. And bad research, barring getting lucky and guessing right, almost inevitably leads to bad decisions.

Good Research is Essential to Association Success

No matter how tired we may be of hearing about “Big Data,” “data, data, everywhere,” and “data-driven decision making,” the truth is our efforts to capture reliable and honest insight about (and for) those we serve has always been a key element to our success as associations.

The reason research exists, the reason research projects are undertaken in the first place, is to help explain the world. People have questions about what’s happening and why, and research endeavors to answer those questions, on the way toward advancing knowledge.

Similarly, association executives turn to research to make sense of our own domains – that is, to understand the current state and to identify the challenges and opportunities that our organizations, our members, and their stakeholders may face in the future. We use research to reach fact-based conclusions that will help us make good choices.

If you don’t have formal training in research methods or the time to take a graduate-level research methods course (which most of us don’t), how can you be sure that the research you’re using or sponsoring is giving you the insight you need to make good decisions? How can you protect your association’s reputation as a trusted source of unbiased information for the profession or industry you serve?

Step Up Your Skills

You need to build your information and research literacy muscles.

That starts with asking questions like:

• How was the data in this study collected?
• Who was included in the research? Who was excluded?
• Who collected the data? Are they a reputable source?
• Does the study draw conclusions without data to support them?
• Are the findings transferrable to your audience, research questions, or inquiry?
• When was this research done? Are the results still relevant/reliable now?

You’ll also need to:

• Apply these questions to external research reports and datasets, as well as to research your association conducted previously. Get a second opinion if you are uncertain. Ask research experts within your organization, volunteer leadership, or professional network for their assessment.
• Familiarize yourself with key research concepts such as: sampling, margin of error, confidence interval, statistical significance, data hygiene, and research bias.
• Understand your options for collecting and interpreting data and the pros and cons of each, for instance:
  • Quantitative research, which is designed to ask questions and collect responses numerically with the goal of coming to the kinds of answers that can be described by levels of statistical significance and degrees of confidence.PRO: Surveys are the most common form of quantitative research, and they provide reassuringly specific answers.CON: However, surveys are particularly susceptible to design flaws in either the questions, the answer options, or both that can bias the results.
  • Qualitative research, which is designed to gather data through conversation. Interviews and focus groups are the most common types of qualitative research.PRO: While qualitative results are necessarily impressionistic and anecdotal, they allow you to delve into subjects’ motivations, the why behind what they think, prefer, and do.CON: However, interviewers walk a fine line between creating a high level of trust with interview subjects, which is necessary for success, and inappropriately influencing their responses which can bias results.
  • Mixed-methods research is designed to gather both quantitative and qualitative data, either in parallel or sequentially, and can lead to richer set of results than one method alone.
• Develop a basic understanding of the different types of bias that can creep into research (sampling bias, response and nonresponse bias, the Hawthorne effect and social desirability bias, bias in your instrument or measurement scales, and the like) so that you can take steps to guard against them.
• Learn the key ethical practices that support the responsible conduct of research. Associations face additional complications in designing research studies related to ethical practices in research, given that we’re almost always working with data about people (otherwise known as “human subjects research”), we gather sensitive data such as salary information and personally identifiable information (PII), and we often need to take steps to avoid antitrust liability.

Getting Smart About Research

Ultimately, your goal is to “get smart about research” and help your association become a learning organization, where your entire team develops a deep curiosity about what drives your members, about their worlds, and their operating environments. You need to get to know members beyond their purchasing and posting habits to uncover their biggest problems and most important goals and provide the right solutions to the right people at the right time (and price).

You accomplish that by becoming a sponge for information, and by sharing that information openly and transparently within your association’s staff team and with your volunteer leaders and members. Over time, you’ll create a virtuous cycle of ongoing studies that ask increasingly insightful questions, working with the entire community your association serves to keep advancing your research and your association’s mission.

It’s important for associations to get this right, both so that association executives have the best possible chance of making good decisions about how to invest limited association resources to generate the best return for members, and because associations are viewed as trusted, unbiased sources of information for the members and other audiences we serve. It’s incumbent on us to provide quality research products so we remain worthy of that trust.

To Learn More

Does this all sound complicated?

It can be, but don’t worry – we’ve got you covered!

You can learn more about these and other critical research concepts and read stories of associations doing great work in their research programs to serve their members and the professions and industries they serve in our new whitepaper, Caveat Emptor: Becoming a Responsible Consumer of Research (available at www.getmespark.com/whitepapers), in which we tackle the sometimes thorny issue of what you need to know and do to be a savvy consumer and sponsor of research even if you DON’T have a formal background.

The monograph includes:

• Case studies with the American Association of Colleges of Pharmacy, the Association of American Medical Colleges, the Casualty Actuarial Society, and IEEE.
• A plain English review of key research terms, and a brief explanation of the rules of formal logic (and how they affect research work).
• An interview with Dr. Sharon E. Moss, co-editor (with Sarah C. Slater) of The Informed Association: A Practical Guide to Using Research for Results, on ethical practices in research.
• An interview with Dr. Joyce E. A. Russell, The Helen and William O’Toole Dean at Villanova School of Business, on developing discernment in assessing research.
• An interview with Jeff Tenenbaum, Managing Partner at Tenenbaum Law Group PLLC, on avoiding antitrust liability.
• Recommendations for books, articles, websites, podcasts, and courses you can use to improve your research skills.
• A series of thought questions for you to use to spark discussion with your team.
• An extensive list of resources in case you want to dig deeper on any of the topics addressed.