Web-Based Surveys

    collage of natural resource and outdoor recreation photos
Click for a list of reports.
Jump to:
Click for a list of reports
Overview
Appropriately designed Web-based surveys are methods of public opinion polling where a known group of potential respondents are invited to participate in completing a Web-based survey, and their responses are submitted electronically by means of the internet. Web-based surveys are an excellent survey method to use when the sample consists of known respondents with Web access, as would be found in an internal survey of an agency or organization in which all potential respondents were known and had guaranteed Web access through their workplace. Web-based surveys are also highly effective in augmenting response rates when respondents contacted through another medium (mail/telephone) indicate that they would prefer another alternative method of responding.
Recent Responsive Management web-based suveys include a study for the National Oceanic and Atmospheric Administration's (NOAA) Coastal Services Center (CSC) to assess opinions on and interaction with the CSC among coastal resource stakeholders for which a web-based survey was appropriate for the known population of stakeholders. Another recent web-based study conducted by Responsive Management entailed a market inventory and needs assessment for the Sapelo Island National Estuarine Research Reserve (SINERR) to assess existing programs related to coastal training and to determine current and desired levels of coastal training among decision-makers in Georgia. The web survey was distributed to a known population of regional Georgia coastal development officials and staff.
Response Rates
Web-based surveys hold promise. Among the many advantages Web-based surveys bring are rapid return of data, reduced effort in data handling, and potentially lower study costs. With convenience come caveats. The most important caveat is that there are challenges calculating response rates and generalizing findings from data collected in an open Web-based survey. In theory, when performing an open Web-based survey every Web browsing person who sees the survey and does not complete it is a nonrespondent, so keeping track of every visitor may indicate phenomenally low response rates. Another problem with open Web-based surveys is that the results of open Web-based surveys are not generalizable to the general population. Surveys posted on the Web are not viewed by the general population but rather a self-selected group of Web users, and, according to much research, Web users are significantly different in many ways from non-Web users. Further complicating results, those Web users who arrive at the survey page are different than those Web users who do not arrive at the survey page. And finally, those Web users who arrive at the survey page and complete the survey are different from those Web users who arrive at the survey page and do not complete the survey. Even small differences can make a substantial impact on the conclusions of a study.
The Web today is much like the telephone was in the past: a convenient, useful tool with some drawbacks. In the 1948 presidential election, The New York Times declared, inaccurately, that "Thomas E. Dewey's Election as President is a Foregone Conclusion." Many survey researchers today believe that the inaccurate election prediction was the direct result of an inappropriate sampling strategy due to reliance on telephone polling. In 1948, houses with telephones were not representative of the voting public. Telephones were less common at the time, which, among many other issues, made the sample non-random, and, therefore, not generalizable to the general population. The level to which the sample reflects the target population defines the level to which one may rely on the accuracy of the results.
Web and e-mail usage will likely become as commonplace as telephone usage with the passage of time; however, at the present time, there is a need for great care in sample designs when using Web-based surveys. The Web, however, is an excellent tool for targeting known audiences who have access to the Web, and for inducing responses among those who prefer the Web over other survey mediums.
The level of error for and the meaning of every finding in a study are inextricably linked to the response rates. Good response rates are often evidence of good survey design and implementation. This may be the reason that some organizations put great efforts into "redefining" what a "response rate" means. Obfuscation of true response rates gives an inaccurate accounting of actual survey performance. Responsive Management consistently achieves high response rates despite using conservative, industry-accepted methods for calculating them.
Web Survey Procedures and Facilities
High-quality data collection is critical to good survey research. With this in mind, Responsive Management has designed a facility that stresses the importance of highly-accurate computerized surveys on natural resources and outdoor recreation. Because Responsive Management specializes in researching natural resource issues, we conduct surveys only on natural resources and outdoor recreation issues.
Questionnaire Design and Pretesting
Responsive Management assists in developing web-based survey questionnaires, providing guidance, expertise, and professional survey writing services throughout the survey instrument development process. Responsive Management designs the survey questionnaires based on close consultation, discussions, and reviews with the client. Responsive Management assists in all phases of survey development based on client input and our past experience with similar studies.
After the design of the survey questionnaire, the survey instrument is pretested to check for proper wording, proper answer sets, and logistical clarity. Revisions are made after the pretest, if necessary, to the survey instrument before the actual implementation of the survey. Many research firms cut costs by failing to rigorously pretest their survey instruments, which results in poor data collection. Responsive Management always conducts full evaluative pretests of questionnaire survey instruments.
Data Analysis
All survey data are processed and analyzed using Statistical Package for the Social Sciences (SPSS) 11.5 for Windows software, which is specifically designed for data analysis. SPSS software is used to perform straight tabulations, cross tabulations, statistical significance tests, and nonparametric analyses of how survey responses relate to demographic, behavioral, and attitudinal characteristics. Data processing and analyses includes coding, cleaning of data, preparation of straight tabulations, computer processing with cross tabulations, and preparation of study printouts.
Responsive Management provides a wide variety of statistical methods that are tailored to the purposes of the study. Descriptive analyses are used to examine the characteristics of the sample, while inferential statistics are used to project these analyses to make statements about the population as a whole. Nonparametric analyses are generally performed on data that are entirely categorical (e.g., gender) or entirely ordinal (e.g., increasing levels of support of a statement). Parametric analyses are generally performed on interval data (e.g., age). Univariate procedures are used to examine relationships and differences among individuals on a single characteristic. Multivariate procedures examine these same relationships and differences among individuals using multiple characteristics.
Responsive Management Experience
Responsive Management has extensive experience in the use of quantitative research on natural resource and outdoor recreation issues. Responsive Management has conducted almost 1,000 quantitative and qualitative projects over the past 18 years. Clients include the federal natural resource and land management agencies, most state fish and wildlife agencies, state departments of natural resources, environmental protection agencies, state park agencies, tourism boards, as well as most of the major conservation and sportsmen's organizations. Many of the nation's top universities use Responsive Management for data collection because they recognize the quality of Responsive Management's data services. Responsive Management's success in the quantitative research field comes from our expertise and strong efforts in survey design, methodology, and survey implementation. Because Responsive Management specializes in researching only natural resource and outdoor recreation issues, our senior research staff and research associates conduct surveys only on these topics and understand the nuances involved in conducting such research.

Click for a list of reports.
RM Conducts:
Telephone Surveys
Mail Surveys
Focus Groups
Personal Interviews
Park/Outdoor Recreation Intercepts
Web-Based Surveys
Needs Assessments
Programmatic Evaluations
Literature Reviews
Data Collection for Universities and Researchers
RM Develops:
Marketing Plans
Communications Plans
Business Plans
Policy Analysis
Public Relations Plans

 130 Franklin Street, Harrisonburg, VA 22801    Phone (540) 432-1888    Fax (540) 432-1892    mark@responsivemanagement.com
Content © Responsive Management, unless otherwise noted.