How Are You Going to Tabulate The Data?

Conducting An Effective Ministerial Search Survey
Prepared for the Unitarian Universalist Association
By Dr. Paul Riedesel

The process of actually tabulating a survey is usually the least apparent to volunteers, and most often an after-thought. It can be frustrating and error-prone if you have not thought out the system in advance. The decisions you make in writing the questionnaire directly affect the complexity of this tabulation process. And if you are careless in writing the questionnaire, you will have a mess on your hands when it comes time to tally the answers.

Even by 1900, machines were being used to tabulate the U.S. Census. In the 21st century, you would be foolish not to rely on computers to process your survey. Assume that you are going to need a computer file containing your data. How do you get it there?

• If you have someone in your congregation with the expertise or contacts, it may be possible to have the completed questionnaires scanned electronically. This will be the exception!!!
• The most flexible approach is to key the responses into a spreadsheet such as Excel. Each row would reflect one person's answers, with each column representing a specific question. Many people know how to use spreadsheets. Furthermore, the data can usually be read by a variety of software depending on the format in which you save it.
• A professional data entry (a.k.a. keypunching) service could quickly convert the questionnaires to an ASCII data file. If you don't know what that means, then this is not a choice for you!

How Are You Going to Do the Tabulations?
By far the best choice is to use statistical software such as SPSS. Your church will not want to buy this or any other statistical software however. Now is the time to poll your congregation for someone who has access to something like SPSS. This could be anyone in a university position in the social sciences or business. This could include anyone who is employed in a research function.

"Plan B" would be to use the database functions that are built into Excel and similar programs. That means someone has to know how to use them (I don't, because we rely on heavy-duty statistical software). For example, these functions can count the number of times that the value of "2" appears in Column E of the spreadsheet. You would have to know that Column E is where you put the codes for gender, and that "2" means male.

"Plan C" dispenses with computers completely. To tabulate gender, you make three piles of questionnaires--those that checked Female, those that checked Male, and those that have no answer or checked both or wrote in an answer. Then you count each pile (preferably twice), and make a new set of piles for the next question. Or you could leaf through them and make hatch marks to count. That's, like, s-o-o-o 20th century.

Editing and Cleaning
You cannot hand raw questionnaires over to someone to key the data. Somebody needs to scan each one for inconsistencies or other problems. The chief things to look for are:

• People who ignored "skip" instructions and answered questions they should not have. Mark out those answers in some way to show that they are to be ignored.
• Multiple answers where there should only have been one. What are you going to do if somebody checked two answers anyway? You have to make a judgment call before something is entered into the computer.
• Do you leave this person's cell blank?
• Or make a judgment about which code to keep?
• Or maybe add another column if you have a lot of people giving multiple answers?

It is also customary and helpful to assign a unique ID number to each questionnaire, and to include that ID with the computerized data.

Coding
This is a fancy word for converting the check marks, etc. on your questionnaires to numeric values. For our purposes here, computers are still idiot savants. They are incredibly fast, but incredibly literal. You have to make all the judgments. That means that you have to make the input data consistent. Think back to what a spreadsheet looks like (even if you plan to use some other method of entering the data). Each column comprises one "field" or variable. I would create a row of labels at the top of the spreadsheet with the question number or something that shows what they refer to.  Each cell can contain, at most, one code. It is almost always to your advantage to use numeric codes--hence the pre-coded values we have suggested elsewhere (e.g. '1' means less than a high school education).

• Many of your questions will have asked for a single answer, and you would normally plan to allow just a single column for this answer. It could be a single digit code as indicated on the questionnaire (e.g. "2" for male), or it could be real number such as age.
• Other questions explicitly invite multiple answers ("check all that apply"). If you offered six choices, you will probably need to allow six spreadsheet columns--let's say they are Columns G, H, I, J, K and L. If a given person checked numbers 1 and 6, you would enter some code in Columns G and L (i.e. the first and sixth).
• One approach would be to enter a 0 or 1 in all six columns, where "1" means that choice was selected.
• Another convention would be to enter a "1" in Column G and a "6" in Column L, leaving the others blank.
• If you asked open-ended questions, I would not try to type those comments into spreadsheet cells. They are not amenable to statistical analysis anyway. Have them typed separately in a word processing document.

Here is a modest example of a spreadsheet being used for data entry.

We have assigned each person an ID number. Person #104 did not answer Question 1, so it is left blank. Those whose had a code of "2" in Q2a were supposed to skip to Q3. They have a blank in Column E (Q2b). Question 4 had six choices, and people could check as many or few as they wanted.   The first person (101) checked the first and sixth choices so has "1s" in Columns G and L.   The second person (102) checked the second, third and fourth choices so has "1s" in Columns H, I and J.

Statistics!
In most circumstances, all you will need are simple percentages. One decision that you need to make is whether to calculate and report percentages based on the total number of questionnaires or the total number of valid answers to a given question. Either choice is acceptable, but if you use the former (base=all) then you need to report the percentage of missing cases as well as the rest. If you offer an explicit "don't know" or "not sure" answer category with its own code, think about whether you want to include them in your base.

A special case is a so-called contingency question. If you only asked certain questions of those who have children, then the base for the percentages should be only those who met that qualification--not the full sample.

Even without a computer, we can easily calculate that 40% chose answer 4 in Question 2 (2 ÷ 5 = 40%). Half (50%) of those answering Question 1 chose answer 1, but only 40% of the total sample chose answer 1.

When reporting results from something like Question 4, you usually make a single table out of it. With our five cases and assuming that the six choices were types of Sunday morning involvement, the table might look like this:

Reporting out attitude data is more complicated. If you have a lot of questions, having a table for each one may overload the reader. It is often sufficient to use a simple summary measure. Some detail is lost, but you make up for it in clarity.

• Adding and presenting the combined percentage who "agree" and "agree strongly" is a decent method for comparing the strength of various attitudes you have measured. Impress your neighbors by calling these the "top-two-box scores."
• For the more visually-oriented, you are free to make bar charts showing either the top-two-box scores or the mean rating (having assigned a value of 5 to "strongly agree", etc.). If you have assigned a value of "6" to "Not Sure" answers, leave them out of the computations!

Final Comments
However  you actually report your survey findings, the reader should have ready access to:

• The exact, original wording of the questions and answer categories
• The number of questionnaires upon which the percentages are based (sample size)
• The qualifications for any contingency questions (e.g. "Based on only those with children in grades K-12")

    Return to Main Page
1. What tables and data do you wish to include in the congregational packet that is shown to potential ministerial candidates?
2. What must you do to get the questionnaire into everyone's hands--and get it back again?
3. What questions are you going to pose to survey participants to arrive at the tables you want, and how will you pose them?