by Joel Best
While a fascinating read for all of us, this is most applicable if you're combining secondary research with your surveys. You'll never look at "facts" the same way again.
]]> by Joel Best
While a fascinating read for all of us, this is most applicable if you're combining secondary research with your surveys. You'll never look at "facts" the same way again.
]]>
by Darrell Huff
There’s a reason this is still in print after 50 years, and that’s because we still fall for the same creative charting tricks.
]]>For non-cash rewards, your first thoughts may be of items related to your survey or industry. The challenge there is coming up with an item they haven't already bought for themselves. Instead, try to think outside the box—like the best gifts, it may be something the respondent wouldn't think to buy themselves:
Business populations are also better motivated through emotion than logic. If you're running a conference, you may be inclined to give away a free pass for next year. That's a great carrot if attendees are self employed or personally paying for their continuing education. However, if most of the attendees are sent by their employer, they'll be less motivated by your $1,500 pass than a $200 goodie. Likewise, offering a copy of the survey results is only a significant incentive if the respondent's company wouldn't be willing to buy a copy.
Since the point of incentives is to increase your response rate, it's worth a moment to consider whether some of the funds may be more effective elsewhere. This could take the form of improving the respondent experience by getting help tuning the survey or using better technology. It could also be used for follow-up, such as hiring a temp to call non-respondents.
When you have the total you plan to spend on respondents, look at what you could spend individually, on a few items, or on one prize—a chance at a big item can be more motivating than a modest guaranteed amount (there's a reason we have the lottery).
Remember that what matters is the respondent's perception of the reward. I saw one project where they planned to offer $100 per person, which is almost an insult when asking a CEO to complete a 3 stage 60+ minute survey. However, that $100 could become a nice sign of appreciation when sent to the charity of the CEO's choice, or a tempting carrot if the budget for all 1,000 respondents was pooled into a few prizes.
If you are providing individual payments, there are a few logistics:
by Howard Schuman & Stanley Presser
Despite being in a quantitative industry, surveyors rarely conduct tests to measure what happens when we rearrange questions, add a neutral point in a scale, or make other adjustments. If you're ready to absorb some more advanced issues, complete with footnotes, this is a great book to pick up. Note that the 1996 copyright is simply a reprint of the 1981 text.
]]>
by Harry Beckwith
When you want to broaden your perspective, this will help you understand how customer and employee satisfaction mixes with and reinforces other marketing efforts. While the author focuses on services, it's useful in any industry—we're all competing on intangibles these days.
]]>Would you recommend a friend or family member attend University of California Davis?
Recommend for what? I'm aware of the school's solid reputation in engineering and veterinary medicine, but have no notion where their other programs rank.
Recommend for whom? I have to think of an individual as to whether the programs, lifestyle, location and tuition (in-state resident vs. full) would be a fit.
]]> Therefore, my answer to the broad question was "It depends" but that would be my answer for almost any university—not very useful for making distinctions among schools. Regrettably, it was a Yes/No question, so the interviewer was left with no response.In your own surveys, do you have questions people would answer with "It depends..."?
There are a few common ways they'll slip in:
Sometimes, all it takes is a little qualifier:
Would you recommend Tesla to someone shopping for a sports car?
Or even making the question more generic:
Overall, how would you rate our service?
In general, do you...
The more generic approach helps respondents whose "It depends" is based on inconsistent experiences.
]]>First and foremost, if your organization is new to surveys and doing this in-house, start simple! The goal of any survey is better information for decision-making, and a modest quantity of information that you actually use is far more valuable than a complex picture that may be flawed or too troublesome to maintain.
]]> For any business, there are probably several ways in which you interact with customers—the following example is for a B2B software company.Short surveys on customer service/support are a great way to begin, because they're straightforward to write, conduct, and analyze. By short, I mean 3-5 rating questions, 1-3 demographics, and a comment field. While this won't be everything you want to know, it will include a few key satisfaction measures to highlight areas needing work and begin your trending. The comments are your catch-all for anything you didn't include on the survey, as well as providing the why behind rating values.
These are perpetual surveys, a link in the footer of every support ticket going to a customer, or a message sent on closing a ticket. The simplest way to manage them is with a generic link, but if you want to get more sophisticated you can embed information in the URL, such as a ticket ID (more at the end of this article). Typically ongoing surveys are revised periodically, but you want to keep at least a core steady so you can maintain the trending.
A software company will often provide installation or training services, which is an opportunity to touch base on that experience. In other industries, you may have similar events, such as when a consultant concludes a large project. In addition to providing aggregate data, these may be reviewed upon receipt so that poor experiences can be remedied.
The last piece—once you get the others up and running—is a detailed customer survey. Some companies do this annually, but for others it may make sense to run the survey at a milestone, such as before development begins on the next product generation. This will capture their satisfaction with specific features of your products and services, overall impression of your company, and where you can grow or shift to better serve their needs. Some of the questions will come from repeated themes in the other surveys' comments. The big survey is where you want to make an extra effort for the response rate, perhaps including an incentive.
The above is one approach. Some companies just do the big periodic survey, and not the ongoing feedback. Others will do the ongoing small surveys, but use an exploratory telephone interview for annual feedback instead of a quantitative survey. Be open to the best fit to your organization's information needs, customers, expertise, and budget. Customer service measurement is an ongoing function, so you'll have plenty of opportunities to refine your system over the years.
The customer service/support surveys are often very straightforward, making them well-suited to a low cost service provider.
There are two places where you'll start needing more power:
One strategy is to start with the simple software as you get acquainted with surveys. Then when you're ready to invest in a more sophisticated application, you'll be in a much better position to evaluate how the tools fit your needs.
]]>For customer surveys, event or training feedback, and similar topics, this sort of identification is generally not an issue—you can simply e-mail individual passwords to your list.
For market research, if you use a panel vendor your organization can remain anonymous while still providing compensation.
For employee satisfaction and other sensitive surveys, you may need to forgo the restriction or use a third party consultant or host who will assure anonymity. You can also issue passwords in a way that isn't tied to an e-mail invitation, such as picking a slip of paper or having people checked off as they arrive at a set of kiosks to complete.
When a pre-issued code isn't an option and you do think people will stuff the ballot box, here are some strategies you can use:
Version 1
Written by my client, and primarily driven by their contact, the CEO. The survey focused on evaluating the organization as a whole, though a significant emphasis was on communication and fundraising.
Version 2]]> The challenge? How to find the best balance—diplomatically of course.
Written by one of the non-profit's board members. The questionnaire was far more granular, focusing on specific programs offered.
One way to deconstruct a situation like this is to list all the topics/departments you're evaluating, and then simply count how many questions you have for each one.
| Version 1 | Version 2 | |
|---|---|---|
| Overall organization | 24 | 1 |
| Facilities/environment | 13 | 9 |
| Communication | 7 | 0 |
| Fundraising | 4 | 1 |
| Program A | 0 | 7 |
| Program B | 0 | 6 |
| Program C | 0 | 6 |
| Program D | 0 | 6 |
| Program E | 0 | 6 |
| Program F | 0 | 6 |
| Program G | 0 | 7 |
| Demographics | 2 | 3 |
| General Comments | 0 | 0 |
| Total | 50 | 58 |
They couldn't simply meld the two questionnaires as that would make the survey far too long. However, one can look at these tallies, recognize there's a lot of ground to cover, and say "We can only ask 1-4 questions on each topic." Naturally there are situations where an issue calls for more or less detail, but this can be a useful starting point.
If you really want to look at one specific issue, another option is targeted polls. While you don't want to flood your population with questionnaires, consider having monthly or every other month quick polls that are available from your Website or newsletter. And "quick" means 3-6 questions, not 25.
As always, comments act as a safety net when you're removing quantitative detail in the name of completions.
]]>The number of people who begin your survey should be pretty close to the number who finish it, with the few abandoned sessions due to random interruptions for respondents. However, whenever I teach workshops, I ask participants who has abandoned a survey due to length and every single hand goes up.
Yesterday I completed a survey and am embarrassed to admit I lied on one question—the only way I could get to the end and finish the screen captures for my example file. Well, technically I could have answered the question, but I was out of patience and it was easier to click "I have not attended an event in the past 2 years" (logically inconsistent with my earlier answers) than to read the list of 21 events, try to remember which I'd attended, and rank my top three.
Why was I out of patience? This survey managed to hit all the common problems:
On your surveys, are people dropping off? For telephone, and interviewer-conducted surveys you can pinpoint where people are leaving, and with Web surveys you should be able to find out the last page they submit. You may have a specific question that's a problem, but more often it will be an accumulation of smaller inconveniences that cause people to leave at different points.
Even if you're happy with the total number of responses you're getting, remember surveys are all about hearing from a representative sample of your population. The people who drop off are more likely to be in your middle majority than the passionate ends, so you really want them to finish.
]]>For Web surveys, there's a huge range of tools and services, so someone may already offer your dream feature. However, there are times when you just need something custom.
]]> I got lucky and didn't have to quest for my coder. A former colleague, who spent several years tweaking code for SurveyHost and QuestionWeb, is now freelancing. So when I have a bright idea, I can drop her an e-mail and see if she can manifest it, such as:An e-mail alert when a survey is submitted.
This is great for low volume surveys, so I just take a peek whenever I get a response. I've packaged up the code for you to use.
An address forwarder which passes along embedded data values.
For long-running surveys I always prefer to make the public address my company site rather than wherever I happen to be hosting my surveys. With this function I can not only do a simple redirect, I can also embed values such as the origination of the link, password, or product name.
A time-conditional Web page.
Now I can make paragraphs appear and disappear within a Web page based on a date/time, such as adding the link to my speaking handouts right as I begin my session.
My coder on call is Molly Magai http://sparkoid.com/. I expect that at some point I'll ask her for a double-byte translation of a survey to Kanji or some other goodie. So far I've only tapped her for general Web work, but it's reassuring to know she has the experience to tweak my survey pages themselves without changing anything critical.
]]>It doesn't have to be subtle, it can simply be a question like:
Are there any small irritations in using our product/service which we could remove for you? List up to three.
For various companies I would have given these answers:
They're all relatively small issues, but ones which make me frown a little every time I run into them and which I'd love to see fixed.
What small stuff are you overlooking?
]]>Now all of this assumes you've invested in a reasonable set of technology to process the forms, which will run you somewhere from $2,000-10,000+ depending on your volume. If you're relying on bargain basement software with poor logic for ambiguous marks you'll be spending all your time telling it whether that was a mark or a smudge, and if you're using a scanner you picked up at OfficeMax you may find yourself standing over it all day as you move stacks of paper in and out.
From start to finish, four components have to pass information back and forth, so making sure all the pieces are compatible and testing fully is a must:
Some survey software embeds the scanning function in their code, while others will have recommended partners to make it all run smoothly. Whatever you do, don't print your forms before buying and testing your scanning setup! Or scan all 5,000 forms before testing the data format imports into your reporting software. Seriously, I've seen it tried and it's just technology roulette.
The scanner itself is a simple-minded device. It will recognize light and dark on the page—everything else is software. There are two main types of scanners: mark readers and image scanners. Mark readers are the Scantron and NCS type forms we all grew up with, where the bubbles are in tidy columns and an edge of black rectangles marks the rows. While these still have their place and you may have one around your office, they tend to be more expensive and less flexible, so this article is about image scanners.
Image scanners use the same technology as the multi-purpose office scanners you see in your local office supply store. They take a picture of an entire page, including all the instructions, and save it just like you would a scan of a photo. Software then interprets the picture, and extracts data from the pixels.
The two main types of image scanners are flatbed and sheet feeder.
Flatbed scanners have a glass surface on which you place the document to be scanned. Some will have a sheet feeder option as well (also known as an ADF or automatic document feeder) which will pass pages across the scanning surface. Flatbeds tend to be general-purpose office scanners, with fairly high resolution, color, single sided scanning, and a modest page per minute scanning rate.
Pure sheet feeders, on the other hand, are designed to quickly process stacks of paper. They may only scan in shades of gray, and may be a lower resolution than the general purpose scanners because they’re being used to capture data rather than illustrations. The advantage of sheet feeders is their greater speed, greater sheet feeder capacity, the option to scan both sides of a sheet at once (duplex), and longer duty cycle.
In the sheet feeder realm, popular scanner manufacturers are Fujitsu, Canon, Panasonic and Ricoh—not the brands you're used to seeing in a store. For most surveyors handling batches of a few hundred to 10,000 forms, you should be good with a scanner from $1,000-4,000. If you don't already have a departmental scanner (often these make sense as a shared resource, like workgroup laser printers), find the software that fits you first and they can point you toward a compatible device.
Since the scanner simply produces an image, what’s next? OMR (Optical Mark Recognition) or OCR (Optical Character Recognition) software converts the scanner’s file into a data file that you import into your database, reporting software, or survey program. While individual applications vary, you’ll typically “train” the software to understand your survey form. This is done by running a blank survey through the scanner. When the software brings up the page image, you then identify regions on the page which contain data fields (often by drawing boxes around a set of bubbles). For each of these fields you’ll specify the valid responses, such as single or multiple answer, values, and a field name. If the program supports it, you’ll also identify write-in or image fields, for which it will capture pictures of the responses for manual data entry.
Once the software has all this information, you start scanning completed surveys. It compares each respondent form to the template, ignoring all the pre-printed information, and converting the extra marks to data fields per your definition. If your survey has write-in fields, a data entry person will need to review each one and type the value (don’t worry, this is done quickly as a batch after all the forms are scanned). There’s also usually a cleaning pass for borderline marks, such as a respondent only partially filling in a bubble, or erasing a response. Then you’ll export the data file for analysis in your survey application, spreadsheet, database, or statistical software.
Most surveys are scanned with the less expensive OMR software because of the limitations of “character recognition.” At this time, software can only automatically recognize machine printed text, such as the pages in a book, or block printed handwriting spaced out in one character per box as you’ve seen on forms. Both of these methods can have significant error rates, so they require auditing and/or interpretation of characters the software identifies as questionable. And software which can recognize block printed script is still very expensive—sometimes costing ten times more than OMR software.
The software which comes bundled with your scanner is unlikely to process your surveys well (though the bar is constantly being raised). When shopping for survey processing OMR software, this is the basic feature set:
Additional niceties include:
There are a number of applications which may suit your needs, but one which I've worked with for many years as a solid fit and good value for surveyors is Remark Office OMR http://www.gravic.com/remark/officeomr/.
]]>This is a great little function, useful for combining bits of information into a single value. Say you have the following data:
| A | B | C | |
|---|---|---|---|
| 1 | Jane | Doe | jdoe@domain.com |
You want to end up with a standard name and address format:
Jane Doe <jdoe@domain.com>
Your formula would be:
=CONCATENATE(A1," ",B1," <",C1,">")
Individual pieces are separated by commas, with any new text you're adding in double quotes—here spaces and angle brackets.
Update: Some of you appear to be searching for a solution that adds double quotes as part of the string, such as combining the two cells below into Adding "quotes". I've found two inelegant approaches.
| A | B | |
|---|---|---|
| 1 | " | |
| 2 | Adding | quotes |
The first formula relies on a platform-specific character number— 34 on Windows and 22 on the Mac:
=CONCATENATE(A2," ",CHAR(34),B2,CHAR(34))
Or you can reference a fixed cell containing a double quote:
=CONCATENATE(A2," ",$A$1,B2,$A$1)
Ever receive a name list in ALL CAPS? This function is a fast way to change the format. Proper can be combined with Concatenate—imagine our original example started with JANE DOE in cells A1 and B1. You have two options for the nesting:
=PROPER(CONCATENATE(A1," ",B1," <",C1,">"))
generates
Jane Doe <Jdoe@Domain.Com>
while
=CONCATENATE(PROPER(A1)," ",PROPER(B1)," <",C1,">")
generates
Jane Doe <jdoe@domain.com>
UPPER and LOWER work in the same manner. If the list is of modest size I'll take a visual scan to check for special name formats like ones with a "de", but if it's 20,000 names I'm satisfied with knowing the list is in a better format than it started.
I use Paste Special Values both to bring information into Excel without formats, but also to convert cells which contain formulas into simple data (such as from Concatenate or Proper). You probably know you can access this through the Edit menu, but did you know there's a toolbar button you can add?
Follow the steps below to add buttons—Paste buttons are in the Edit category.
At the bottom of the Paste Special dialog is an option to "Transpose." When you use this, row 1 becomes column A, row 2 column B, etc. There's no direct hotkey for Transpose, but you can add a Paste Special button to the toolbar the same way I describe for Paste Values.
Hands down, the four buttons I use most on my toolbar are:
These insert a row, insert a column, delete a row, and delete a column. When you have a row or column copied to the clipboard and use the insert buttons, Excel will paste as a new row/column instead of over the selected row/column.
You can add these to the toolbar just like how I describe for Paste Values. The two insert buttons are in the Insert category, and the deletes in the Edit category.
Imagine an expense tracking spreadsheet. For your mileage, the value is the same for a given expense report, but does change every year. So, set up your spreadsheet something like this:
| A | B | C | D | |
|---|---|---|---|---|
| 1 | Rate | 0.485 | ||
| 2 | Start | End | Miles | $ |
| 3 | 12,023 | 12,045 | =B3-A3 | =C3*$B$1 |
You can now copy the formula in D3 for all subsequent rows, with it always referring back to B1. If you ever cut and paste the value in B1 to another cell, the reference will update to the new location.
While typing a formula, you can fix the location with the F4 key. Sometimes you only need to fix the column relative to your formula, in which case you'd use $B1, or to fix the row B$1.
This is great for data cleaning and for any time you're using Excel for project tracking. What conditional formatting does is change a cell's background and/or font when its contents matches your criteria. For projects, I'll often have a series of milestones where I'm tracking completion, and I'll color code the cells with "p" as partial, "c" as complete, or "na". For data, I'll sometimes be looking for exceptions, such as this capture here which highlights empty cells as red, and cells 1,000 or higher as green.
Click image to see full size
In the first rule, that's two double quotes after the equal sign.
For your projects, just:
Note: If you have mixed numbers and text data, a rule of Greater or equal to 0 will highlight the text (such as your column or row headings). However, if you set the rule to Between 0 and a million or so, it will just highlight numbers.
I find If statements hugely useful, both to transfer data between cells and also to deal with missing data points. For example, if you have the following formulas:
| A | B | C | Result for C | |
|---|---|---|---|---|
| 1 | 25 | 1000 | =B1/A1 | 40 |
| 2 | [blank] | 2500 | =B2/A2 | #DIV/0! |
I find the #DIV/0! error distracting in spreadsheets, because often the blank or 0 cell is allowed in my data set. So instead I can make the formula a little smarter with an IF:
=IF(A2>0,B2/A2,"")
This translates to: If A2 is greater than 0, Then use the formula B2/A2, Else make the cell empty (two double quotes in a row). When you can't get the result you want with a single If, you can start nesting them in the Else clause, such as:
=IF(A2>0,B2/A2,IF(A2<0,B2/A2,""))
Which works with any positive or negative number in A2.
]]>This is great—unless you have certain types of data such as East coast Zip codes.
]]> Here's an example .CSV file: three fields listing a name, date in EU format, and Zip code.Name,Date in EU format,Zip
Jane Doe,1/5/07,01252
John Smith,19/7/07,98107
If you double-click on this file, Excel opens it as:
What Excel has done is classify all the columns as "General" format. The names are fine, but the May 1st date has been changed to January 5th. While it may look like the date is still salvageable as the EU format, Excel codes dates with an underlying 5 digit value and this cell is now the January 5, 2007 value. On that same row Jane's Zip code has lost the leading zero, which you can see in the formula bar as completely gone, not simply hidden from view. On John's record the July 19 date is being treated like text (note the left alignment) because Excel doesn't recognize months greater than 12.
If at this point you close the file without saving, it will still be OK, but the minute you save, the values will be permanently changed.
Dates in formats other than M/D/Y and pure number IDs such as Zips and some customer or employee IDs are the most vulnerable. If you don't have those in your data file, you'll generally be OK with a double-click to open.
When you do need to be careful of the formats, you want to go through Excel's import dialog. You can get to this dialog by importing, or by opening a file with an extension Excel doesn't recognize (.txt will sometimes work).
If importing:
Now the wizard, which is identical for both importing and opening an unrecognized file:
Click image to see full size
In Excel, what Text essentially means is "Don't change anything." While I could go through the individual columns during Import and set the proper date, etc., the all Text approach is faster. Once the data is open, I can always select a column and use the Format menu, Cells to adjust the Category if needed, but as long as it's set to Text Excel will leave the data alone.
]]>