How to make a survey for your conference

October 8, 2009

in Basics, How to

Photo by Thomas Schlijper during PICNIC 09

Mobile Polling Station - Photo by Thomas Schlijper during PICNIC '09

This article deals with some of the alternatives for executing a survey or poll during or after your conference is over. Making a survey to know the attendee’s opinion about your event, its contents and execution, is a straightforward action to obtaining information firsthand (but not the only one, see note below).

There is no “one best method” to make a survey and it depends mostly on the resources available and the type of audience you have, eg: If you’re at a BarCamp or web-related conference where most people might have a laptop and access to the internet, an online form will be the fastest way… but if you’re organizing a traditional event with older people, distributing a paper form with the questions will surely be the best choice.

For events, in general I suggest that you make multiple choice or yes/no questions & answers but also add space for text comments when necessary. People often want to tell you something more about their experience and this is a good moment to let them do so.

The following are some alternatives I have practiced in the past. I encourage you to share your own methods in the comments at the end of this article.

Paper survey

This might sound as “old school” -and indeed it is- but it’s still one of the most successful options in a traditional conference or when you don’t have a direct means of contact with all the delegates after the conference is over (eg: you don’t have their email addresses to send them an online survey). Be sure to format the the survey so that fits in one page, and preferably don’t print double face. Clearly state whom they have to give the survey to (to the stewards, etc) or where they have to leave it (at the info desk, on the chair, etc)  after they have completed it. Remember to instruct the stewards to collect them in one particular place and have someone from your team to be responsible for them. Distribute the survey well before the end of the event so that people that leave early can complete it too. Remind the attendees about the importance of completing the survey to give their opinion and help you improve future ones.

The biggest drawbacks I find with paper surveys are: 1) it’s not eco-friendly and you often end up throwing away many unused surveys and 2) you need to manually input all the results into a database for further analysis. The main advantages are that you obtain feedback while it is still “warm” and you don’t require extra technology to implement it, it is a rather universal solution. Most people will have a pen for filling in the questions or they might have been given one during the conference.

Email & Online surveys

Email/Online surveys are amongst the most effective systems nowadays because almost everyone has an email account (don’t give that for granted though, it depends on who’s your audience). The most important issues when using this method is to have a good timing on the delivery of the email (eg. if you send it at night it might get lost in the inbox under other emails arrived during the morning), a clear message and an attractive incentive for people to fulfill it.  Keep the email with the invitation to the survey short and concise, clearly stating how much time it will take. Remember that even if for you the cost of making an online survey might tend to zero, the time of the person answering has a value too, and for him/her it is much higher than whatever it costs to you to reach him/her.

Incentivize the answering by offering a prize such as a [limited number of] free tickets or a discount for your next event. Clearly state what the prize is, how many are you giving away, etc. If for example you’re rewarding 5 people with a free ticket, offer the rest a nice discount so that everyone is a winner in the end.

If you have a closed community gravitating around a website, you can decide to publish the survey on the homepage. Just be sure to activate the option that tracks the origin of the answers (usually based on the IP number) so that you don’t have multiple answers from the same person.

Create a url shortcut like so that people can input the address directly on the browser.

Read more tips ob Email surveys by here.

I have often used Survey Monkey. It is not perfect or fancy looking, but it does its job. The interface and graphic design options cold be much better or at least more “2.0” but I guess it’s producing enough money as it is so they don’t see the point in upgrading.

Interactive “live” survey

This is one of my favorite methods for certain kinds of qualitative or “mood” surveys. It requires the interaction of the conference’s delegates with a physical object (like a voting totem) or person (voting aid) to produce the results. A good example of it was recently implemented with success by Mediamatic during PICNIC ’09 using their Mobile Polling Station. This method was based on the fact that every attendee to PICNIC ’09 had an ikTag based on RFID technology that was connected to their user profile on the PICNIC network. The mobile polling stations consisted of a very visible character (see photo) that carried around a voting system based on RFID readers that allowed the attendee to choose one answer by positioning their tag on top of it. The result was automatically transferred to the system and the statistics were updated live on the website. This method is particularly useful to map the “mood” of the people around a specific topic. As an example, some of the questions asked where:

  • How do you feel today?
    • Possible answers: Overwhelmed; Bored; Like dancing; Horny and so on…
  • Will you attend PICNIC ’09?
    • Possible answers: Yes, I loved it! ; No, but I cannot tell you that ; Maybe, only if it’s as good as this one

Mobile survey

The Mobile Survey -answering to a survey using your mobile phone- will be the killer app when it will finally become mainstream. You can make the survey instantly available by directing people to a url or sending it through a push notification. It could also be embedded in the mobile application of your event so that it automatically appears when you announce it. The Mobile Survey unites the benefits of all three methods explained above:

  • Low cost, as email or online forms: the cost of developing the survey or buying a mobile survey system will drop in time.
  • Ubiquitous and Instant: almost everyone carries a cell phone nowadays, and in the near future most of them will have internet access by default, as it happens in Japan or Korea. People are able to answer instantly.
  • Results are automatically available: no manual input is required, opposite to what happens with paper forms.

Statistical significance and sample size

Making a valid survey is not only about using the right timing, most appropriate delivery method and asking good questions: you must respect some basic statistical principles to make the results of your survey valid from the quantitative point of view. While I don’t intend to describe the required knowledge of statistics here, there are two concepts you should bear in mind:

  • Statistical relevance: “the amount of evidence required to accept that an event is unlikely to have arisen by chance”. Read more about Statistical significance on Wikipedia or on
  • Sample size: the amount of people you need to interview in order to obtain results that reflect the behavior of  the population being studied (with a given level of precision). This is a useful sample size calculator tool.

Note: Surveys are not the only way to find out what people think of your event. Search for comments and opinions on twitter, blogposts and other sources that talk about your event. Engage in a [positive] conversation with the authors and commentators, even if the they said negative things. Actually, engage into conversation ESPECIALLY if they had negative comments. As Jeff Jarvis says in his book “What will Google do?“, “Your worst customer is your best friend” (because he cares enough about your conference to talk about it).

Previous post:

Next post: