Introduction
Conducting a survey with 10,000 interviews or more, especially when the field needs to be conducted in several dozen cities, is not an easy task. Especially if the interview is long (e.g., 1 hour).
To complete the field on time, ensure high data quality, and reduce questionnaire losses, it is necessary to use software specifically designed for data collection when working with large fields.
This article describes how to effectively manage a large field using FlowSurvey software.
Challenges in Managing a Large Field
First, let's list the problems faced in managing a large field, and then show how FlowSurvey helps solve these problems:
- It is difficult to monitor quota fulfillment
- High probability of quota overflow
- It is difficult to monitor the quality of interviews
- It is difficult to provide interim results
- It is difficult to consolidate all data into one report
- Technical issues on interviewers' devices
- Problems in training personnel
- Difficulty in making changes to an already launched survey
Let's analyze each problem separately.
1. It is difficult to monitor quota fulfillment
If there are many interviewers working in areas without internet access, it is difficult to track who has completed how many interviews and how many are still left. This problem is compounded if quotas are used, as it is necessary to monitor not only how many interviews each interviewer has completed but also which quotas are closed and which still need to be filled.
2. High probability of quota overflow
This problem is related to the first one. With complex quota fulfillment tracking, there is a possibility that different interviewers will collectively conduct more interviews than required. This leads to additional expenses.
3. It is difficult to monitor the quality of interviews
Data should not only be collected but also reliable (valid). For example, does the interviewer list all response options when reading the question to the respondent? This significantly affects the quality of collected data, as the respondent might choose an option that was not read out. Additionally, unscrupulous interviewers might complete the questionnaire themselves if, for example, the respondent refused to continue the interview halfway through and left.
4. It is difficult to provide interim results
Conducting a survey with a large field can take a considerable amount of time. During this time, you, managers, or research clients may need to view the interim collected data. If the system does not allow for simple reports to be built on the already collected data at any given time, this can also be a problem.
5. It is difficult to consolidate all data into one report
If the interview is long (about 1 hour), there can be many questions and response options in each interview. For a large field, this means a large data set. The system should allow for the export of such a large data set.
6. Technical issues on interviewers' devices
If the questionnaires are lengthy, there can be interruptions during the interview (an incoming call on the device, the battery ran out, the interviewer accidentally closed the application). In case of interruption of a long interview, it should be possible to continue it from the question where it was stopped (even if the application was unloaded from memory). Otherwise, there can be many interrupted and, consequently, lost interviews.
7. Problems in training personnel
A large number of interviewers may be involved in a large field. This means that each interviewer must be trained to use the program. The program should be easy to learn and use, otherwise, it creates a load on the support department and can lead to delays in conducting the field.
8. Difficulty in making changes to an already launched survey
After the survey starts, there may be a situation where, for example, an additional response option needs to be added to a question or a whole question needs to be added, or there was a typo in the survey that needs to be corrected. In the case of a large field, this can also be a problem. The software should allow for changes to be made to an already launched survey.
This is not an exhaustive list of problems that can be encountered when conducting a survey on a large field. However, solving these problems can help prevent related issues.
Field Planning
Now let's show how FlowSurvey software helps solve these problems with proper planning.
- Location Planning
- Quota and Target Setting
- Device Binding
- Controller Binding (if required)
- Launching and Conducting the Survey
- Quality Control of Questionnaires, Rejection of Questionnaires
- Quota Filling Control
- Quota Overflow
- Technical Issues on Devices
- Viewing Interim Results
- Data Export
Let's consider each of the steps.
Location Planning
Locations refer to the places where interviews are conducted and/or assignments for interviewers.
We assume that the quota for conducting the survey has already been provided, and we know how many interviews need to be completed in which region, city, and street.
Suppose the quota looks like this.
Location planning depends on how many places/assignments are present in the survey. With a large number of locations, it is more convenient to break them down into categories/subcategories. This allows for easier use of filters to search for questionnaires, more convenient monitoring of quota fulfillment, easier binding of controllers, and more.
Suppose we have 4 regions, each with 5 cities, and each city with 5 assignments. We plan for interviewers in each city to complete one or more assignments.
An assignment usually means completing several interviews in a specific place, possibly with a certain quota.
Thus, the hierarchy will look like this: Region 1 -> City 1 -> Task 1.
FlowSurvey supports location/assignment nesting at any level. It is not necessary for each level to have the same nesting. For example, at the first level, the nesting could be Region 1 -> City 1 -> Task 1, while at the second level, it could be Region 2 -> Task 2.
In the case of a large number of rows in the table, it makes sense to split the table into parts (e.g., by regions). FlowSurvey allows for the creation of several separate quota groups (several separate tables).
Quota and Target Setting
Locations and quotas are set in the quota settings. More details can be found in the documentation.
It is worth noting that transferring quotas and locations from Excel to FlowSurvey is quite convenient and quick. If your quota is prepared in the format shown in the screenshot (nesting levels are made as different columns such as Region 1, City 1, Task 1, etc.), you only need to select these levels, copy them to the clipboard, and paste them into FlowSurvey in the first text field for locations, as shown in the next image.
Entering target values is also easily transferred from Excel to FlowSurvey via the clipboard. To do this, select the cells with values, copy them to the clipboard, and paste them into the first cell for entering quota values, as shown in the next image.
Thus, the entered quota looks like this:
Device Binding
A separate login and password are created for each interviewer, which they will use on their device. One login and password can only be used on one device. If an attempt is made to log in with a login that is already used on another device, an error will be displayed. This prevents situations where two interviewers work on the same login and problems with identifying which questionnaire was completed by which interviewer.
When we assign an interviewer a task, we bind their login to the task (location). We can bind several logins to one task or several logins to several tasks. We can also bind logins at any level of nesting, for example, to City 1, then all tasks in City 1 will be bound to the specified logins.
The FlowSurvey system allows for any number of logins to be bound to any number of tasks at any level of location/task nesting.
However, it is worth considering that if several logins are bound to the same tasks, quota overflow may occur. Since tablets are not connected to the Internet during the survey, they cannot check the quota fulfillment. When the quota is almost filled, two interviewers might start the same task, resulting in overflow. Therefore, it is recommended to assign personal tasks (assign only one login to specific tasks).
However, assigning multiple logins, for example, to several tasks in a city, can be more convenient. In this case, it is recommended to synchronize with the server as often as possible to get up-to-date data on quota fulfillment.
Device binding is performed on the "Device Binding" page.
Controller Binding
In some cases, interviewers in one city or region may be supervised by one supervisor, while interviewers in another region or city are supervised by another. The supervisor can monitor both the quota fulfillment and the quality of the interviews.
A supervisor from one region should not have access to questionnaires or quotas of another supervisor.
The FlowSurvey software allows a supervisor to have access only to view quota fulfillment and/or control the quality of questionnaires, while restricting access only to their region.
To do this, you need to create a controller in the "Controllers" section and give them the required rights.
Then bind the controller to the desired region/city or tasks.
A controller in the FlowSurvey system differs from a regular user in that they only have access to specific locations/tasks, while a user has access to all locations and tasks of the project if they have access to the project.
A controller can also be a moderator who analyzes the quality of the questionnaires, and their access to specific locations should be restricted.
Quality Control of Questionnaires, Rejection of Questionnaires
The quality of the conducted interview is most effectively checked by listening to the background audio recording of the interview. On the background audio recording, you can check whether the respondent is present when answering all questions, whether all questions are read correctly, whether all response options are asked correctly, and more.
If an audio recording of the interview is available, it will be displayed in the questionnaire. FlowSurvey allows you to listen to the audio recording separately for each question.
Additionally, for quality control of the interview, you can use the GPS coordinates of the start and end of the survey to check if the interviewer was in the right place. However, it should be noted that GPS is not reliable enough and can be spoofed on Android devices.
You can also use the function of automatically moving to the next question when the audio recording of the current question is fully played.
Audio recordings can also be exported in a ZIP archive, also separately for each question.
If you determine that the current questionnaire is of low quality, you can set the questionnaire status to "Rejected." In this case, the questionnaire is excluded from the quota. Upon the next synchronization of devices, the quotas closed by this questionnaire will be reduced by 1 and available for re-interviewing. Thus, the interviewer will need to redo the questionnaire.
If the questionnaire is of sufficient quality, the status "Approved" is set. Further, questionnaires that have not been checked can be filtered by the status "Not verified." This way, the controller or the person checking the quality of the questionnaires can easily find unchecked questionnaires and assign them the required statuses.
Quota Filling Control
During the survey, it is necessary to constantly monitor the quota filling and respond if some quotas are not filling fast enough or if other problems arise.
For this purpose, there is a "Quota Statistics" table, which contains up-to-date information on the filling of all quotas and the overall totals. If the quota has been divided into several quota groups, there will be several separate tables.
The table displays the full structure of the nested locations and tasks, as well as intersecting and non-intersecting quotas (parallel quotas). In the previous image, you can see intersecting (strict) quotas: "age" in relation to "gender". You can also see non-intersecting (parallel) quotas: "presence of children" in relation to "gender" and "age".
In each cell, you can see how many interviews out of the required number have been completed.
At the bottom, you can see the total sum and percentage completion of the current table.
If you need to check the list of questionnaires for a specific quota, you can go to the questionnaire list page and set the filter to the desired location and quota.
These two tools (quota statistics and questionnaire list) are sufficient to effectively control quota filling.
Quota Overflow
Quota overflow for device surveys (offline) can occur in the following case:
- One task is assigned to two or more devices
- The device synchronized before another device from the same task sent questionnaires to the server
- Both devices together completed more questionnaires than required by the quota
If quota overflow occurs, the questionnaire that arrived at the server later will be marked with the "Out of quota" marker.
It is recommended to avoid quota overflow by assigning personal tasks. Otherwise, it is necessary to synchronize as often as possible so that the devices receive data on how many quotas are closed by other devices.
Technical Issues on Devices
During the survey, technical issues may arise on the devices. There are two cases:
- A technical issue actually occurred
- The interviewer did not understand the technical nuances and simply needs qualified technical support.
In both cases, you can contact our technical support, and we will help you. The application (if it can be launched) has a "Send logs" function (located in the main menu of the application), which allows you to send the application's logs to the FlowSurvey technical department. The logs contain a list of the interviewer's activity in the application and a list of errors. The logs will help our technical department determine:
- Whether there was really an error
- Find the source of the error, if any
- Answer your question, for example: find questionnaires that the interviewer considers "lost"
- And more
If there is indeed an error and the cause of the error is in the application or server, contacting technical support is free of charge.
Viewing Interim Results
During the project, there may be a need to view an interim report and provide a report to the client or interested party while restricting access to other parts of the survey.
For this purpose, FlowSurvey has a "Report". The report contains "live" data, and it is possible to give interested parties access only to the report. To do this, you can create a separate user, specify which projects they have access to, and set the "Report viewing" right while restricting all other rights.
You can also provide access to "Quota Statistics," and the interested party will be able to view quota filling and monitor the process.
Data Export
A large number of questionnaires means a large data set. For example, 10,000 questionnaires and 3,000 columns (each column corresponds to one data unit, one answer to a question) means an export of 30 million cells. Our software allows for the export of such a large data volume at once. For such a large volume, it is recommended to use the SPSS format, as the Excel format may have problems opening the file.
Conclusion
Thus, managing a large field can bring many problems; however, with proper planning and the use of appropriate software, this process can be easy, high-quality, and fast.
Our FlowSurvey software has been tested on large projects and has proven its effectiveness. It is appreciated by both managers and interviewers.
If you have any questions, please contact our specialists. And if you want to try the product, please register using this link.