A Methodology for Evaluating Algorithms That Calculate Social Influence in Complex Social Networks
Table 5
Pre-questionnaire (content validity) given to experts.
Instructions. The pre-questionnaire contains 30 questions which you need to score 1 to 5. The goal is to explore which questions are most suitable in determining social influence. A very suitable question is given a score of 5; the least suitable is given a 1. Best-scored questions in this pre-questionnaire will be used in compiling a new questionnaire which will subsequently be forwarded to participants. The scores you provide directly affect the process of screening for the most suitable questions. By participating in this pre-questionnaire as an expert, you are providing support to the final phase of Vanja Smailović’s PhD research. It is important to understand the meaning of social influence. While scoring the questions, keep these definitions in mind at all times: - Social influence is a measure of how people, directly or indirectly, affect the thoughts, feelings, and actions of others; - social influence is the ability to drive action; and - social influence occurs when a person’s emotions, opinions, and behaviours are affected by other persons. The newly compiled questionnaire (using some of the questions below) will be given to nonexperts who initially will not be aware of its purpose (surveying social influence). They, that is, the nonexperts, will have 4 pairs of their Facebook friends offered as answers, unlike the scale of 1 to 5 noted here. In other words, each nonexpert will choose the more influential friend in a pair—without being directly asked about their respective social influence. Your task is to score the given “criterion” in each of the questions, with respect to how much it conforms to the definitions of social influence (given above). The pre-questionnaire, unlike the next questionnaire, does not show you Facebook-friend pairs as answers, because they will be tailored and specific for each nonexpert. Your role as an expert is to focus on the questions, not the answers given to future nonexpert participants.
Question text
Available answers
Passed
Chosen
You’ve noticed a post “Dangerous levels of chlorine detected in our hot water used for showering.” A greater impression on you would leave a post by your Facebook friend: _______ or _______.
1 2 3 4 5
2.95
✓
You’ve noticed a post “If every one of us recycled, we would have CO2 emissions and receive state/country stimulus for it.” You would recycle more frequently if it were posted by your Facebook friend: _______ or _______ .
1 2 3 4 5
3.36
✓
You’ve noticed a post “Disaster has struck, the Nepalese are left with no food, water and electricity. I’ve donated money, here are instructions for you to do the same.” You would donate a greater amount if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
3.05
✓
You are dissatisfied with your mobile operator. You’ve noticed a post “I’ve moved to my new telco X, I think they are better.” You would more likely change your mobile operator if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
3.59
✓
✓✓
You’ve noticed a post “Gas station X has the best fuel.” You would more likely refuel more frequently at the mentioned gas station if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
3.36
✓
You’ve noticed a post “Smoking while pregnant greatly increases chances of health issues in a child.” You would more likely spread this information if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
2.55
✓
You’ve noticed a post “Electricity bills will soon go up.” You would search for more details if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
3.50
✓
You’ve noticed a post “Video out showing New Zealand’s prime minister slipping on a banana.” You would more likely watch the video if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
2.82
✓
You’ve noticed a post “Hidden camera caught them in adultery.” You would watch the video to a greater length if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
2.09
✘
You’ve noticed a post “World leaders at their last meeting decided to increase nuclear armament.” You would search for more details if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
3.68
✓
✓✓
You’ve noticed a post “Toothpaste X discounted in all shopping malls.” You more likely go and buy the toothpaste if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
2.41
✓
You’ve noticed a post “Travel agency X offers phenomenal discounts for Asia.” You would search for more details if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
2.86
✓
You’ve noticed a post “Concert tickets to see X selling out soon.” You would search for more details if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
2.91
✓
You’ve noticed a post “I’m calling everyone to join a public protest against getting rid of future generation pensions.” You would more likely join this protest if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
4.0
✓
✓✓
You’ve noticed a post that explains the proven downside of your preferred political party. You would more likely change your vote if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
4.27
✓
✓✓
You’ve noticed a post “People killed in a terrorist attack in Ireland.” You would search for more details if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
3.23
✓
You’ve noticed a post “Home visits by National TV bill collectors more frequent in the following month.” You would less likely open your doors to strangers if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
3.14
✓
You’ve noticed a post “Quickly pay your monthly bills, otherwise fines follow within 24 hours according to latest news.” Reading the post would to a greater extent cause restlessness if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
3.77
✓
✓✓
You’ve noticed a motivational post with thoughts about a brighter future, more jobs and possibilities, and greater salaries where you live. Reading the post would more likely cause peacefulness in you if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
4.0
✓
✓✓
You’ve noticed a post “Tensions between Balkan EU members might lead to war.” Reading the post would more likely cause restlessness if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
3.64
✓
✓✓
You’ve noticed a post “Immigrants in EU constantly on the rise.” Reading the post would more likely spark an interest if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
3.91
✓
✓✓
You’ve noticed a post “Parliament representatives physically confront each other at the morning sitting.” Reading the post would more likely surprise you if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
2.68
✓
You’ve noticed a motivational post about exercise, more physical activity, and health benefits. Reading the post would more likely motivate you if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
3.95
✓
✓✓
You are planning on seeing the movie X. You’ve noticed a post “I’ve seen X, it’s horrible.” Reading the post would more likely dissuade you from watching the film if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
4.41
✓
✓✓
You are planning a trip to a neighboring country/state X, it’s snowing outside. You’ve noticed a post “X’s police officers fine drivers without winter tires.” Reading the post would more likely persuade you to buy winter tires if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
3.23
✓
You are driving on X section of road. You’ve noticed a post “Police radar-tracking speed at section X.” Reading the post would more likely cause you to obey the speed limit if it were posted by your Facebook friend: _______ or _______”
1 2 3 4 5
3.0
✓
You are driving on the road section X. You’ve noticed a post “Terrible traffic accident at section X.” Reading the post would more likely disturb you if it were posted by your Facebook friend: _______ or _______”
1 2 3 4 5
2.64
✓
You are planning a trip to city X. You’ve noticed a post “City X caught in bad weather.” Reading the post would more likely worry/disappoint you if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
2.27
✓
You are planning a trip to city X. You’ve noticed a post “City X has seen a rise in crime-rates in recent years.” Reading the post would more likely worry you if it were posted by your Facebook friend: _______ or _______
1 2 3 4 5
3.18
✓
You are planning a trip to city X. You’ve noticed a post “City X has seen a rise in crime-rates in recent years.” Reading the post would more likely persuade you to re-plan the trip if it were posted by your Facebook friend: _______ or _______
