A Methodology for Evaluating Algorithms That Calculate Social Influence in Complex Social Networks
Table 6
Pre-questionnaire (face validity) given to nonexperts.
Instructions. Before beginning, pull out a piece of paper and neatly write down 8 Facebook friends that first come to your mind. After doing so, assign each friend a letter—A, B, C, D, E, F, G, and H—and proceed to complete the questionnaire. This questionnaire contains 10 questions. By participating, you are supporting the final phases of Vanja Smailović’s PhD research. Every question requires reading an imaginary Facebook post. For each of the questions, answers are offered as PAIRS of your Facebook friends. Refer to your annotations A–H above. All of the questions have identical answers (Facebook-friend pairs). In other words, you always get to choose between the same Facebook-friends—it is the questions that changes and differs, not the answers.
Question text
Available answers
Passed
Chosen
You are dissatisfied with your mobile operator. You’ve noticed a post “I’ve moved to my new telco X, I think they are better.” You would more likely change your mobile operator if it were posted by your Facebook friend: _______ or _______
A or H B or G C or F D or E
3.32
✓
✓✓
You’ve noticed a post “World leaders at their last meeting decided to increase nuclear armament.” You would search for more details if it were posted by your Facebook friend: _______ or _______
A or H B or G C or F D or E
3.73
✓
✓✓
You’ve noticed a post “I’m calling everyone to join a public protest against getting rid of future generation pensions.” You would more likely join this protest if it were posted by your Facebook friend: _______ or _______
A or H B or G C or F D or E
3.45
✓
You’ve noticed a post that explains the proven downside of your preferred political party. You would more likely change your vote if it were posted by your Facebook friend: _______ or _______
A or H B or G C or F D or E
3.86
✓
✓✓
You’ve noticed a post “Quickly pay your monthly bills, otherwise fines follow within 24 hours according to latest news.” Reading the post would to a greater extent cause restlessness if it were posted by your Facebook friend: _______ or _______
A or H B or G C or F D or E
3.55
✓
You’ve noticed a motivational post with thoughts about a brighter future, more jobs and possibilities, and greater salaries where you live. Reading the post would more likely calm you down if it were posted by your Facebook friend: _______ or ______
A or H B or G C or F D or E
3.59
✓
✓✓
You’ve noticed a post “Tensions between Balkan EU members might lead to war.” Reading the post would more likely cause restlessness if it were posted by your Facebook friend: _______ or _______
A or H B or G C or F D or E
3.77
✓
✓✓
You’ve noticed a post “Immigrants in EU constantly on the rise.” Reading the post would more likely spark an interest if it were posted by your Facebook friend: _______ or _______
A or H B or G C or F D or E
3.45
✓
You’ve noticed a motivational post about exercise, more physical activity, and health benefits. Reading the post would more likely motivate you if it were posted by your Facebook friend: _______ or _______
A or H B or G C or F D or E
3.45
✓
You are planning on seeing the movie X. You’ve noticed a post “I’ve seen X, it’s horrible.” Reading the post would more likely dissuade you from watching the film if it were posted by your Facebook friend: _______ or _______
A or H B or G C or F D or E
3.55
✓
✓✓
You have reached the final question. It is unique and very important. You will reply to it in the same manner as you replied to the questions earlier. Once more, you will choose a Facebook friend from the 4 pairs offered in answers—only this time, pay attention to the definitions of social influence below: - social influence is a measure of how people, directly or indirectly, affect the thoughts, feelings, and actions of others; - social influence is the ability to drive action; and - social influence occurs when one’s emotions, opinions, and behaviours are affected by others. Read the definitions of social influence above. For each of the pairs, choose the Facebook friend whom you consider has the GREATER social influence on you on Facebook. While doing so, try to encompass all 3 definitions above as best as you can.
