Introduction to Prolific

class: center, middle, inverse, title-slide

# Introduction to Prolific
## online testing tutorial
### Yaxin Liu
### 07/28/2021

---

class: center, middle

### hyperlinks are clickable in this tutorial
You may also search the content

press letter `o` to toggle slides

---
class: center, middle

# Tools you will need

###  [Qualtrics](https://emorycollege.co1.qualtrics.com/) 
### [PsychoPy3 desktop + Pavlovia](https://pavlovia.org)
### [Prolific](https://prolific.co)

---
class: left, middle

# Goals of testing adults online

- Automate things to reduce errors

- An elegant work flow that makes life easier for your participants

- **Chaining with Prolific, Qualtrics, and Pavlovia**

---
class: inverse, center, middle
# Qualtrics basics

---
# Qualtrics

Obtain your **licenced** account from [Emory IT department](http://college.emory.edu/business-operations/information-technology/faqs.html):

You are recommended to use the [Qualtrics](https://emorycollege.co1.qualtrics.com/) to create consent and demographics forms <sup>1</sup>

- Create a new survey (consent form) from the menu `Create New Project -> Edit Question -> Question type -> Text/Graphic`<sup>2</sup>

- Click the `Question type` button to include a multiple choice question;

--
- Click the `Import from library` to import existing demographic questions

.footnote[
[1] You can use PsychoPy3 to create consent forms, but the UI is less ideal

[2] Per IRB guideline, include a departmental logo

]

---
# Qualtrics branching

- You will have two branches. One for participants who complete the survey, the other for participants who click "Do not consent".

- Create a End of study message 
`Messages Library -> New Message`

---
# sample consent form.can-edit[]

---
class: inverse, center, middle

# Pavlovia Basics

Assuming you already have an activated study uploaded to Pavlovia

---
# Passing in a participant ID to _Pavlovia_

You most likely have a main experiment for your participants. Participants'IDs should be send to Pavolovia.

1. Use the Embed field in Qualtrics; `add a new field`:  `prolific_ID` , `gender`, or more

```yaml
prolific_ID = ${q://QID107/ChoiceTextEntryValue}
gender = ${q://QID127751975/ChoiceGroup/SelectedChoices}
```
--

2.Add "End of message" in Qualtrics. Each participant gets a unique link;
```yaml
https://run.pavlovia.org/Yliu668/mrt_rtb21/html?participant= ${q://QID107/ChoiceTextEntryValue}&gender=${q://QID127751975/ChoiceGroup/SelectedChoices}
```
Basically you want to match your field code in Qualtrics with the field in Pavlovia

---
# sample study with chaining

```yaml
https://run.pavlovia.org/Yliu668/mrt_rtb21/html?participant= yaxin&gender=female
```

---
# Passing an ID from Prolific to Qualtrics

## Option 1

- Add a question in your Qualtrics form
- Ask participants to copy and paste their Prolific IDs

---
# Passing an ID from Prolific to Qualtrics

### Option 2 (recommended)

#### Prolific
- Select     **`I'll use URL parameters`** and click `configure URL parameters`

```yaml
https://emorycollege.co1.qualtrics.com/jfe/form/SV_5oRtfSVUgYc8yvc?PROLIFIC_auto_ID={{%PROLIFIC_PID%}}
```
#### Qualtrics
- Add .can-edit[query strings] to your question (`right click ->Default Choices -> Embedded Data Field`)

<img src="data:image/png;base64,#auto_ID.png" width="50%" style="display: block; margin: auto;" />
- Now your URL will be:

``` yaml 
https://emorycollege.co1.qualtrics.com/jfe/form/SV_5oRtfSVUgYc8yvc?PROLIFIC_auto_ID=5eb864599d66767bb037f277`
```

---
# Passing an ID from Prolific to Pavlovia

Since you will most likely to have a Qualtrics survey to begin with, we will skip the chaining with Prolific -> Pavlovia

Check out [this documentation](https://www.psychopy.org/online/prolificIntegration.html)

Linking to Pavlovia directly, use a URL like this one:

```yaml
https://pavlovia.org/run/Yliu668/mrt_ganis_rep/html/?participant={{%PROLIFIC_PID%}}&study_id={{%STUDY_ID%}}&Group={{%SESSION_ID%}}
```

---
# How to confirm participants have completed your study

- Copy the completion url from Prolific
- Make sure participants click this link at the **end** of your study

``` yaml
https://app.prolific.co/submissions/complete?cc=284E0002
```
---

# Examples

- if _Pavlovia_ is your final step, you can add it to the `Completed URL`

--
- You can also add to _Qualtrics_

---
# How do you send IDs from Pavlovia to Qualtircs?
If you have post-experiment questionnaires, chances are you need to re-direct your participants to Qualtrics again after their completion of the main experiment

<br>

In `Completed URL` dialog, 
you can save yourself time by using the sample code below:

```yaml
$"https://yourqualtricsURL?id="+expInfo['participant']
```

Just make sure you have the embedded data in your Qualtrics form. Here the **query string** is .can-edit[id].

---
# tips and tricks for _Prolific_  
- Add prescreening questions. For example:

- Gender
  - Vision
  - Language

--
<br>

- Set `Previous Studies screener` to exclude multiple attempts
<br>

- Approval by upload
  - I usually filter out participants who got correct attention checks in a csv file

---
# New feature: download demographic data

This is a sample demographic output

<div id="htmlwidget-b5072d0b7d6c24e98f7b" style="width:100%;height:auto;" class="datatables html-widget"></div>
<script type="application/json" data-for="htmlwidget-b5072d0b7d6c24e98f7b">{"x":{"filter":"none","fillContainer":false,"data":[["1","2","3","4","5"],["600f80b4af87","603f249dfcae","5dd5325d1c93","60df59d91d7f7","60sec714bee59"],["AWAITING REVIEW","AWAITING REVIEW","AWAITING REVIEW","AWAITING REVIEW","AWAITING REVIEW"],["05:03:00","18:26:00","31:56:00","41:03:00","42:00:00"],["32:30:00","03:12:00","58:00:00","59:28:00","59:33:00"],[1647.18,2685.484,1564.392,1104.965,1052.445],[26,20,34,23,27],[47,23,66,15,28],[2,0,0,0,0],[95,100,100,100,100],[null,null,null,null,null],["709148C1","709148C1","709148C1","709148C1","709148C1"],["United States","India","Hungary","United Kingdom","Canada"],["United States","Canada","United Kingdom","United Kingdom","Canada"],["Full-Time","Part-Time","Unemployed (and job seeking)","Other","Unemployed (and job seeking)"],["English","Hindi","Hungarian","English","English"],["Male","Male","Male","Male","Male"],["United States","Canada","Hungary","United Kingdom","Canada"],["Male","Male","Male","Male","Male"],["No","Yes","No","Yes","No"],["Yes","Yes","Yes","Yes","Yes"]],"container":"<table class=\"display\">\n  <thead>\n    <tr>\n      <th> <\/th>\n      <th>participant_id<\/th>\n      <th>status<\/th>\n      <th>started_datetime<\/th>\n      <th>completed_date_time<\/th>\n      <th>time_taken<\/th>\n      <th>age<\/th>\n      <th>num_approvals<\/th>\n      <th>num_rejections<\/th>\n      <th>prolific_score<\/th>\n      <th>reviewed_at_datetime<\/th>\n      <th>entered_code<\/th>\n      <th>Country of Birth<\/th>\n      <th>Current Country of Residence<\/th>\n      <th>Employment Status<\/th>\n      <th>First Language<\/th>\n      <th>Gender identity<\/th>\n      <th>Nationality<\/th>\n      <th>Sex<\/th>\n      <th>Student Status<\/th>\n      <th>Vision<\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"pageLength":20,"scrollY":600,"dom":"ti","odering":false,"columnDefs":[{"className":"dt-right","targets":[5,6,7,8,9]},{"orderable":false,"targets":0}],"order":[],"autoWidth":false,"orderClasses":false,"lengthMenu":[10,20,25,50,100]}},"evals":[],"jsHooks":[]}</script>

---
class: center, middle

# Thanks! Now you will be ready!
# More questions?