Systematic Information Gathering with AI: Beyond Manual Collection

The Promise of Structured Data Collection Information gathering across large, distributed sources has long been challenging to systematize effectively. Whether you're mapping expertise, compiling directories, or surveying landscapes in any domain, traditional automated approaches often struggle with inconsistent data extraction, parsing errors, and poor data quality. Recent advances in AI…Read More

Longitudinal Surveys in Qualtrics

We’ll be using embedded data, authenticators, directories, and workflows to carry individual participant data from Prolific to Qualtrics, and from one Qualtrics survey to the next. This example will use Prolific to provide the participant ID, other panel services should work similarly. Initial opt-in survey Qualtrics surveys can read and…Read More

Protect Qualtrics Surveys from Bots

If you collect behavioral or market research observations online using Qualtrics surveys, data quality is a serious concern. Unlike in-person labs, paid data collection online is highly susceptible to fraud. In this post, we will revisit established best practices and adapt them to recent threats posed by Large Language Models…Read More

Using OpenAI Batch API

This tutorial demonstrates how to use the OpenAI API's batch endpoint to process multiple tasks efficiently, achieving a 50% cost savings with guaranteed results within 24 hours. The service is ideal for processing jobs that don't require immediate responses. Eligible researchers  can email research-programming@wharton.upenn.edu to start the process of obtaining…Read More