PsyToolkit Debugger

Hi all!

This extension reads PsyToolkit code, identifies potential syntax errors, and simulates experiment execution to generate sample Excel data sheets. Its primary purpose is to help researchers identify problems in their experimental code without having to run through an entire experiment manually.

IMPORTANT: This extension does NOT generate valid participant data. All simulated responses and reaction times are synthetic and must never be used as actual experimental observations. We already have a replication crisis. Let's not make it worse.

Features

Command Overview

Run Debugger

Runs the active .psy file using your configured trial limit. It simulates keypresses and reaction times and exports the results to Excel.

For inputs that should not be automatically generated (such as participant ID or age), the debugger will prompt you to provide a value.

Run Full Experiment

Runs the active script's full trial schedule, bypassing your configured trial limit. Text and choice prompts remain interactive.

Run Fuzzer

Runs the full trial schedule and automatically generates seeded responses for text, choice, and unresolved input prompts. It may ask you to locate a missing trial table.

Debugger Settings

Configure your default trial limit and determine how the debugger handles skipped input prompts.

If no trial limit is specified, the debugger follows the experiment's full trial schedule.

Set Random Seed

Control how random seeds are generated and reused. Three modes are available:

• Generate a new seed for every run.
• Use a fixed seed for every run.
• Generate a seed on the first run and reuse it for subsequent runs.

Rerun Last Seed

Reruns the previous script using its last random seed and execution mode.

For interactive runs, you must provide the same answers to reproduce the same results.

How to Use

Open the VS Code Command Palette using the top menu.

Enter > to display available commands.

Type PsyToolkit after the > to filter for commands provided by this extension.

Select the command you want to run.

Additional Notes

• Each run generates an Excel report.
• The Trials sheet contains one row per simulated trial.
• The Saved Data sheet contains one row per executed save statement.
• This extension was primarily developed and tested using visual search experiments. Other experimental paradigms may not be fully supported.
• The extension's design and feature requirements were developed by a human, with substantial AI assistance used during implementation.

Please verify the debugger's output against the expected behavior of your experiment. A successful simulation does not guarantee that the original PsyToolkit script will execute correctly or produce valid experimental data.

Known Issues

If an input prompt is canceled while automatic blank-value insertion is enabled, the debugger may still generate an Excel file even when a required variable cannot be left blank.

Release Notes

1.0.0

Initial release of the PsyToolkit Debugger.

Acknowledgment Request

If this extension has been useful in developing or debugging your research experiments, an acknowledgment in any resulting publications would be greatly appreciated, but is entirely optional.

Suggested acknowledgment:

The authors used the PsyToolkit Debugger, developed by Sreyas Chukkapalli, to assist with debugging experimental code.

If you choose to acknowledge the extension, I'd love to hear about it! Feel free to send me an email at the address below with a link or copy of your publication.

No acknowledgment or citation is required to use this extension. I'm just hoping to document its impact on academic research, particularly as I prepare for future graduate school applications.

Contact Information

If you encounter any issues, discover bugs, or have suggestions for improvements, feel free to reach out:

psytoolkitdebug@gmail.com