rPPG Heart Rate Monitor 💓

Real-time heart rate monitoring using rPPG (remote photoplethysmography) with MediaPipe face detection, scipy peak detection, and maximum stimulation mode featuring Subway Surfers.

Features

• 🫀 Non-contact heart rate measurement using camera and computer vision
• 👤 MediaPipe face detection for robust facial landmark tracking
• 📊 Scipy signal processing with bandpass filtering and peak detection
• 🎮 Maximum Stimulation Mode with Subway Surfers video at the bottom
• 📈 Real-time BPM display with visual feedback
• 🎨 Modern UI with animated heart icon and gradient backgrounds

How It Works

1. Face Detection: MediaPipe identifies facial landmarks in real-time
2. ROI Extraction: Forehead region is extracted as the region of interest
3. RGB Signal: Mean RGB values are collected over time (green channel is most sensitive)
4. Signal Processing:
   • Detrending to remove slow variations
   • Butterworth bandpass filter (0.7-4.0 Hz / 42-240 BPM)
   • Peak detection using scipy
5. BPM Calculation: Heart rate calculated from peak intervals

Requirements

Python Dependencies

Install Python 3.8+ with the following packages:

pip install opencv-python mediapipe scipy numpy

Or using conda:

conda install -c conda-forge opencv mediapipe scipy numpy

VS Code

• VS Code 1.107.0 or higher
• Webcam access

Installation

1. Clone or download this extension
2. Install Python dependencies (see above)
3. Open the extension folder in VS Code
4. Press F5 to run in debug mode

Usage

1. Open Command Palette (Ctrl+Shift+P / Cmd+Shift+P)
2. Run command: "rPPG: Start Heart Rate Monitor"
3. Click "Start Monitoring" button
4. Position your face in the camera view
5. Wait 5-10 seconds for the algorithm to stabilize
6. Your heart rate will appear in the top-right overlay

Tips for Accurate Readings

✅ DO:

• Sit still and face the camera directly
• Ensure good, even lighting on your face
• Wait at least 5 seconds for signal stabilization
• Keep your face in frame

❌ DON'T:

• Move your head rapidly
• Use in very dim or harsh lighting
• Cover your forehead
• Expect instant readings (signal processing needs time)

Technical Details

Signal Processing Pipeline

Camera Frame
    ↓
MediaPipe Face Detection
    ↓
Forehead ROI Extraction
    ↓
Mean RGB Signal (Green Channel)
    ↓
Detrending
    ↓
Butterworth Bandpass Filter (0.7-4.0 Hz)
    ↓
Peak Detection (scipy.signal.find_peaks)
    ↓
BPM Calculation (60 / median_peak_interval)

Configuration

Edit scripts/rppg_processor.py to adjust:

• BUFFER_SIZE = 150: Number of frames to buffer (5 seconds at 30fps)
• FPS = 30: Target frame rate
• LOWCUT = 0.7: Minimum heart rate (42 BPM)
• HIGHCUT = 4.0: Maximum heart rate (240 BPM)
• MIN_PEAK_DISTANCE: Minimum distance between peaks

Troubleshooting

"Python error: ModuleNotFoundError: No module named 'mediapipe'"

• Install missing dependencies: pip install mediapipe scipy

"Failed to spawn Python process"

• Verify Python is in PATH: python --version or python3 --version
• On Windows, try py -3 --version

"No Face Detected"

• Ensure good lighting
• Face the camera directly
• Check camera permissions

BPM reads as 0 or shows no value

• Wait 5-10 seconds for signal stabilization
• Ensure face is detected (green landmarks visible)
• Check that forehead is visible and well-lit

Known Limitations

• Requires good lighting conditions
• Accuracy varies with skin tone and lighting
• Not a medical device - for educational purposes only
• Requires ~5 seconds of stable face detection before readings
• Motion artifacts can affect accuracy

Future Plans

JavaScript/Browser-Native Pipeline

Currently, the rPPG processing relies on a Python backend with OpenCV, MediaPipe, and scipy. Future development will include a pure JavaScript/TypeScript implementation to eliminate the Python dependency:

• Face Detection: Migrate to TensorFlow.js with MediaPipe BlazeFace or Face Landmarks models
• Numerical Processing: Replace NumPy with typed arrays and math.js or custom DSP utilities
• Signal Processing: Implement Butterworth bandpass filter and peak detection in JS (e.g., using ml-signal-processing or custom FFT/filter implementations)
• Benefits:
  • No Python installation required
  • Fully sandboxed in the webview
  • Easier distribution and cross-platform compatibility
  • Faster startup time

Target libraries/approaches:

• @tensorflow/tfjs + @tensorflow-models/face-landmarks-detection
• ndarray / scijs for array operations
• Custom DSP functions for filtering and peak detection

Contributions welcome!

Credits

Original concept and Subway Surfers integration inspired by Surya Dantuluri.

Special thanks to the rPPG research community and open-source contributors for MediaPipe, scipy, and OpenCV.

Disclaimer

⚠️ This extension is for educational and entertainment purposes only. It is NOT a medical device and should NOT be used for medical diagnosis or treatment. Always consult healthcare professionals for medical advice.

Enjoy!