User Manual

First use

  1. Use the adjustable camera arm to mount the camera approximately 18-30 inches (45-76 cm) above the table, facing mostly downward but slightly towards the front of the table. You will want it to be angled forward enough to fully capture the front of the table.

    The further away the camera is from the table, the larger the active area of the drum set can be. However, tracking accuracy may suffer. Some experimentation will be necessary to find the best camera position for your table. Also, avoid using shiny tables.

  2. Use masking tape (or something else) to mark where on the table you want the designated MIDI notes to be (image now directly from perspective of mounted camera):

  3. Make sure the camera is plugged in, and start 3d-beats.

    3d-beats should automatically connect to the camera, and start calibrating against the flat table. While it is calibrating, keep the active area free of arms and other items. The screen should look something like this:

    Once calibration completes, the screen will look like this:

    If the calibration didn't capture the full surface, or needs to be reset for any reason, click the 'reset' button to restart the calibration from scratch:

  4. On the screen, click on each masking-tape mark to register a MIDI note in each location:

    At this point, you should be able to play notes with your hands by tapping on the masking-tape marks on the table. The notes should light up on the screen, and your DAW should be receiving MIDI notes.

Overview of UI

A piece-by-piece breakdown of all the buttons:

  1. Settings
    • Pick which MIDI-out port to send notes to.
    • Re-calibrate active surface
    • Open this user manual
    Advanced settings:

    These options allow fine-tuning a few parameters of the hand detection algorithm. Beware, if you change these settings, your changes will still persist for future uses of the app, or until you click 'Reset all'
    • Note on threshold: How 'thick' your hand is: thicker fingers increase this number (170.0 means 1.70 cm.). If 3d-beats is missing some taps, consider increasing this number.
    • Note off threshold: How far your hand has to come up after tapping for the note to be turned off (10.0 means 0.1 cm). If 3d-beats is playing too many notes all at once when actually only playing one note.
    • Detect note strength: When disabled, all notes are played at 100% velocity. When enabled, use speed of hand to determine note velocity.
    • Min note strength: Minimum velocity percentage for a note when detecting note strength. Value of 1 is equivalent to disabling note strength detection.

  2. Virtual 3D camera controls
    Use the arrow keys or arrow buttons to control the camera that is rendering the simulated image on-screen. Zoom using the mouse wheel or the slider on the bottom.

  3. Framerate monitor
    The first graph plots frames per second (FPS), the second graph plots ms/frame. The RealSense camera input video runs at 90 FPS, so 3d-beats needs to process each frame in under 11ms for the best real-time performance.

    If you are seeing framerates significantly below 90 FPS, then the latency between when you play a note by tapping on the table and when you hear it come out of your speakers may be high enough to not feel real-time. Check out the Troubleshooting section below for possible remedies.

  4. Notes manager
    Use this section to change, and delete notes. Click on the rendered surface on the main screen to create new notes.

  5. Simulated 3D rendering
    Simulation of the active surface and any detected hands.
    • Click on the active surface to create new notes.
    • Watch the simulation just for fun and for troubleshooting.

Tapping technique

3d-beats hand tracking works best when you follow a few very simple techniques. It may take some time to learn how to play accurately and quickly (just like any musical instrument), but these tips will help:



Found a bug? Please report it and send any other feedback to info@3d-beats.com