corrscope

Corrscope Help

Homepage at https://github.com/corrscope/corrscope. Report bugs at https://github.com/corrscope/corrscope/issues/ or https://discord.gg/CCJZCjc.

Corrscope is named because it cross-correlates the input wave and a history buffer, to maximize wave alignment between frames.

Tutorial

Screenshot of Corrscope and video preview

Start by adding channels to be visualized: look at the bottom-right table and click the “Add…” button.

To add audio to play in the background, look at the top-right “FFmpeg Options” and click the Master Audio “Browse…” button.

To make the waves taller, go to the left panel’s General tab and edit Amplification. Afterwards, click the Appearance tab and customize the appearance of the oscilloscope. (Note that colored lines will be discolored and blurred by Youtube’s chroma subsampling.)

Click Preview to launch a live preview of the oscilloscope with audio.

Click Render to render the oscilloscope to video.

Click Save to save the current project configuration to a file. These project files can loaded in corrscope, previewed or rendered from the command line, or shared with corrscope’s author when reporting issues.

Configuring the Trigger

Unlike other triggering algorithms (which only depend on the current frame’s worth of data), Corrscope uses a correlation trigger, which remembers the waveforms of past frames to help align future frames.

Corrscope’s triggering algorithm is configurable, allowing it to track many types of waves well. However it may be intimidating to newcomers. This will provide several types of waves, along with suggestions for how to tune trigger options.

Triggering options are found on the left panel. Trigger Width is located in the General tab. All other options are found on the Trigger tab. (Per-channel triggering options are found in the table.)

Sampled Trumpets and Trigger Direction (screenshot from Tales of Phantasia)

Screenshot of trumpets in corrscope

Sampled trumpets generally consist of a sharp falling edge, followed by gibberish with one or more rising edges.

Complex Waves and Trigger Direction (screenshot from Midori Mizuno - Sinkhole)

Screenshot of complex wave in corrscope

Corrscope’s standard “edge trigger” does not look for “steep edges” but instead “sign changes”. It operates by maximizing (signed area in right half) - (signed area in left half). This waveform has a clear falling edge from positive to negative, but no clear edge from negative to positive.

Either:

NES Triangle Waves

NES triangle waves are stair-stepped. In theory, Area Trigger would work and properly locate the best zero-crossing on each frame. However, on every frame, corrscope looks at a different portion of the triangle wave, computes the average value (DC offset), and subtracts it from all samples. Unfortunately since the exact amount of DC (positive or negative) fluctuates between frames, corrscope will shift the wave vertically by different amounts, causing it to jump between different rising edges.

Try the following:

NES triangle waves have 15 rising/falling edges. The NES high-pass removes DC and low frequencies, causing waveforms to decay towards y=0. As a result, “which edge crosses y=0” changes with pitch.

FDS FM Waves

FDS FM changes the width of waves, but not their height.

The NES high-pass removes DC and low frequencies, causing waveforms to decay towards y=0. If FDS waves contain anything other than pulse/saw, “which part of the wave crosses y=0” may change with FM and pitch.

Yamaha FM and SNES/sampled Waves

Options

All tabs are located in the left pane.

Variables Remembered

Obtaining Data (each frame)

On each frame, corrscope fetches [from the channel] a buffer of mono data, centered at the current time. The amount of data used is controlled by Trigger Width, which should be increased to keep low bass stable.

Sign Triggering

Some waves do not have clear edges. For example, triangle waves do not have clear rising edges (leading to suboptimal triggering), and NES triangles have 15 small rising edges, causing corrscope to jump between them.

If Sign Strength (Sign Triggering on the GUI) is set to nonzero strength, corrscope computes peak = max(abs(data)). It adds peak * strength to positive parts of data, subtracts peak * strength from negative parts of data, and heavily amplifies parts of the wave near zero. This helps the correlation trigger locate zero-crossings exactly.

Mean/Period

To remove DC offset from the wave, corrscope calculates the mean of input data and subtracts this averaged mean from data.

Corrscope then estimates the fundamental period of the waveform, using autocorrelation.

Corrscope multiplies data by data window to taper off the edges towards zero, and avoid using data over 1 frame old.

(optional) Pitch Tracking

If Pitch Tracking is enabled:

If period changes significantly:

Pitch Tracking may get confused when data moves from 1 note to another over the course of multiple frames. If the right half of buffer changes to a new note while the left half is still latched onto the old note, the next frame will latch onto the mistriggered right half of the buffer. To prevent issues, you should consider reducing Buffer Responsiveness (so buffer will not “learn” the wrong pitch, and instead be rescaled to align with the new note).

Correlation Triggering (uses buffer)

Corrscope cross-correlates data with (Buffer Strength * buffer) + (Edge Strength * edge_finder) + (Slope Strength * slope_finder) to produce a score for each possible data triggering location. Locations which line up well with the complex expression (line up well with the previous frame, transition from negative to positive, or increase in value) have high scores. Corrscope then picks the location in data with the highest score as the position to be used for rendering.

(Optional) Post Triggering

If post triggering is enabled:

Zero Crossing Trigger

Setting Post Trigger to “Zero Crossing Trigger” causes corrscope to “slide” towards edges. The maximum distance per frame is determined by GUI Post Trigger Radius (in samples).

Updating Buffer

Video Encoding

Corrscope uses FFmpeg to encode videos. All video encoding settings (both picking an encoder and options) are configured in Corrscope’s “Video Template” textbox, which is passed to FFmpeg. By default, it tells FFmpeg to use the x264 video encoder (producing H.264 videos). Tuning video encoders like x264 is a complex task, but this is a brief summary of the default settings:

Videos are first converted from RGB pixel values to YUV (brightness and color).

Afterwards, the video is sent to the video encoder, which has its own arguments:

Avoiding Color Artifacts

Video encoding/compression degrades color more than brightness, especially fine color detail. As a result, thin colored lines look desaturated, fuzzy, or discolored. (Thin lines generally arise when rendering at a low resolution, or when YouTube takes a high-resolution video with thick lines and reencodes lower-resolution streams with thinner lines.) To avoid this loss of quality, corrscope defaults to white lines on a black background.

Loss of color information is especially damaging with “Color Lines By Pitch” enabled. At the default settings (720p, 1.5 pixel thick lines), the vibrant line colors seen in the preview lose saturation when rendered, and turn into grayish messes when uploaded to YouTube (blues and purples lose the most color).

To render colored lines while minimizing quality loss, render at a higher resolution (slower) with thicker lines. This will improve color fidelity for people who watch the resulting videos above 720p.

I do not have experience with other encoders (like x265, VP8, VP9, or AV1), but the principle of losing fine color detail to chroma subsampling and lossy codecs should remain the same. AV1 should preserve colored lines better due to chroma-from-luma, but AV1 encoders are still impractically slow.

Audio Encoding

corrscope defaults to rendering to .mp4 files, which support a limited set of audio codecs. MP3 is a “good enough” audio codec. AAC is better in theory, but ffmpeg’s AAC encoder (which corrscope uses by default, -c:a aac) is bad. However, corrscope defaults to 384 kilobits/sec (-b:a 384k), which should be sufficient to produce audio without obvious artifacts.

In the future, I may enable VBR encoding, or switch to an audio codec not supported in .mp4 files, like Vorbis or Opus (which has better quality at any bitrate than even good AAC encoders). This would requires switching to a different file format like .mkv.