Optical Spectrum Analyzers: Operating Principles and Measurement Configuration

Scope

This article describes the operating principles, key specifications, and standard measurement configurations of optical spectrum analyzers (OSAs). Coverage includes the three dominant OSA architectures (diffraction grating, Fabry–Pérot, Fourier-transform), the implications of resolution bandwidth and dynamic range for common measurements, and the standard configurations for laser source characterization, optical signal-to-noise ratio (OSNR) measurement, and EDFA gain characterization. OSA selection guidance for specific application classes is included; detailed product comparison across vendors is outside scope.

What an OSA measures

An optical spectrum analyzer measures the optical power spectral density $S(\lambda)$ of an input optical signal as a function of wavelength. The measurement output is power per unit wavelength bandwidth, conventionally displayed in dBm per nm or dBm per pm.

The instrument differs from a Fabry–Pérot wavemeter (which measures absolute wavelength of a single-line source with high precision but provides no power information) and from an electronic spectrum analyzer (which measures the electrical spectrum after photodetection, recovering only the relative frequency offsets within the detector bandwidth, not the absolute optical wavelength).

Standard OSA measurements include:

Operating principles

Three OSA architectures dominate, each with characteristic strengths.

Diffraction grating OSA

The dominant commercial architecture. Light from the input fiber is collimated, dispersed by a diffraction grating, and re-focused through a tunable slit onto a photodetector. The grating-and-slit assembly is rotated to scan wavelength.

Resolution is set by the slit width and the grating dispersion, typically 0.01–2 nm (10 pm to 2 nm). Wavelength range covers 600–1700 nm for telecom-band instruments and extends further for specialty IR instruments.

Single-pass and double-pass configurations exist; double-pass instruments (which return the dispersed light through the grating a second time) achieve higher resolution and better stray-light rejection at the cost of throughput.

The Yokogawa AQ6370 series, Anritsu MS9740, and the Keysight 86142B are representative high-end diffraction grating OSAs.

Fabry–Pérot OSA

A tunable Fabry–Pérot etalon serves as the wavelength-selective element. The free spectral range (FSR) of the etalon limits the unambiguous wavelength range; aliasing requires an additional bandpass filter to identify the operating order.

Resolution can be sub-pm — far finer than diffraction-grating OSAs — but typical FSR is only a few nm. Used for high-resolution measurements within a known narrow band, such as DFB linewidth measurement or resolved structure within a single ITU channel.

The Thorlabs OSA20x series and the Bristol Instruments 771 series are representative FP OSAs.

Fourier-transform OSA

A scanning Michelson interferometer with the unknown source as the input. The Fourier transform of the time-domain interferogram gives the spectrum. Wavelength accuracy depends on the reference wavelength used to calibrate the optical path delay (typically a stabilized HeNe laser at 632.99 nm).

Resolution depends on the maximum optical path delay; for a scanning interferometer with $\pm 10$ cm travel, resolution is approximately 5 pm.

FT-OSAs excel at absolute wavelength accuracy (sub-pm against internal reference) but are slower than grating-based instruments and have more complex calibration. The Thorlabs OSA series and HP 86120 series are representative.

Key specifications

The following specifications determine which measurements an OSA can perform.

Resolution bandwidth (RBW)

The narrowest wavelength interval the OSA can resolve, set by the slit width and grating dispersion (diffraction grating) or interferometer geometry (FP, FT). Smaller RBW = finer resolution.

Standard RBW values for diffraction grating OSAs: 0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0 nm. The lower RBW values trade signal-to-noise for resolution — a 0.01 nm RBW measurement on a broadband source delivers $\sim 20$ dB less signal than a 1 nm RBW measurement, requiring proportionally longer averaging.

For measurements where peak position matters (single-line lasers), RBW should be comparable to or smaller than the line width. For broadband measurements (LEDs, ASE sources), larger RBW improves SNR without losing meaningful information.

Dynamic range

The ratio of the maximum measurable signal to the noise floor, expressed in dB. Two distinct meanings:

Total dynamic range. Maximum signal level minus noise floor with full averaging. Typical: 60–80 dB.

Dynamic range at offset. The minimum-detectable signal at a given wavelength offset from a strong signal at another wavelength. Limited by the stray-light response of the grating and slit. Typical: 50–60 dB at 0.5 nm offset, 60–70 dB at 1 nm offset.

For SMSR measurements on DFB lasers (where the side modes are 30+ dB below the main mode and offset by $\sim 1$ nm), the at-offset dynamic range is the relevant specification, not the total dynamic range.

Wavelength accuracy

Absolute uncertainty of the displayed wavelength. Typical: $\pm 10$ pm for entry-level instruments, $\pm 1$ pm for high-precision instruments with built-in wavelength references.

Wavelength accuracy is maintained by periodic calibration against a known reference (gas absorption cell, atomic transition, or stabilized reference laser). High-precision OSAs include internal wavelength references and apply continuous calibration during sweeps.

Wavelength repeatability

The repeatability of the measured wavelength between sweeps, typically much better than the absolute accuracy. Important for time-resolved measurements where the same source is repeatedly measured.

Polarization-dependent response

The variation in measured power as input polarization changes. Specified as polarization-dependent loss (PDL). Typical: 0.05–0.5 dB for diffraction grating instruments. Polarization-diversified OSAs (using polarization beam splitters internally) achieve $<0.05$ dB.

For polarized sources (most lasers), polarization scrambling at the input or a polarization-insensitive OSA is required for accurate absolute power measurements.

Sweep speed

Time required to acquire a full-range spectrum at a given RBW. Trades against noise floor. Typical: 100 ms to 30 s depending on RBW and span. Critical for measurements requiring temporal resolution or for high-throughput production testing.

Standard measurements

Single-laser characterization

The basic OSA measurement: connect the laser output to the OSA input via SMF, set the span to bracket the expected emission wavelength, choose RBW based on the laser type.

Extracted parameters depend on the source type. For a DFB: peak wavelength, FWHM, SMSR, total integrated power. For a Fabry–Pérot: peak wavelength, peak spacing (= FSR), number of resolved modes, integrated power. For a broadband source: spectral peak, FWHM, integrated power, spectral shape.

Side-mode suppression ratio

SMSR is the ratio (in dB) of main-mode power to the most powerful side mode. For a single-mode DFB, SMSR is the primary metric of mode purity:

$$\text{SMSR} \;=\; 10 \log_{10}\!\left(\frac{P_\text{main}}{P_\text{side, max}}\right) \text{ [dB]}.$$

Typical telecom DFB requirements: SMSR $\geq 35$ dB. High-grade DFBs reach 50+ dB.

The SMSR measurement is dynamic-range-limited if the OSA's stray-light floor is above the actual side mode. For instruments with 50 dB at-offset dynamic range, a true 50+ dB SMSR cannot be measured directly — only the instrument floor is observed. Improving this requires either a higher-dynamic-range instrument or a tunable bandpass filter to suppress the main mode and measure the side mode separately.

Optical signal-to-noise ratio (OSNR)

OSNR is the ratio of signal power to the amplified spontaneous emission (ASE) noise floor within a reference bandwidth (typically 0.1 nm). For a single DWDM channel:

$$\text{OSNR} \;=\; 10 \log_{10}\!\left(\frac{P_\text{signal}}{P_\text{ASE in 0.1 nm at } \lambda_0}\right) \text{ [dB]}.$$

The ASE level is measured between channels where the signal is absent; the in-channel ASE is interpolated. OSA RBW must be smaller than the inter-channel spacing for valid OSNR measurement. For 50 GHz channel spacing (0.4 nm), RBW $\leq 0.1$ nm is required.

OSNR is the primary metric of optical link quality in WDM systems. Telecom transmitter specifications typically require OSNR $\geq 30$ dB at the launch.

EDFA gain characterization

For an erbium-doped fiber amplifier (EDFA), the wavelength-dependent gain is measured as the ratio of output spectrum to input spectrum:

$$G(\lambda) \;=\; \frac{P_\text{out}(\lambda)}{P_\text{in}(\lambda)} \text{ [dB]}.$$

The measurement uses a broadband input source (typically the EDFA's own ASE source operating without gain, or a separate ASE source), with the OSA capturing both input and output spectra.

EDFA gain is wavelength-dependent across the C-band (1530–1565 nm) and L-band (1565–1625 nm). A typical C-band EDFA exhibits 20–30 dB peak gain with 2–4 dB gain variation across the band. Gain flatness over a specified bandwidth is a key specification for DWDM applications.

Sources of error

RBW too large for the feature being measured. A 1 nm RBW measurement on a 0.1 nm-wide DFB peak underestimates the peak height and overestimates the width. Match RBW to the feature scale.

Polarization-dependent loss. Source polarization affects measured power for non-polarization-diversified OSAs. For absolute power measurements, polarization scrambling at the input or a polarization-insensitive OSA is required.

Detector saturation. Strong signals (above $\sim +10$ dBm at the input) saturate the OSA's internal detector, producing distorted spectra. Attenuate strong sources with a calibrated attenuator before the OSA.

Stray light at offset. Strong signals produce a stray-light floor in the OSA that exceeds the instrument's nominal noise floor at small wavelength offsets. SMSR and OSNR measurements at high contrast levels are limited by this effect.

Connector loss not in calibration. Each fiber connection has 0.1–0.3 dB of insertion loss. For absolute power measurements, the connection loss between source and OSA must be calibrated separately and subtracted.

Modulation sidebands mistaken for source structure. For modulated sources (datacom transmitters running real data patterns), the modulation sidebands appear at the OSA. The measured spectrum is then the convolution of the source spectrum with the modulation spectrum, not the unmodulated source spectrum.

Wavelength calibration drift. OSA wavelength accuracy degrades over time and with temperature. For applications requiring $\leq 10$ pm wavelength accuracy, periodic calibration against an internal or external reference is required.

Validation

For a new OSA setup, the following checks verify proper operation:

• A known reference source (DFB at known $\lambda_0$, gas absorption cell) at the input should produce the expected wavelength and power values within the instrument's specifications.
• Total integrated power across the spectrum should match the absolute power measured with a calibrated power meter on the same input.
• Wavelength accuracy verified against a wavelength reference (typically the internal reference in a high-end OSA, or an external HCN/H₂O absorption cell).
• Noise floor verified by capturing a spectrum with no input — the displayed floor should match the instrument's specification within $\pm 1$ dB.

References

For the comprehensive operating-principles reference on diffraction grating OSAs, see the Yokogawa application note AN-029-00 on OSA fundamentals. For FT-OSA principles and the role of the internal reference laser, see the Bomem/ABB application note on FTIR-based optical spectrum analysis. For OSNR measurement standards in DWDM systems, see ITU-T Recommendation G.697.