Mode Solvers
Author: the photonics expert Dr. Rüdiger Paschotta (RP)
Definition: algorithms and software for calculating modes of waveguide structures such as fibers
Category:
- mode solvers
- scalar mode solvers
- full-vector mode solvers
- semi-vectorial mode solvers
Related: modeswaveguidesfibersnumerical beam propagationfiber simulation software
Page views in 12 months: 451
DOI: 10.61835/q47 Cite the article: BibTex BibLaTex plain textHTML Link to this page! LinkedIn
Content quality and neutrality are maintained according to our editorial policy.
What are Mode Solvers?
Mode solvers are algorithms and fundamental computational software tools in photonics that calculate the electromagnetic field distributions and propagation characteristics of optical modes in waveguide structures. For each mode, computed for a given optical frequency or wavelength, they provide various results such as:
- transverse amplitude profile (one or several field components, in the latter case defining the polarization of light)
- optical intensity profile
- effective mode area
- fraction of optical power propagating in the core
- effective refractive index
- propagation constant
- propagation losses
- group index and group velocity dispersion (calculated from frequency derivatives)
- cut-off wavelength
- bend losses (for a given bend radius)
Mode solvers may be applied to different kinds of waveguides:
- standard all-glass optical fibers
- photonic crystal fibers and other types of photonic crystals
- waveguides in photonic integrated circuits
Some mode solvers are limited to truly guided waves, while others can also handle leaky modes and cladding modes. Some are part of fiber simulation software, while others are specifically made for the modeling of photonic integrated circuits or for photonic crystal waveguides.
Not all mode solvers can compute all of the above quantities; for example, some are not suitable for evaluating bend losses. In complex situations, it can also be difficult to ensure that all modes are reliably found.
There are also mode solvers applied in other areas of physics and technology, e.g. in microwave technology and acoustics. Some of those are still relevant for photonics; for example, one may calculate acoustic modes of fibers, which are needed for investigating Brillouin scattering. But this article focuses on photonic mode solvers.
What are Mode Properties Needed For?
The concept of propagation modes of waveguide structures (for example, optical fibers) is of fundamental importance in photonics. If the details of guided modes and possibly also cladding modes are known, they can be used for a wide range of computations which are often relevant for the performance and use. Some examples:
- Overlap integrals between mode fields are used for calculating coupling efficiencies, e.g. for coupling of light between dissimilar fibers, or between fibers and waveguides of photonic integrated circuits (PICs).
- Similarly, overlap integrals between arbitrary input light beams and fiber modes can be used for calculating the efficiency of coupling light into a fiber.
- The effective mode area of a fiber is essential for the strength of nonlinearities such as self-phase modulation and four-wave mixing.
- Overlaps of modes with laser-active dopants in active fibers influence the modal gain.
- Different group velocities of modes of multimode fibers (→ intermodal dispersion) can degrade signal quality in optical fiber communications.
- The analysis of leaky modes can allow the determination of radiation loss.
- The design of fibers (i.e., of their refractive index profiles) is usually based on considering a number of mode properties, such as mode area and shape, and chromatic dispersion. The latter is vital for some applications like supercontinuum generation.
Many such investigations can alternatively be done with numerical beam propagation, possibly not using the mode concept at all. However, mode-based calculations are often computationally less demanding and provide more physical intuition.
Mode Solver Algorithms
LP Mode Solvers
For standard all-glass optical fibers, which usually exhibit circular symmetry and a low refractive index contrast, one can use the simplest kind of scalar paraxial wave equation and compute so-called LP modes. “LP” means “linearly polarized”, but actually in this context one does not consider the polarization of light at all, regarding this aspect as separable from mode profiles and unrelated to mode properties like effective refractive indices. One considers only one electric field component, neglecting any coupling between such components, as occurs in cases with high index contrast.
One exploits the circular symmetry by writing the mode function as a product of a radial function ($F_{lm}(r)$) and an azimuthal part ($\cos l \varphi$) or ($\exp i l \varphi$). While the azimuthal part is simple to handle, the radial part at least means numerically solving only a single second-order differential equation, which can be transformed into two coupled first-order differential equations.
The radial equation contains a parameter ($\beta$), the initially unknown propagation constant (phase constant). Modes require solutions of the radial equation which converge to 0 for ($r \to \infty$). Such solutions exist only for specific values ($\beta = \beta_{lm}$), where ($l$) is the chosen azimuthal index and ($m$) is a counter index; for other ($\beta$) values, the function rapidly goes to infinity after some number of oscillations. (The number of possible ($m$) values is also initially not known.) So the numerical algorithm must reliably identify all ($\beta$) values for which the mentioned convergence to zero is achieved. It can start the propagation at ($r = 0$), going up to the core radius, i.e., the radius beyond which the refractive index stays constant (the cladding index). As the solutions in the area of the cladding are known to have the form ($K_l(w r)$), where ($w$) depends on ($\beta$), one can consider a boundary condition at the core–cladding interface (also including first-order derivatives) to indirectly check convergence to 0 for ($r \to \infty$). The number of zero crossings of the radial function can be used to ensure that no mode is overlooked.
Although the technical details are altogether certainly not trivial, highly robust and efficient LP mode solvers have been developed which can calculate the details of thousands of guided modes of a large-core multimode fiber within a few seconds, even on a standard PC. Fig. 1 shows an example with a fiber having only a few modes.
For more details, see the article on LP modes.
Only for rough estimates can LP modes be used in the context of index-guiding photonic crystal fibers. In such cases, the effective index method replaces the structured cladding with an average refractive index, treating the fiber as a step-index fiber. This gives reasonable qualitative results but is too approximate for precise properties such as chromatic dispersion or confinement loss.
Fully Vectorial Mode Solvers
If the refractive index contrast is not small — as is the case, for example, in photonic crystal fibers with air holes — the scalar wave equation cannot be used. In addition, such air holes break the assumption of radial symmetry — a second reason why LP mode solvers cannot be used. Similar conditions are found for waveguides on photonic integrated circuits, which often exhibit high index contrast (especially when the optical field reaches the interface to air, but also at many semiconductor interfaces) and reduced symmetry.
The polarization of light cannot be neglected in that regime. There can be substantial coupling e.g. between ($E_x$), ($E_y$) and ($E_z$), so that one cannot consider just one transverse field component. There is a possible hybridization between TE-like and TM-like modes where symmetry and separability are lost. The refractive index now depends on both ($x$) and ($y$), the boundary conditions mix components of electric and magnetic field, and as a result there are no longer pure TE or TM solutions; one has non-negligible longitudinal components ($E_z$) and/or ($H_z$). This leads to hybrid modes of type HE (hybrid electric, TE-like with dominantly transverse electric field) and EH (hybrid magnetic, TM-like with dominantly transverse ($H$) field). For example, the HE11 mode in an optical fiber corresponds to the LP01 approximation; it is mostly TE-like, but with nonzero ($E_z$) and ($H_z$). Some higher-order modes like EH12 are more TM-like. Obviously, some understanding of these more complicated types of modes is a prerequisite for using a vectorial mode solver.
Note that a scalar model would not, for example, be able to predict birefringence of a fiber, introduced by an asymmetric design. For that, a vectorial treatment is essential.
So a vectorial mode solver must fully account for all three electric field components ($E_x$), ($E_y$) and ($E_z$). It must respect electromagnetic boundary conditions at material interfaces and correctly handle polarization coupling and hybridization. The goal is to find fields ($E(x,y)$) and propagation constants ($\beta$) (or effective refractive indices ($n_\textrm{eff} = \beta / k_0$)) such that the fields satisfy Maxwell's equations without making the scalar (LP) approximation or paraxial approximation. The starting point is Maxwell's curl equations, often in the frequency domain, i.e., for a given optical frequency ($\nu$) (or angular frequency ($\omega$)):
$$\nabla \times \vec{E} = -i \omega \mu_0 \mu_\textrm{r} \vec{H}$$ $$\nabla \times \vec{H} = i \omega \epsilon_0 \epsilon_\textrm{r} \vec{E}$$where ($\epsilon_\textrm{r} = n^2$) is generally a function of the position coordinates ($x$) and ($y$) (and can be a tensor for non-isotropic media). One assumes that there is no ($z$) dependence of the structure (as usual when seeking modes). Guided modes have
$$\vec{E}(x, y, z) = \vec{e}(x, y) e^{-i \beta z}$$ $$\vec{H}(x, y, z) = \vec{h}(x, y) e^{-i \beta z}$$Plugging this into Maxwell's equations eliminates the ($z$) derivative (replaced by ($-i\beta$)), leaving two-dimensional coupled equations in ($x$) and ($y$). By eliminating the ($H$) field, one obtains a generalized eigenvalue equation for ($\beta^2$):
$$\nabla_{\perp} \times \left( \frac{1}{\mu} \nabla_{\perp} \times \vec{e} \right) = (n^2 k_0^2 - \beta^2) \vec{e}$$ where ($\nabla_{\perp}$) is the transverse differential operator.
For finding the eigenvalues and eigenvectors (field distributions ($\vec{e}$) and ($\vec{h}$), and finally ($\vec{E}$) and ($\vec{H}$)), one applies some kind of discretization. Different numerical schemes exist:
- Finite-difference (FD) method: One uses a regular ($x-y$) grid, approximates derivatives by central differences, and applies Maxwell's equations on this grid (often using the Yee cell scheme) [9]. That leads to a system of linear equations containing a sparse matrix, to which an efficient algorithm (often based on an iterative method) can be applied.
- This approach is relatively simple to implement, but is not very accurate for curved boundaries or nonrectangular shapes, as encountered e.g. with photonic crystal fibers — while for photonic integrated circuits that solution can be fully satisfactory.
- Finite-element (FE) method: One uses triangular or tetrahedral elements, represents fields inside each element using vector basis functions, and assembles the global matrix enforcing field continuity across elements [6, 10].
- Such an algorithm is difficult to implement, but can well handle curved and complex geometries with high accuracy.
- Fourier / plane wave expansion (PWE) methods: One expands fields and dielectric function as Fourier series (plane wave superpositions). This leads to a matrix eigenvalue equation in the Fourier domain [7]. This works well for periodic photonic crystal structures, but requires extra methods for handling the central “defect” (e.g., a missing air hole). For example, one may assume periodically repeated defects as an approximation, which is reasonable provided that the period is long enough. Convergence can be slow for sharp boundaries or high index contrast.
Different boundary conditions can be applied depending on the physical situation:
- Perfectly Matched Layers (PMLs) absorb fields, suitable for simulating open boundaries for leaky modes.
- Metallic walls are used for confined or symmetric waveguides.
- Symmetry planes can reduce the computational domain for structures with mirror symmetry.
For each computed field, one can calculate additional quantities like the effective mode area, the group index (using solutions for slightly different frequencies) and chromatic dispersion parameters, and figures for polarization (e.g. field amplitude ratios). Propagation losses may also be estimated, e.g. for lossy materials (with complex refractive index) or leaky modes. Note that photonic crystal fiber modes are inherently leaky due to a limited area being covered with air holes, although the leakage loss can be very small.
Substantial technical difficulties can be encountered, and cannot always be fully solved:
- The matrices obtained from discretization can have millions of unknowns. Solving for all eigenvalues would require full diagonalization, which is computationally infeasible. Therefore, practical solvers generally use iterative algorithms (e.g. Arnoldi or Lanczos) that find only the lowest few eigenvalues near a target effective index.
- Guided modes often cluster near certain effective indices (especially near the cladding index). Higher-order or weakly guided modes may be very close together, and numerical searches can then miss some nearly degenerate modes.
- Some structures (e.g. circular fibers) have degenerate mode pairs. Small numerical asymmetries can cause the solver to find only one of the degenerate pair, or to mix them into a single mode. Reliably detecting and distinguishing degenerate or near-degenerate modes requires extra care.
- Leaky modes have complex effective indices, and their fields extend beyond the computational domain. Depending on boundary conditions (PMLs, absorbing layers), these may not appear as true eigenmodes or may appear distorted. Thus, some solvers capture only bound modes, missing leaky or radiation-type solutions.
- Using symmetry planes (to reduce the computational load) can restrict the solver to certain symmetry classes; modes of other symmetry are then not computed.
- Discretization errors or poor mesh resolution can lead to spurious eigenvalues (non-physical solutions) or numerical “pollution” that hides true modes.
- Poorly chosen initial guesses or convergence thresholds can cause missing weakly confined modes, whose fields are mostly outside the core.
Of course, missing modes can have major implications. For example, the simulated modal content, coupling, and losses are incomplete, which affects e.g. coupled-mode simulations. To minimize such risks, a user may need to perform multiple searches with different initial guesses or solver parameters, check convergence as mesh resolution is improved, and use mode overlap integrals or field orthogonality to confirm completeness.
Recent research has begun exploring alternative paradigms beyond classical discretization. These include meshless deep-learning eigenmode solvers [11], physics-informed neural-networks that solve full-vector waveguide modes [13], and physics-aware surrogate neural models for rapid prediction of effective indices [15]. Parallel to these, hybrid analytical/mode-matching solvers have advanced high-index-contrast waveguide modeling [14]. These methods are not yet as widely used as FEM/FD solvers, but they point toward future directions such as real-time mode computation, inverse design integration, and mesh-free geometry handling.
Generally, vectorial mode solvers are not only difficult to make but also slow to operate — although the computational load depends substantially on (a) the quality of used algorithms and their implementation, and (b) on the devices to be modeled. Obviously, it also matters whether one needs solutions only for a specific optical frequency, or scanning of all modes (in multimode regimes) over a substantial frequency range. Fortunately, computers (even personal computers) have become far more powerful in recent decades, and substantial progress has also been made on numerical algorithms. Many utilize special SIMD instructions of modern CPUs which result in much enhanced speed.
Semi-vectorial Mode Solvers
There are also simpler versions, called semi-vectorial mode solvers, which utilize additional simplifying approximations for lowering the computational load. They can be appropriate for waveguides exhibiting a moderate refractive index contrast (e.g. <0.3 between different semiconductor materials, but not between solids and air) and are only mildly asymmetric.
Essentially, one treats only the dominant transverse field component (for TE-like or TM-like modes, treated separately), assuming that the coupling between components is weak. Polarization hybridization cannot be investigated. One still accounts for the full refractive index profile, using correction factors for treating discontinuities of refractive index with approximations.
The resulting accuracy can be good if the conditions are reasonably fulfilled, and the CPU time can be substantially lower than for a full vectorial solver.
Modern Approaches
As mentioned above, new classes of mode solvers have emerged that leverage machine-learning or meshless formulations rather than traditional grid or element discretization. Such novel approaches are still maturing but point toward future trends in photonic simulation.
Even analytical (or semi-analytical) techniques are evolving for high-index-contrast and complex geometries [14], so besides numerical solvers, analytical and hybrid methods remain active research areas.
Quality and Performance Criteria for Mode Solvers
Mode solver software has been developed for quite different application areas, and can differ substantially in various performance and quality criteria:
- Applicability and physical accuracy: Some products are very efficient and reliable in simple cases with LP modes, but restricted to waveguides with low refractive index contrast and radial symmetry. Others are fully vectorial or at least semi-vectorial, suitable for devices with higher refractive index contrast.
- Computational efficiency: There are huge differences in computational speed: LP mode solvers can compute details of thousands of modes in one second, while others may require minutes to hours, especially for finding all modes in some wavelength range. Efficiency can strongly depend not only on used algorithms, but also the quality of implementation and the use of SIMD CPU instructions.
- Reliability and robustness: While particularly LP solvers can be highly reliable, even in situations with thousands of modes for a given frequency, some implementations easily miss modes, or yield inaccurate results, and are dependent on a non-obvious choice of parameters.
- Usability and practical functionality: Various practical aspects can be important. For example, software may be more or less convenient and flexible in defining refractive index profiles, exporting calculated data and making diagrams.
- Documentation: Accurate and helpful technical documentation, ideally well introducing non-expert users to the field, can be vital.
- Technical support: For such highly non-trivial software, competent technical support is indispensable.
The optimal choice of mode solver software therefore depends strongly on the intended application: Rapid analysis of standard fibers requires only a simple LP solver, whereas accurate modelling of high-contrast integrated-optics structures demands fully vectorial methods and careful numerical control.
Relations to Numerical Beam Propagation
Using numerical beam propagation is a fundamentally different approach: Without considering modes, one can propagate fields along a waveguide [2, 3]. This works even if the structure has a ($z$) dependence — for example, a varying core diameter for tapered fibers. Both methods ultimately describe solutions of Maxwell's equations; beam propagation computes field evolution directly, whereas mode solvers yield the stationary eigenmodes. Not only mode solvers, but also beam propagation can work with fully vectorial methods [5].
There are various relations between beam propagation and mode solvers:
- A mode solver is often used to obtain an initial field distribution for numerical beam propagation. For example, one may study bend losses of fibers or other waveguides starting with an individual mode. Even if a mode solver works only for a straight fiber, one can use beam propagation to get the modified field distributions of bent fibers. Similarly, when studying a fiber coupler with single-mode fibers, it is appropriate to input the computed mode in order to focus the investigation on the coupling.
- Results of beam propagation may be analyzed using calculated modes. For example, one may use overlaps with mode functions for determining mode content.
- By applying beam propagation to an initial field which is a superposition of multiple modes, one obtains a ($z$)-dependent field. One can apply Fourier analysis to that field to obtain ($\beta$) values. That may be useful, for example, for scanning certain ranges with a mode solver.
- One may transition from modal analysis to beam propagation in highly multimode situations, where the concept of modes is often less useful.
- Numerical beam propagation can also serve as the core of a mode solver, going well beyond the simple analytical LP modes. The basic idea is to exploit the fact that guided modes are eigenfunctions of the propagation operator. Starting from a suitable initial field distribution and repeatedly propagating it over a long distance, one finds that the higher-order modes gradually decay faster than the lowest-loss (fundamental) mode. The field envelope then converges to the fundamental mode. Artificial absorbing regions or peripheral losses (e.g., complex refractive index near the boundaries) are often introduced to enhance mode discrimination, thus accelerating convergence [3, 4].
- A method called Eigenmode Expansion (EME) calculates propagation of light, but instead of beam propagation, it utilizes calculated modes. This works also for waveguide structures with variable properties; one can slice such a structure into segments along the propagation direction, where the cross-section (and thus the set of local modes) is constant within each segment. The electromagnetic field is expanded into these local modes. Propagation through a segment is modeled by simple phase changes of the mode coefficients, while the interfaces between segments are handled by scattering matrices calculated from mode overlap integrals.
- EME is particularly efficient for bidirectional propagation in structures with strong reflections (e.g., Bragg gratings) and for long tapers, where standard beam propagation might be slow or inaccurate regarding reflections. A fast and robust mode solver is the core engine of any EME simulation tool.
Frequently Asked Questions
This FAQ section was generated with AI based on the article content and has been reviewed by the article’s author (RP).
What is a mode solver?
A mode solver is a computational tool used in photonics to calculate the electromagnetic field distributions and propagation characteristics of optical modes within waveguide structures like optical fibers or integrated circuits.
What key information does a mode solver provide?
A mode solver calculates properties for each mode, including its transverse field profile, effective refractive index, propagation constant, mode area, group velocity dispersion, and potential losses like bend or confinement loss.
When is a simple LP mode solver sufficient?
An LP (Linearly Polarized) mode solver is suitable for waveguides with low refractive index contrast and circular symmetry, such as standard all-glass optical fibers. It offers high computational speed but neglects polarization-dependent vector effects, which are essential in some cases.
Why are fully vectorial mode solvers necessary for some waveguides?
Vectorial mode solvers are required for structures with high refractive index contrast or complex geometries, like photonic crystal fibers and photonic integrated circuits. They accurately model polarization, field component coupling, and birefringence, which simpler scalar models cannot.
What are the main numerical methods used in vectorial mode solvers?
Common methods include the finite-difference (FD) method on a regular grid, the finite-element (FE) method using a flexible mesh for complex geometries, and the plane wave expansion (PWE) method for periodic structures.
What are some common challenges when using mode solvers?
Challenges include high computational demand, the risk of missing nearly degenerate or weakly guided modes, and the appearance of non-physical solutions. Reliable results often require careful parameter setup and convergence checks.
How do mode solvers relate to numerical beam propagation?
Mode solvers find the stationary eigenmodes of a uniform waveguide, while beam propagation simulates the evolution of an arbitrary field. They are complementary: Modes can provide the input for beam propagation, which in turn can analyze structures with no simple modes, such as tapered fibers.
Suppliers
Sponsored content: The RP Photonics Buyer's Guide contains five suppliers for mode solver software. Among them:

The RP Fiber Power software contains a powerful LP mode solver which can calculate the amplitude profiles, effective mode areas, cut-off wavelengths, propagation constants, group velocities and chromatic dispersion of all guided fiber modes from a given radially symmetric refractive index profile. This works very reliably and fast even for cases with thousands of modes. The software is also extremely flexible concerning data import and export, diagrams and further data processing.
Bibliography
| [1] | A. W. Snyder and W. R. Young, “Modes of optical waveguides”, J. Opt. Soc. Am. 68 (3), 297 (1978); doi:10.1364/josa.68.000297 |
| [2] | M. D. Feit and J. A. Fleck, “Light propagation in graded-index optical fibers”, Appl. Opt. 17 (24), 3990 (1978); doi:10.1364/ao.17.003990 |
| [3] | J. Van Roey, J. van der Donk and P. E. Lagasse, “Beam-propagation method: analysis and assessment”, J. Opt. Soc. Am. 71 (7), 803 (1981); doi:10.1364/josa.71.000803 |
| [4] | Y. Chung and N. Dagli, “An assessment of finite difference beam propagation method”, IEEE J. Quantum Electron. 26 (8), 1335 (1990); doi:10.1109/3.59679 |
| [5] | J. M. Liu and L. Gomelsky, “Vectorial beam propagation method”, J. Opt. Soc. Am. A 9 (9), 1574 (1992); doi:10.1364/josaa.9.001574 |
| [6] | B. M. A. Rahman, “Finite element analysis of optical waveguides”, Prog. Electromagn. Res. PIER 10, 187 (1995) |
| [7] | S. Johnson and J. Joannopoulos, “Block-iterative frequency-domain methods for Maxwell's equations in a planewave basis”, Opt. Expr. 8 (3), 173 (2001); doi:10.1364/oe.8.000173 |
| [8] | P. Bienstman and R. Baets, “Optical modelling of photonic crystals and VCSELs using eigenmode expansion and perfectly matched layers”, Optical and Quantum Electronics 33 (4-5), 327 (2001); doi:10.1023/a:1010882531238 |
| [9] | A. B. Fallahkhair, K. S. Li and T. E. Murphy, “Vector finite difference modesolver for anisotropic dielectric waveguides”, J. Lightwave Technol. 26 (11), 1423 (2008) |
| [10] | P. Pintus, “Accurate vectorial finite element mode solver for magneto-optic and anisotropic waveguides”, Opt. Expr. 22 (13), 15737 (2014); doi:10.1364/oe.22.015737 |
| [11] | G. Alagappan and C. E. Png, “Meshless optical mode solving using scalable deep deconvolutional neural network”, Sci. Rep. 13 (1) (2023); doi:10.1038/s41598-022-25613-4 |
| [12] | N. Zhang and Y. Y. Lu, “Spectral Galerkin mode-matching method for applications in photonics”, Phys. Rev. E 109 (5) (2024); doi:10.1103/physreve.109.055303 |
| [13] | W. Xu et al., “High precision, full-vector optical mode solving in waveguides via fourth-order derivative physics-informed neural networks”, Opt. Expr. 33 (18), 38317 (2025); doi:10.1364/oe.571156 |
| [14] | B. Li, C. Sun and W. Huang, “High-accuracy analytical solver for guided and unguided complex modes of optical waveguides”, Opt. Expr. 33 (6), 12868 (2025); doi:10.1364/oe.556019 |
| [15] | H. S. Ünal and A. C. Durgun, “A physics-aware neural network for effective refractive index prediction of photonic waveguides”, Opt. and Quantum Electron. 57, 107 (2025), https://link.springer.com/article/10.1007/s11082-024-08009-8 |
(Suggest additional literature!)




