Skip to content

Detailed derivation of On-Manifold IMU Preintegration

  • C. Forster, L. Carlone, F. Dellaert, and D. Scaramuzza, “On-Manifold Preintegration for Real-Time Visual--Inertial Odometry,” IEEE Trans. Robot., vol. 33, no. 1, pp. 1–21, Feb. 2017, doi: 10.1109/TRO.2016.2597321.

  • Z. Yang and S. Shen, “Monocular Visual–Inertial State Estimation With Online Initialization and Camera–IMU Extrinsic Calibration,” IEEE Trans. Automat. Sci. Eng., vol. 14, no. 1, pp. 39–51, Jan. 2017, doi: 10.1109/TASE.2016.2550621.

Inertial Measurement Unit (IMU) preintegration is a fundamental technique in visual-inertial odometry that efficiently combines high-frequency IMU measurements between keyframes. This approach, pioneered by Forster et al. and Yang et al., formulates the integration process on the manifold of rigid body motions SE(3), addressing the nonlinear nature of 3D rotations through Lie group theory.

The key innovation lies in separating the integration of IMU measurements from the global state, enabling computationally efficient optimization by precomputing relative motion constraints. This derivation details the mathematical foundations, including the special orthogonal group SO(3), perturbation models, uncertainty representation on manifolds, and the complete preintegration theory that handles sensor biases and noise characteristics while maintaining real-time performance. The resulting preintegrated terms serve as constraints in factor graph optimization frameworks for robust state estimation.

Special Orthology Group SO(3)

The special orthology group SO(3) is defined as the set of all R3×3 rotation matrix:

SO(3)={RR3×3:RTR=I,det(R)=1}

And its tagent space on the manifold so(3), consist of skew-symmetric matrices:

(1)ω=[ω1ω2ω3]=[0ω3ω2ω30ω1ω2ω10]so(3)

The hat operator () maps a vector ωR3 to so(3), while the vee operator () performs the inverse mapping.

Rodrigues' Rotation Formula

Let vR3 be a 3D vector to be rotated by a unit vector axis n=(nxnynz) by an angle θ. The resulting rotated vector is denoted vrot.

v can be decomposed to two orthogonal components: the parallel to n part v and the perpendicular to n part v. Which satisfy,

{v=(vn)nv=v(vn)n=n×(n×v)

The whole vector v is then rotated by:

vrot=vrot+vrot=v+cosθv+sinθn×v=(vn)n+cosθv+sinθn×v=cosθv+(1cosθ)(nv)n+sinθn×v

We get Rodrigues' Rotation Formula. Write it on the manifold, we then have:

R=cos(||ϕ||)I+[1cos(||ϕ||)]ϕϕT||ϕ||2+sin(||ϕ||)ϕ

The exponential map exp:so(3)SO(3) converts a skew-symmetric matrix into a rotation matrix via the Rodrigues rotation formula:

(3)R=exp(ϕ)=I+sinϕϕϕ+1cosϕϕ2(ϕ)2

For small angles, this simplifies to RI+ϕ.

Exponential MappingR=exp(ϕ)=n=0(ϕ)nn!=I+(||ϕ||||ϕ||33!+||ϕ||55!+)ϕ+(||ϕ||22!||ϕ||44!+||ϕ||66!)(ϕ)2=I+sin||ϕ||||ϕ||ϕ+1cos||ϕ||||ϕ||2(ϕ)2

The logarithmic map $ \log : SO(3) \to \mathfrak{so}(3) $ extracts the axis-angle representation from a rotation matrix:

(5)ϕ=log(R)=ϕ2sinϕ[r32r23r13r31r21r12]
Logarithmic MappingR=I+sin||ϕ||||ϕ||ϕ+1cos||ϕ||||ϕ||2(ϕ)2

So the trace of the rotation matrix satisfied:

tr(R)=tr(I)+sin||ϕ||||ϕ||tr(ϕ)+1cos||ϕ||||ϕ||2tr[(ϕ)2]=3+0+1cos||ϕ||||ϕ||2(||ϕ||2)=3+0+1cos||ϕ||||ϕ||2(2||ϕ||2)=3+2cos||ϕ||2

The angle θ is calculated by

||ϕ||=arccos(tr(R)12)+2kπ

When ϕ0RI, we construct a skew part and a non-skew part.

R=sin||ϕ||||ϕ||ϕ12(RRT)+I+1cos||ϕ||||ϕ||2(ϕ)212(R+RT)

So

(5)ϕ=log(R)=[||ϕ||(RRT)2sin||ϕ||]=||ϕ||2sin||ϕ||[r32r23r13r31r21r12]

For simplicity of notation, Exp and Log are defined as mappings between vector space R3 and Lie Group SO(3), while exp and log operate between Lie Algebra so(3) and SO(3)

Perturbation Models and Jacobians

For small perturbations δϕ of exponential and logarithm, we use first order approximation:

(7,9){Exp(ϕ+δϕ)Exp(ϕ)Exp(Jr(ϕ)δϕ)Log(Exp(ϕ)Exp(δϕ))ϕ+Jr1(ϕ)δϕ

The right jacobian and its inverse are given by

(8)Jr(ϕ)=I1cos(||ϕ||)||ϕ||2ϕ+||ϕ||sin(||ϕ||)||ϕ||3(ϕ)2Jr1(ϕ)=I+12ϕ+(1||ϕ||2+1+cos(||ϕ||)2||ϕ||sin(||ϕ||))(ϕ)2
Perturbation Jacobians

Since a general increment cannot be defined on the special orthogonal group R1+R2SO(3),R1,R2SO(3), we use perturbation models defined above.

(7)Exp(ϕ+Δϕ)=Exp(JlΔϕ)Exp(ϕ)=Exp(ϕ)=Exp(JrΔϕ)

The Lie bracket (binary operator on Lie groups) is defined as:

[A,B]=ABBA

Using the Baker-Campbell-Hausdorff (BCH) formula:

(9)log(Exp(α)Exp(β))=log(AB)=n=1(1)n1nri+si>0,i[1,n](i=1n(ri+si))1Πi=1n(ri!si!)[Ar1Bs1Ar2Bs2ArnBsn]=A+B+12[A,B]+112[A,[A,B]]112[B,[A,B]]+{β+Jl(β)1α,α0α+Jr(α)1β,β0

Additional Jacobians:

Jl(ϕ)=I+1cos(||ϕ||)||ϕ||2ϕ+||ϕ||sin(||ϕ||)||ϕ||3(ϕ)2Jr(ϕ)=I1cos(||ϕ||)||ϕ||2ϕ+||ϕ||sin(||ϕ||)||ϕ||3(ϕ)2Jl1(ϕ)=I12ϕ+(1||ϕ||2+1+cos(||ϕ||)2||ϕ||sin(||ϕ||))(ϕ)2Jr1(ϕ)=I+12ϕ+(1||ϕ||2+1+cos(||ϕ||)2||ϕ||sin(||ϕ||))(ϕ)2

For any vector vR3, using the properties of the cross product and the special orthogonal group, we have:

(Rp)v=(Rp)×v=(Rp)×(RR1v)=R[p×(R1v)]=RpRTv

Since RpRT=(Rp) holds for each term in the Taylor expansion of the exponential map, it follows that:

Rexp(ϕ)RT=exp((Rϕ))

Equivalently:

(10)RExp(ϕ)RT=Exp(Rϕ)

Uncertainty Description in SO(3)

An intuitive way to define uncertainty on rotation matrices is to right-multiply the matrix by a small perturbation that follows a normal distribution:

(12)R~=RExp(ϵ), ϵN(0,Σ)

For Gaussian distributions, we have the normalization condition:

(13)R3p(ϵ)dϵ=R31(2π)3det(Σ)exp(12||ϵ||Σ2)dϵ=1

Substituting ϵ=Log(R1R~), we obtain

SO(3)1(2π)3det(Σ)exp(12||Log(R1R~)||Σ2)|dϵdR~|dR~=1

The scaling factor, known as the Jacobian determinant, is given by the right-perturbation model:

|dϵdR~|=|1Jr(Log(R1R~))|

Rewriting gives:

(14)SO(3)1(2π)3det(Σ)|1Jr(Log(R1R~))|exp(12||Log(R1R~)||Σ2)dR~=1

From this, the probability density function on SO(3) is:

(15)p(R~)=1(2π)3det(Σ)|1Jr(Log(R1R~))|exp(12||Log(R1R~)||Σ2)

For small perturbations, the normalization term 1(2π)3det(Σ)|1Jr(Log(R1R~))| can be approximated as constant, leading to the following expression for the negative log-likelihood:

(16)L(R)=12||Log(R1R~)||Σ2+c=12||Log(R~1R)||Σ2+c

Gauss-Newton Method on Manifolds

For standard Gauss-Newton optimization:

x=argminxf(x)x=x+argminΔxf(x+Δx)

On manifolds, this becomes:

(18)x=argminxMf(x)x=RxargminδxRnf(Rx(δx))

Where Rx() is a retraction mapping from the tangent space to the manifold.

In the case of the SO(3) group, the retraction is defined as:

(20)RR(δϕ)=RExp(δϕ), δϕR3

For the SE(3) group, it is:

(21)RT(δϕ,δp)=[RExp(δϕ)p+Rδp], [δϕδp]R6

IMU Preintegration

The state of the system at time k is represented by the IMU's orientation, position, velocity, and sensor biases

(22)xk=[Rwbk(qwbk),pwbk,vkw,bgbk,babk]

Let a^b(t) and ω^b(t) denote the measurements from the three-axis accelerometer and gyroscope, respectively. These are corrupted by noise and time-varying biases:

(27, 28)ω^b(t)=ωb(t)+bgb(t)+ngb(t)a^b(t)=Rbw(t)[aw(t)+gw]+bab(t)+nab(t)

In this notation, the superscript w refers to the world (inertial) frame, while b denotes the body (sensor) frame. The subscripts a and g refer to the accelerometer and gyroscope, respectively.

The time dirivatives of R,p,v are given as:

(29)p˙wb(t)=vw(t)v˙w(t)=aw(t)R˙wb(t)=Rwb(t)Exp[ωb(t)]q˙wb(t)=qwb(t)[012ωb(t)]

Using these dynamics, we can express the system state at time t+Δt as follows:

pwb(t+Δt)=pwb(t)+vw(t)Δt+tt+Δtaw(τ)dτ2=pwb(t)+vw(t)Δt+tt+Δt[Rwb(τ)(a^b(τ)bab(τ)nab(τ))gw]dτ2=pwb(t)+vw(t)Δt12gw(Δt)2+tt+ΔtRwb(τ)[a^b(τ)bab(τ)nab(τ)]dτ2vw(t+Δt)=vw(t)+tt+Δtaw(τ)dτ=vw(t)+tt+Δt[Rwb(τ)(a^b(τ)bab(τ)nab(τ))gw]dτ2=vw(t)gwΔt+tt+ΔtRwb(τ)[a^b(τ)bab(τ)nab(τ)]dτ2Rwb(t+Δt)=tt+ΔtRwb(τ)Exp(ωb(τ))dτ=tt+ΔtRwb(τ)Exp(ω^b(τ)bgb(τ)ngb(τ))dτqwb(t+Δt)=tt+Δtqwb(τ)[012ωb(t)]dτ=tt+Δtqwb(τ)[012(ω^b(τ)bgb(τ)ngb(τ))]dτ

However, recomputing Rwb(τ) at each step leads to repeated integration and unnecessary computational cost. To mitigate this, we use the identity:

Rwb(τ)=RwbtRbtb(τ)

This allows us to factor out Rwbi from the integrals over the interval [ti,tj], resulting in:

Rwbj=RwbititjRbib(τ)Exp(ω^b(τ)bgb(τ)ngb(τ))dτqwbj=qwbititjqbib(τ)[012(ω^b(τ)bgb(τ)ngb(τ))]dτpwbj=pwbi+viwΔt12gw(Δt)2+RwbititjRbib(τ)[a^b(τ)bab(τ)nab(τ)]dτ2vjw=viwgwΔt+RwbititjRbib(τ)[a^b(τ)bab(τ)nab(τ)]dτ

We define the following preintegrated terms:

αbibj=titjRbib(τ)[a^b(τ)bab(τ)nab(τ)]dτ2βbibj=titjRbib(τ)[a^b(τ)bab(τ)nab(τ)]dτγbibj=titjRbib(τ)Exp(ω^b(τ)bgb(τ)ngb(τ))dτ

Thus, the final update equations become:

pwbj=pwbi+viwΔt12gwΔt2+Rwbiαbibjvjw=viwgwΔt+RwbiβbibjRwbj=Rwbiγbibj

Finally, since the gyroscope and accelerometer biases are modeled as Gaussian white noise processes with zero mean:

b˙N(0,Σ)

we assume that the bias remains approximately constant between two consecutive time steps:

bgbi=bgbj,babi=babj,i,j