← Back to Home

Overview

This project asks a deliberately narrow question: when a single low-cost IMU is rigidly attached to a moving object, what can be recovered about its ego-centric motion from the raw inertial stream? This is a sensor-centric first step toward object-centric sensing—not yet a complete 6D tracker or a visual-inertial fusion system.

The motivation comes from dynamic-scene understanding. Static geometry and appearance can now be reconstructed reliably with methods such as 3D Gaussian Splatting [1], NeRF [2], NeuS/VolSDF [3], [4], and COLMAP [5]. Motion under contact and occlusion remains substantially harder. Dense multi-view systems provide accurate capture but require calibrated, synchronized cameras; monocular pose tracking is easier to deploy but becomes fragile when objects occlude one another or share repeated appearance. Figure 1 illustrates this failure mode: segmentation and tracking break down for visually similar bowling pins under strong inter-object occlusion.

IMUs offer a complementary route: they require no line of sight, operate at high rate, and are inexpensive enough to attach to individual objects. Human motion-capture systems already exploit these properties, but our target regime is more difficult. A manipulated object may undergo non-periodic motion, abrupt changes of direction, and far weaker motion priors than pedestrian gait or robot locomotion. Rather than applying naive double integration to estimate pose, we use velocity direction as an interpretable intermediate target and map out the conditions under which it is—and is not—recoverable.

The main outcomes are clear. A single sensor recognizes coarse, homogeneous motion very well (95.82% on AXIS-7), but performance falls sharply across data-collection conditions and for windows containing direction changes. These are useful negative results: they identify condition invariance and temporally richer supervision as the central requirements for practical inertial object sensing.

Why Single-IMU Ego-Motion Is Difficult

Most learning-based inertial odometry work targets pedestrians [7], [8], [9] or robotic platforms [10], [11], [12]. Such platforms have structured dynamics—quasi-periodic gait or controller-regularized trajectories—so learning a mapping from an IMU window to average velocity can be effective within the observed motion distribution. Physically, however, integrating acceleration over a finite interval yields a velocity increment \(\Delta\mathbf{v}\), not an absolute velocity. Window-to-velocity prediction therefore relies on a learned local motion prior.

That assumption is much weaker for arbitrary object-like motion. The sensor may be attached to diverse bodies with different couplings, accelerations, and contact events. We consequently use direction classification as a diagnostic formulation. It removes velocity magnitude as a confound, produces interpretable motion primitives, and lets us increase motion complexity in a controlled way. It does not claim that a discretized class label is the final form of object tracking.

Hardware and Robot-Assisted Data Collection

To obtain repeatable kinematic supervision without the ambiguity of freehand motion, we mount a WitMotion WT901WIFI on the end-effector of a UFACTORY xArm6 using a custom 3D-printed fixture. The robot executes predefined programs while the sensor records raw 3-axis acceleration and 3-axis angular velocity at 100 Hz. Software synchronization aligns the robot and IMU streams; the sensor center and axes are registered to the configured gripper frame.

(a) Straight-line motion

(b) Waypoint-driven polyline motion

Although the WT901WIFI is a 9-axis device, we do not rely on its fused orientation in typical environments. That estimate depends strongly on magnetometer observations and can be corrupted by nearby metal or electronic equipment. Instead, with the fixed IMU-to-gripper alignment and a level robot base, we compute gravity in the body frame from robot forward kinematics and compensate it from the acceleration signal. The learned model therefore uses the IMU as a practical low-cost 6-axis sensor. At roughly 100 RMB, it is attractive for large-scale experiments even though it cannot match premium motion-capture hardware.

The hardware also constrains the learning problem. We perform no explicit bias removal beyond gravity compensation. Preliminary Allan-variance analysis [13] reveals axis-dependent noise and bias instability, especially on the accelerometer’s \(z\) axis. This makes imperfect gravity removal and continuous regression particularly sensitive along the vertical direction, a pattern that reappears in the classification errors.

Formulation and Baseline

Each example is a 100-frame window (\(W=100\)) of body-frame angular velocity \(\boldsymbol{\omega}_{k:k+W}\) and gravity-compensated acceleration \(\mathbf{a}_{k:k+W}\). We learn

\[\mathcal{F}: [\boldsymbol{\omega}_{k:k+W}, \mathbf{a}_{k:k+W}] \rightarrow c, \qquad c \in \mathcal{C},\]

where \(\mathcal{C}\) is a set of discrete velocity-direction bins. Labels are derived from xArm6 motion in the end-effector body frame. Sequences are divided into train/validation/test partitions at a ratio of 0.8/0.1/0.1 before window extraction, preventing direct leakage between overlapping windows.

Our baseline is iTransformer [14] with a lightweight 1D-convolutional embedding of the raw six-channel signal, learnable temporal position encoding, and a classification head. This is intentionally a capable but conventional sequence model: the project evaluates inertial sensing limits rather than proposes a new network architecture.

As a capacity sanity check, we also evaluate the backbone on the public RoNIN inertial-odometry benchmark [8]. This experiment is separate from the robot-collected classification study: it only checks that the chosen sequence model can extract useful information from IMU streams without extensive task-specific tuning. ATE and RTE are in meters; lower is better.

Method	RONIN-ResNet [8]	CTIN [15]	iMoT [16]	DiffusionIMU [17]	M2EIT [18]	Ours
Seen ATE/RTE	3.70/2.78	4.62/2.81	3.78/2.68	3.64/2.72	3.58/2.76	3.80/2.75
Unseen ATE/RTE	5.48/4.56	5.61/4.48	5.31/4.39	5.27/4.31	5.19/4.57	5.47/4.61

Progressive Evaluation: What the Sensor Can Recover

We collected four robot datasets spanning approximately 25.5 hours and progressively increasing motion complexity.

Dataset	Motion and labels	Scale
AXIS-7	\(\pm x, \pm y, \pm z\) and static homogeneous motion	~500 sequences/class; ~6 h
DIR27-L	27 body-frame directions, \(v_i \in \{-1,0,1\}\)	200 sequences/class; ~7.5 h
DIR27-S	Same 27 directions under a separate condition	100 sequences/class; ~4 h
POLY-27	Waypoint-driven, non-homogeneous polylines	200 sequences/class; ~8 h

Here a condition includes the motion program, initial pose, fixture coupling, sensor state, and recording context. For AXIS-7 and DIR27, the robot begins each segment from diverse orientations. These rotations preserve body-frame labels while varying gravity projection and orientation-dependent artifacts, testing whether the model learns motion rather than trivial pose cues. Unless noted otherwise, training uses raw six-channel windows, batch size 1024, 10 epochs, and one GeForce RTX 3090 GPU, with no per-channel normalization or extensive hyperparameter search.

Phase 1 — Axis-Aligned Homogeneous Motion

On AXIS-7, the classifier reaches 95.82% accuracy and 0.9581 weighted F1. This demonstrates that one low-cost IMU contains sufficient information to recognize coarse translational primitives when motion is straight, homogeneous, and accurately gravity-compensated. The remaining errors are primarily between opposite directions on the same axis, particularly \(+z\) and \(-z\), consistent with the sensor asymmetry observed above.

Phase 2 — Fine-Grained Directions and Cross-Condition Robustness

For DIR27, each velocity component is quantized to \(\{-1,0,1\}\), yielding 27 direction classes. Matched-condition results remain strong: 92.23% on DIR27-L, 88.84% on DIR27-S, and 91.84% when the two datasets are merged for matched-condition training and testing.

The central limitation appears when the condition changes. Training on DIR27-L and testing on DIR27-S falls to 58.16%; the reverse transfer reaches only 53.38%. Both confusion matrices show similar structures, indicating a systematic condition gap rather than one anomalous split. The model is likely using some condition-specific cues—controller response, mounting and fixture coupling, sensor state, or recording context—instead of fully invariant physical motion features. High matched-condition accuracy is therefore not sufficient evidence of robust physical understanding.

(a) Train DIR27-L, test DIR27-S

(b) Train DIR27-S, test DIR27-L

We also tested rotation-equivariant augmentation inspired by RIO [19]: rotating an inertial window by \(\mathbf{R}\) should rotate its direction label by the same transformation. In the real pipeline, the assumption does not hold cleanly. On AXIS-7, this augmentation reduces accuracy from 95.82% to 85.57%. A label-flip diagnostic that swaps \(+y\) and \(-y\) at evaluation reduces the corresponding F1 score by 24.7%. Sensor-axis bias, controller dynamics, and fixture coupling are all axis dependent, so ideal rigid-rotation augmentation is mismatched to the measured distribution.

Phase 3 — Non-Homogeneous Polyline Motion

POLY-27 contains waypoint-driven trajectories for which the instantaneous velocity can change within a 100-frame window. A constant direction label is no longer available, so we use net displacement as a proxy target,

\[\mathbf{d}_{\text{net}} = \mathbf{p}_{k+W} - \mathbf{p}_k,\]

and assign its nearest directional bin. Accuracy drops to 49.2%—well above 27-way random chance (3.7%), but far below the straight-line settings. The result exposes a mismatch between the target and the signal: net displacement discards the local direction changes within a window. In this regime, the single-label-window formulation becomes much less informative than the raw inertial sequence it summarizes.

Takeaways and Next Steps

The study establishes both the promise and the boundary of active inertial sensing with one low-cost IMU. Under controlled, homogeneous motion, the sensor can recover coarse ego-motion primitives reliably. Yet cross-condition transfer, idealized rotation augmentation, and non-homogeneous trajectories all reveal that the current formulation is not robust enough for practical object tracking.

The most useful next directions are:

• condition-invariant sensing and calibration that explicitly model bias, mounting, and controller effects;
• adaptive or multi-scale temporal windows rather than a fixed 100-frame horizon;
• dense per-frame direction prediction or sequence-to-sequence velocity modeling, replacing a single net-displacement label; and
• fusion with visual observations or physics-informed constraints when the application requires full object trajectories.

Velocity-direction classification remains valuable as a controlled diagnostic target. Its failures clarify what a future inertial object-sensing system must solve before it can support reliable dynamic-scene understanding.

References

1. [1]B. Kerbl, G. Kopanas, T. Leimkuehler, and G. Drettakis, “3D Gaussian Splatting for Real-Time Radiance Field Rendering,” ACM Trans. Graph., vol. 42, no. 4, July 2023, doi: 10.1145/3592433.
2. [2]B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis,” in ECCV, 2020.
3. [3]P. Wang, L. Liu, Y. Liu, C. Theobalt, T. Komura, and W. Wang, “NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction,” arXiv preprint arXiv:2106.10689, 2021.
4. [4]L. Yariv, J. Gu, Y. Kasten, and Y. Lipman, “Volume rendering of neural implicit surfaces,” in Thirty-Fifth Conference on Neural Information Processing Systems, 2021.
5. [5]J. L. Schönberger and J.-M. Frahm, “Structure-from-Motion Revisited,” in Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
6. [6]N. Carion et al., “SAM 3: Segment Anything with Concepts.” 2025. Available at: https://arxiv.org/abs/2511.16719
7. [7]C. Chen, X. Lu, A. Markham, and N. Trigoni, “IONet: learning to cure the curse of drift in inertial odometry,” in Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, in AAAI’18/IAAI’18/EAAI’18. AAAI Press, 2018.
8. [8]S. Herath, H. Yan, and Y. Furukawa, “RoNIN: Robust Neural Inertial Navigation in the Wild: Benchmark, Evaluations, & New Methods,” in 2020 IEEE International Conference on Robotics and Automation (ICRA), 2020, pp. 3146–3152. doi: 10.1109/ICRA40945.2020.9196860.
9. [9]B. Rao, E. Kazemi, Y. Ding, D. M. Shila, F. M. Tucker, and L. Wang, “CTIN: Robust contextual transformer network for inertial navigation,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2022, pp. 5413–5421.
10. [10]S. Zhao, S. Zhou, R. Blanchard, Y. Qiu, W. Wang, and S. Scherer, “Tartan IMU: A Light Foundation Model for Inertial Positioning in Robotics,” in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 22520–22529.
11. [11]Y. Qiu, C. Xu, Y. Chen, S. Zhao, J. Geng, and S. Scherer, “AirIO: Learning Inertial Odometry With Enhanced IMU Feature Observability,” IEEE Robotics and Automation Letters, vol. 10, no. 9, pp. 9368–9375, 2025, doi: 10.1109/LRA.2025.3581130.
12. [12]C. Luo, Y. Wang, W. Cai, and M. Zhao, “AutoOdom: Learning Auto-regressive Proprioceptive Odometry for Legged Locomotion.” 2025. Available at: https://arxiv.org/abs/2511.18857
13. [13]R. Buchanan, Allan Variance ROS. (November 2021). Oxford Robotics Institute, DRS Lab. Available at: https://github.com/ori-drs/allan_variance_ros
14. [14]Y. Liu et al., “iTransformer: Inverted Transformers Are Effective for Time Series Forecasting,” in The Twelfth International Conference on Learning Representations, 2024. Available at: https://openreview.net/forum?id=JePfAI8fah
15. [15]B. Rao, E. Kazemi, Y. Ding, D. M. Shila, F. M. Tucker, and L. Wang, “CTIN: Robust contextual transformer network for inertial navigation,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2022, pp. 5413–5421.
16. [16]S. M. Nguyen, D. V. Le, and P. Havinga, “imot: Inertial motion transformer for inertial navigation,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2025, pp. 6209–6217.
17. [17]X. Teng et al., “DiffusionIMU: Diffusion-Based Inertial Navigation with Iterative Motion Refinement,” in Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25, J. Kwok, Ed., International Joint Conferences on Artificial Intelligence Organization, Aug. 2025, pp. 8787–8795. doi: 10.24963/ijcai.2025/977.
18. [18]Y. Li et al., “M2EIT: Multi-Domain Mixture of Experts for Robust Neural Inertial Tracking,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2025, pp. 28207–28216.
19. [19]X. Cao, C. Zhou, D. Zeng, and Y. Wang, “RIO: Rotation-Equivariance Supervised Learning of Robust Inertial Odometry,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 6614–6623.

---

← Back to Home