Acies Vehicle Classifier

Introduction

A real-time vehicle classification system that processes multimodal sensor data (seismic and acoustic) to identify vehicles using foundation models. The system supports state synchronization, temporal ensemble voting, and distributed processing through the AciesOS middleware.

The Classifier class implements core infrastructure including: stream data buffering to accumulate sensor readings over configurable time windows, energy-based filtering to reject low-energy noise from seismic and acoustic sensors, temporal ensemble voting that combines multiple classification results to improve accuracy over time, automated scheduling of inference cycles and message handling, multimodal data fusion supporting both seismic and acoustic modalities, and comprehensive logging of classification results with confidence scores, ground truth comparison, and latency metrics.

Concrete classifiers must implement two essential methods: load_model() which loads the classification model from a configuration file and sets the supported modalities, and infer() which performs inference on buffered sensor data and returns classification probabilities. The base classifier calls load_model() during initialization and periodically invokes infer() with accumulated multimodal sensor samples when sufficient data is available.

The VibroFM class provides a concrete implementation of the abstract load_model() and infer() methods, building on our foundation model work ([Kimura2024], [Liu2023], [Kara2024]). For details on the expected input format, refer to the infer() docstring. Pre-trained and fine-tuned model weights from [Kimura2024] and [Kara2024], compatible with VibroFM, are available on the GitHub release page.

Implement A New Classifier

You can clone the repository wherever you prefer — in this guide, we’ll use /ws/acies as an example. Assume the repository is set up as follows:

$ cd /ws/acies/
/ws/acies$ git clone git@github.com:acies-os/vehicle-classifier.git
/ws/acies$ cd vehicle-classifier
/ws/acies/vehicle-classifier$ uv sync

To create a new classifier, follow these steps:

1. Copy the example template to create your new classifier file:

vehicle-classifier$ cp src/acies/vehicle_classifier/example.py src/acies/vehicle_classifier/my_classifier.py

2. Implement the required methods in your new classifier file:

   • load_model(): Load your model and set self.modalities

   • infer(): Perform inference on sensor data and return class probabilities

3. Add an entry to pyproject.toml under [project.scripts]:

[project.scripts]
my-clf = "acies.vehicle_classifier.my_classifier:main"

4. Run your new classifier using uv:

vehicle-classifier$ uv run my-clf --weight /path/to/config --namespace ns1 --proc-name clf1

API

Content

• API Reference
  • acies.vehicle_classifier.base
    • Classifier
    • ensemble()
    • get_twin_topic()
    • soft_vote()
    • time_diff_decorator()
  • acies.vehicle_classifier.vfm
    • VibroFM
  • acies.vehicle_classifier.buffer
    • StreamBuffer
    • TemporalEnsembleBuff

References

[Kimura2024] (1,2)

Kimura, Tomoyoshi, Jinyang Li, Tianshi Wang, Yizhuo Chen, Ruijie Wang, Denizhan Kara, Maggie Wigness et al. “Vibrofm: Towards micro foundation models for robust multimodal iot sensing.” In 2024 IEEE 21st International Conference on Mobile Ad-Hoc and Smart Systems (MASS), pp. 10-18. IEEE, 2024.

[Liu2023]

Liu, Shengzhong, Tomoyoshi Kimura, Dongxin Liu, Ruijie Wang, Jinyang Li, Suhas Diggavi, Mani Srivastava, and Tarek Abdelzaher. “Focal: Contrastive learning for multimodal time-series sensing signals in factorized orthogonal latent space.” Advances in Neural Information Processing Systems 36 (2023): 47309-47338.

[Kara2024] (1,2)

Kara, Denizhan, Tomoyoshi Kimura, Yatong Chen, Jinyang Li, Ruijie Wang, Yizhuo Chen, Tianshi Wang, Shengzhong Liu, and Tarek Abdelzaher. “Phymask: An adaptive masking paradigm for efficient self-supervised learning in iot.” In Proceedings of the 22nd ACM conference on embedded networked sensor systems, pp. 97-111. 2024.