In the coming Industry 4.0 era, IoT (Internet of Things) technology plays a more and more critical role. Anomaly detection is one of the essential applications for ensuring safe manufacturing, living, etc. It equips the machine with intelligence for perceiving its situation in real-time to prevent disastrous breakdown. Such monitoring systems were usually based on commercial inertial measurement units (IMU) and artificial intelligence (AI) algorithms that run on resource-rich, power-hungry servers, causing a certain amount of energy for sensing and data transmitting. In this paper, we introduce a novel self-powered sensing system with tiny machine learning (TinyML) technique for anomaly detection. A lightweight piezoelectric self-powered sensor (SPS) is utilized to substitute the IMU. The system runs on a low-cost embedded system, realizing the low-power in-situ inferring. A rich dataset has been collected on a vibration platform and analyzed by six well-known machine-learning models. A compressed deep neuron network (DNN) with three hidden layers achieves an accuracy of 97.6% given only 8-point normalized SPS data. The TinyML model is then deployed on embedded systems for on-device inferring and condition-based monitoring. Power measurement is conducted to compare the systems based on an IMU and an SPS. It has shown that the proposed self-powered sensing approach can save up to 66.74% of energy. The system provides a valuable reference for realizing pervasive sensing and ubiquitous AI.