user09 Posted July 9 Share Posted July 9 Hello everybody, As a computer science student, I would like to develop a system supporting learning to play piano. In particular, the system is going to simulate hands motions for a specified music piece. Now, I am at the starting point and trying to find an appropriate stack of technologies and the path. I am looking at Unigine as a 3D simulator. I see two approaches: 1. A specified for piano MIDI track allows to play with 100% accuracy - reference MIDI track; Create a hands model that follows physical parameters and limitations of real human hands (such as dimensions, weights and inertia, joint mobility, and muscle forces); Create a piano keyboard model that follows physical parameters and limitations of a real piano keyboard that generates MIDI sounds depending on a pressed key and the velocity; Create an algorithm (function) that assesses the quality of the MIDI track generated with the simulator based on the reference MIDI track; Train a ML model for movement of the hands model. 2. Create the same hands and keyboard models and assessment algorithm; Train a ML model for movement of the hands model based on videos of real pianists. I would appreciate if someone gives me his/her own opinion on my vision or propose a different approach to solve the problem. Link to comment
Recommended Posts