Simulator for learning to play piano

[user09]

Hello everybody,

As a computer science student, I would like to develop a system supporting learning to play piano. In particular, the system is going to simulate hands motions for a specified music piece.

Now, I am at the starting point and trying to find an appropriate stack of technologies and the path. I am looking at Unigine as a 3D simulator.

I see two approaches:

1.

• A specified for piano MIDI track allows to play with 100% accuracy - reference MIDI track;
• Create a hands model that follows physical parameters and limitations of real human hands (such as dimensions, weights and inertia, joint mobility, and muscle forces);
• Create a piano keyboard model that follows physical parameters and limitations of a real piano keyboard that generates MIDI sounds depending on a pressed key and the velocity;
• Create an algorithm (function) that assesses the quality of the MIDI track generated with the simulator based on the reference MIDI track;
• Train a ML model for movement of the hands model.

2.

• Create the same hands and keyboard models and assessment algorithm;
• Train a ML model for movement of the hands model based on videos of real pianists.

 

I would appreciate if someone gives me his/her own opinion on my vision or propose a different approach to solve the problem.