Welcome to the Symbolic Creative Hierarchical music Unmasking Bidirectional Encoder Representation Transformer!

This application should only serve as a demo, and was coded in very little time. Use a Desktop-Browser and expect bugs!

You can always switch on and off this tutorial with the tutorial switch, that is currently highlighted.

The main part of this application consists of two pianoroll views, that are currently empty. At the top of the pianorolls, you can see tab controls. If the two pianoroll-views do not appear next to each other, try adjusting your zoom level until they do. Musical pieces generated with SCHmUBERT have a fixed length of 64 bars. The pianoroll views are scrollable (horizontally). In this application, you have 8 slots for musical pieces, the tab control selects, which piece is displayed at the left/right pianoroll.

Click the inpaint button to generate your first piece of symbolic music with SCHmUBERT!

Now the piece is being generated by SCHmUBERT. Usually this should take only a few seconds, depending on the current number of users. You are welcome to have a look at our paper in the meantime.

Very well, now you generated your first piece of symbolic music with SCHmUBERT! To listen to it, click the Play button.

You can click anywhere in the pianoroll, to skip to that position. After you finished listening (stop button is next to the play button), you might have identified a position you didn't particularly like (probably the whole piece?). You can select a part of the generated piece (draw a rectangle with your mouse) and delete it with the mask button.

SCHmUBERT can now re-generate the notes you masked out. To make the best use of this UI, click on the direction button once to make the model use the right hand side as source piece, and the left hand side as destination. Then click the inpaint button again to start the re-generation process.

Now you know the basic interactions with the model. Use the two views to compare different versions of the same piece. The slider on top lets you select the number of timesteps the model uses to generate your pieces (the more steps, the longer it will take, but the quality should be better). The copy button under the inpaint button copies a piece in the selected direction, but it will only overwrite gaps in your destination piece. There are some buttons we did not mention previously: next to the stop button there is an undo button (bet you know what it does), and next to it, there is a button which lets you download the current pianoroll as a MIDI file. You can change the instruments which play the melody and the bass voice (dropdowns).
In the line below, there is a shortcut to select the whole piece, just the melody, bass, drums (choose with dropdown., click Select button). Finally, you can transpose your current selection with the arrow up/down buttons, and shift it backwards/forward in time with the left/right buttons. You can also move the notes you selected with drag and drop.

One last feature we did not yet mention: you can upload and extend your own MIDI files! If you don't have one at hand right now, you can use this one (mario) or this one (imperial) in the meantime (best download them both now). Just use the upload button. If you do now want to overwrite your pieces, choose an empty slot in the left pianoroll tab control.

Now you are ready to play around with SCHmUBERT! You could try doing an interpolation between two known pieces (use upload, time-shift and copy to prepare a piece with a gap for interpolation in between)
Remember to turn off the tutorial if you don't want to see it again.
Good luck, have fun!