GuitarLSTM

Deep learning models for guitar amp/pedal emulation using LSTM with Keras
Log | Files | Refs | README

commit d72f5e1c35813f0242c94236c363d707c369cf63
parent db48a70094d78c4d6aba514157bc2679d41fd7ae
Author: Keith Bloemer <32459398+GuitarML@users.noreply.github.com>
Date:   Fri,  4 Dec 2020 13:27:04 -0600

Update README.md
Diffstat:
MREADME.md | 60++++++++++++++++++++++++++++++++++++++++++++++--------------
1 file changed, 46 insertions(+), 14 deletions(-)

diff --git a/README.md b/README.md @@ -1,9 +1,16 @@ # GuitarLSTM GuitarLSTM trains guitar effect/amp neural network models for processing -on wav data. In comparison to the WaveNet model from PedalNetRT, this -implementation is much faster and better suited for copying the sound -of guitar amps and pedals. +on wav files. Record input/output samples from the target guitar amplifier or +pedal, then use this code to create a digital deep learning model of the +sound. The model can then be applied to other wav files to make it sound +like the amp or effect. This code uses Tensorflow/Keras. + +The LSTM model is effective for copying the sound of tube amplifiers, distortion, +overdrive, and compression. It also captures the impluse response of the mic/cab +used for recording the samples. In comparison to the WaveNet model, this +implementation is much faster and can more accurately copy the sound of +complex guitar signals while still training on a CPU. ## Info @@ -25,18 +32,37 @@ https://colah.github.io/posts/2015-08-Understanding-LSTMs/ ## Usage -**Run effect on .wav file**: +**Train model and run effect on .wav file**: Must be single channel, 44.1 kHz, FP32 wav data (not int16) ```bash -# This will preprocess the input data, perform training, and generate test wavs and analysis plots. -# Output will be saved to "models/out_model_name" folder. +# Preprocess the input data, perform training, and generate test wavs and analysis plots. +# Specify input wav file, output wav file, and desired model name. +# Output will be saved to "models/out_model_name/" folder. + python train.py data\ts9_test1_in_FP32.wav data\ts9_test1_out_FP32.wav out_model_name + # Run prediction on target wav file # Specify input file, desired output file, and model path predict.py data\ts9_test1_in_FP32.wav output models\ts9_model.h5 ``` +**Training parameters**: + +```bash +# Use these arguments with train.py to further customize the model: + +--training_mode=0 # enter 0, 1, or 2 for speed tranining, accuracy traninig, or extended training, respectively +--input_size=150 # sets the number of previous samples to consider for each output sample of audio +--max_epochs=1 # sets the number of epochs to train for; intended to be increased dramatically for extended training +--batch_size=4096 # sets the batch size of data for training + +# Edit the "TRAINING MODE" or "Create Sequential Model" section of train.py to further customize each layer of the neural network. +``` + +**Colab Notebook**: +Use Google Colab notebook (guitar_lstm_colab.ipynb) for training +GuitarLSTM models in the cloud. See notebook comments for instructions. ## Training Info @@ -45,7 +71,8 @@ Helpful tips on training models: chords, individual notes, and playing techniques to get a full spectrum of data for the model to "learn" from. 2. A buffer splitter was used with pedals to obtain a pure guitar signal - and post effect signal. + and post amp/effect signal. You can also use a feedback loop from your + audio interface to record input/output simultaneously. 3. Obtaining sample data from an amp can be done by splitting off the original signal, with the post amp signal coming from a microphone (I used a SM57). Keep in mind that this captures the dynamic response of the mic and cabinet. @@ -56,9 +83,19 @@ Helpful tips on training models: 5. Requires float32 .wav files for training (as opposed to int16). -## Future Work +## Limitations and future work + +This implementation of the LSTM model uses a high amount of +RAM to preprocess wav data. If you experience crashes due to +limited memory, reduce the "input_size" parameter by using +the "--input_size=" flag with train.py. The default setting is 150. +Increasing this setting will improve training accuraccy, but size of +the preprocessed wav data will increase as well. + +Adding a custom dataloader would reduce RAM usage at the cost of +training speed, and will be a focus of future work. -A real time implementation for use in a guitar plugin is +A real-time implementation for use in a guitar plugin is currenty in work. This would theoretically perform much faster (less cpu usage) than the previous WaveNet model. If you want to use deep learning models through a real time guitar plugin, @@ -69,7 +106,3 @@ https://github.com/GuitarML/PedalNetRT<br> SmartGuitarAmp<br> https://github.com/GuitarML/SmartGuitarAmp<br> - - -Note: See Google Colab notebook (guitar_lstm_colab.ipynb) for training - GuitarLSTM models in the cloud. See notebook comments for instructions. -\ No newline at end of file