commit d72f5e1c35813f0242c94236c363d707c369cf63
parent db48a70094d78c4d6aba514157bc2679d41fd7ae
Author: Keith Bloemer <32459398+GuitarML@users.noreply.github.com>
Date: Fri, 4 Dec 2020 13:27:04 -0600
Update README.md
Diffstat:
| M | README.md | | | 60 | ++++++++++++++++++++++++++++++++++++++++++++++-------------- |
1 file changed, 46 insertions(+), 14 deletions(-)
diff --git a/README.md b/README.md
@@ -1,9 +1,16 @@
# GuitarLSTM
GuitarLSTM trains guitar effect/amp neural network models for processing
-on wav data. In comparison to the WaveNet model from PedalNetRT, this
-implementation is much faster and better suited for copying the sound
-of guitar amps and pedals.
+on wav files. Record input/output samples from the target guitar amplifier or
+pedal, then use this code to create a digital deep learning model of the
+sound. The model can then be applied to other wav files to make it sound
+like the amp or effect. This code uses Tensorflow/Keras.
+
+The LSTM model is effective for copying the sound of tube amplifiers, distortion,
+overdrive, and compression. It also captures the impluse response of the mic/cab
+used for recording the samples. In comparison to the WaveNet model, this
+implementation is much faster and can more accurately copy the sound of
+complex guitar signals while still training on a CPU.
## Info
@@ -25,18 +32,37 @@ https://colah.github.io/posts/2015-08-Understanding-LSTMs/
## Usage
-**Run effect on .wav file**:
+**Train model and run effect on .wav file**:
Must be single channel, 44.1 kHz, FP32 wav data (not int16)
```bash
-# This will preprocess the input data, perform training, and generate test wavs and analysis plots.
-# Output will be saved to "models/out_model_name" folder.
+# Preprocess the input data, perform training, and generate test wavs and analysis plots.
+# Specify input wav file, output wav file, and desired model name.
+# Output will be saved to "models/out_model_name/" folder.
+
python train.py data\ts9_test1_in_FP32.wav data\ts9_test1_out_FP32.wav out_model_name
+
# Run prediction on target wav file
# Specify input file, desired output file, and model path
predict.py data\ts9_test1_in_FP32.wav output models\ts9_model.h5
```
+**Training parameters**:
+
+```bash
+# Use these arguments with train.py to further customize the model:
+
+--training_mode=0 # enter 0, 1, or 2 for speed tranining, accuracy traninig, or extended training, respectively
+--input_size=150 # sets the number of previous samples to consider for each output sample of audio
+--max_epochs=1 # sets the number of epochs to train for; intended to be increased dramatically for extended training
+--batch_size=4096 # sets the batch size of data for training
+
+# Edit the "TRAINING MODE" or "Create Sequential Model" section of train.py to further customize each layer of the neural network.
+```
+
+**Colab Notebook**:
+Use Google Colab notebook (guitar_lstm_colab.ipynb) for training
+GuitarLSTM models in the cloud. See notebook comments for instructions.
## Training Info
@@ -45,7 +71,8 @@ Helpful tips on training models:
chords, individual notes, and playing techniques to get a full spectrum
of data for the model to "learn" from.
2. A buffer splitter was used with pedals to obtain a pure guitar signal
- and post effect signal.
+ and post amp/effect signal. You can also use a feedback loop from your
+ audio interface to record input/output simultaneously.
3. Obtaining sample data from an amp can be done by splitting off the original
signal, with the post amp signal coming from a microphone (I used a SM57).
Keep in mind that this captures the dynamic response of the mic and cabinet.
@@ -56,9 +83,19 @@ Helpful tips on training models:
5. Requires float32 .wav files for training (as opposed to int16).
-## Future Work
+## Limitations and future work
+
+This implementation of the LSTM model uses a high amount of
+RAM to preprocess wav data. If you experience crashes due to
+limited memory, reduce the "input_size" parameter by using
+the "--input_size=" flag with train.py. The default setting is 150.
+Increasing this setting will improve training accuraccy, but size of
+the preprocessed wav data will increase as well.
+
+Adding a custom dataloader would reduce RAM usage at the cost of
+training speed, and will be a focus of future work.
-A real time implementation for use in a guitar plugin is
+A real-time implementation for use in a guitar plugin is
currenty in work. This would theoretically perform much faster
(less cpu usage) than the previous WaveNet model. If you want to
use deep learning models through a real time guitar plugin,
@@ -69,7 +106,3 @@ https://github.com/GuitarML/PedalNetRT<br>
SmartGuitarAmp<br>
https://github.com/GuitarML/SmartGuitarAmp<br>
-
-
-Note: See Google Colab notebook (guitar_lstm_colab.ipynb) for training
- GuitarLSTM models in the cloud. See notebook comments for instructions.
-\ No newline at end of file
